8. The Data Cycle System

v8.1.0

Table of Contents

Return to the Table of Contents for this section at any time by selecting Return to Table of Contents. Users may also navigate through the entire Observer's Handbook by using the complete Table of Contents menu to the right.

8.   The Data Cycle System

SOFIA proposals and observation data are managed by the SOFIA Data Cycle System (DCS). The DCS is designed primarily to support Science and Mission Operations activities associated with proposal programs for the observatory. Observatory data flow is cyclic in nature (see Figure 8-1). First a proposal must be developed and submited to the SOFIA Science Mission Operations (SMO). Once approved, the proposal becomes an observing plan that the proposer uses to create and update astronomical observation requests (AORs) that are used in both flight planning and on-aircraft execution. After execution of the observations in-flight, the resulting data are archived and, for Facility Science Instruments (FSIs), automatically processed (pipelined) to produce the final data products, which are then served back to the original proposer via a simple user-interface–thus completing the cycle begun by the original proposal and perhaps driving new proposals. The data products are made available to the astronomical community for archival research purposes after an appropriate proprietary period.

Figure 8-1.
Diagram showing SOFIA data cycle system

Figure 8-1. Cartoon of the SOFIA Data Cycle.

The goal of the SOFIA DCS is to maximize the scientific productivity and efficiency of the observatory by providing a suite of easy-to-use tools and infrastructure that are integrated with each other as well as other applications (e.g. Flight Management Infrastructure) at each step in the data cycle.

The DCS Archive is the permanent repository for all raw scientific and housekeeping data accumulated during SOFIA flights and any pipeline products produced thereafter (for example, pipeline reduced data for FSIs). Access to the SOFIA Data Archive is via the DCS web pages. Data can be retreived from the archive during the proprietary period using an assigned DCS login username and password. The proprietary period for Cycle 8 will be 1-year from the date of ingestion into the DCS Archive. Once the proprietary period expires, the data becomes public and can be accessed by anyone in the general community.

In order to make use of most of the features of the DCS website, users must register by clicking on the Register with DCS link on the DCS home page. After the registration form has been submitted, the user will be notified by e-mail when the account has been created. (Users should verify that mail from ''sofia.usra.edu'' is not blocked by any spam filters.) After logging in, a registered user can change his/her profile and password at any time using the View Profile link displayed next to the login name at the top of the DCS pages.

The DCS includes a suite of software tools to archive data and to assist in the preparion and submission of proposals, including USPOTSITE, ATRAN, and VT (see Section 8.3.1).

In addition, the DCS provides users with a variety of search options for proposals, data, and observing plans.

Proposal Search
SOFIA proposals are created and modified using the Unified SOFIA Proposal Tool (USPOT) and are stored in the DCS observation planning database. Users can retrieve summary data for available proposals by using the Proposal Search page and entering the relevant search criteria for the proposal(s) of interest. Summary information will be displayed for each proposal (appropriate to the permissions of the user).
 
Installation instructions for all supported platforms are provided on the USPOT download webpage. The USPOT Manual contains instrument-specific instructions for submitting proposals. Additional help files are included with the USPOT application distribution.
 
Archive Search
Summary information is available for all public data in the DCS archive. See Section 8.1 below for more details.
 
Observing Plans Search
Once a proposal is approved, it becomes an observing plan, which can be accessed and modified as needed to support science and mission requirements by the proposer and selected SMO staff members. Observing plans can be accessed from the planning database using the Search Observing Plans link on the DCS web page. Summary information will be displayed for each observing plan along with links to the Observing Plan Details page. The Observing Plan Details page provides all the information available for a plan and corresponding links to the editor functions to make changes (for users with appropriate permissions).
 
Data Retrieval
Data can be retrieved by identifying desired data in the summary table and then clicking the Retrieve Data button. In order to retrieve any data, the user must be logged in. Only those with appropriate permissions and SMO staff are able to retrieve data that is still within the proprietary period. Once designated for retrieval, the files are staged to a public FTP server and a direct link is e-mailed to the user. All science data products are available in their original format (e.g., FITS). The same retrieval process is used for the MCCS and Ancillary data files.
 

Return to Table of Contents.

8.1   Data Rights and Availability

The SOFIA Science Center provides raw and calibrated data for all instruments via the Data Cycle System (DCS) Science Archive. All data will be archived as Level 1 data (raw). Where appropriate, Level 2 data (corrected for instrumental and atmospheric effects), Level 3 data (flux calibrated), and Level 4 data (a combination of Level 3 files to produce maps, mosaics, etc) will also be archived. Data that are still proprietary are available only to those on the associated observing plan. These data will be accessible to the general community after a proprietary period of twelve months, starting at the ingestion of the calibrated data into the archive. A DCS account is required to search the archive and download data. Register for a DCS account here. From the Science Archive Search page, MCS Archive Search page, or the Ancillary Files Search page, users can specify search criteria to identify data of interest. From the Summary Information table for science data, users can click through to a list of all data products associated with a particular observation (e.g., raw or pipelined data), or identify data to be downloaded.

Return to Table of Contents.

8.2   Transition to IRSA Data Archive

OFIA is beginning to transition to using the Infrared Science Archive (IRSA) hosted by the Infrared Processing & Analysis Center (IPAC) as its primary data archive. The first release of the SOFIA Archive at IRSA includes FIFI-LS, FORCAST, and GREAT data from Cycle 4 and Cycle 5, and is currently accessible online. By the end of 2019, the second archive release will expand the database to include SOFIA data from all instruments and all cycles.

For tips on SOFIA specific archive searches and taking advantage of IRSA visualization features, video tutorials are availabe here

Return to Table of Contents.

8.3   Resources for Using SOFIA Data

8.3.1.   Data Documents & Tools

Data Documents

The Data Documents webpage contains helpful documentation for using SOFIA data. The Data Handbooks describe the data products, processing steps, calibration procedures, and known issues. In addition, the Cookbook Recipes provide guided examples of common data analysis objectives using SOFIA processed data. They are generally written for a graduate student audience and are intended to be used with the Data Handbooks.

Data Tools 

Data Cycle System (DCS)
SOFIA proposals and observation data are managed by the SOFIA Data Cycle System (DCS). The DCS is designed primarily to support Science and Mission Operations activities associated with Guest Observer (GO) programs for the observatory.

SOFIA Instrument Time Estimator (SITE)
SITE is an instrument-specific exposure time calculator that provides the total integration time or S/N for a given instrument, filter(s), source type (point, extended, emission line), and water vapor overburden. 

Atmospheric Transmission (ATRAN)
The atmospheric transmission as a function of wavelength may be obtained using the on-line Atmospheric Transmission tool (ATRAN), developed and provided to the SOFIA program by Steve Lord. The use of ATRAN is necessary for planning SOFIA high-resolution spectroscopic observations. For a specified observing altitude and wavelength range, ATRAN will generate a downloadable output file and plot of the atmospheric transmission as a function of wavelength.

Target Visibility Tool
The downloadable Target Visibility Tool (VT)—which is also now an integrated feature of USPOT—provides the capability to estimate what date, time, and aircraft heading are required to observe an astronomical target. A target is visible to SOFIA if the elevation is in the range 20° to 60°. For flights that take off from and return to Palmdale, the southernmost allowed declination is -36°. Southern targets will be scheduled for observation during the Southern Hemisphere Deployment that will take place during the southern winter of 2019. Proposers may use VT to check the periods of visibility for their targets, however, the use of VT is not a requirement because detailed flight planning is done by the SMO staff.

SOSPEX
The SOSPEX tool, written by Dario Fadda and Ed Chambers, allows users to explore the final data cubes produced by the data reduction pipeline. An overview of its use can be found on the AAS poster. (See also http://adsabs.harvard.edu/abs/2018AAS...23115011F

FLUXER
The FLUXER tool, written by Christof Iserlohe, allows users to fit the continuum and estimate line strengths in the final FIFI-LS data cubes.

8.3.2.  Citations for Publications using SOFIA Data

When writing publications, the Information for Authors webpage provides instructions for acknowledgements and citations. Additional references may be found on the SOFIA Publications webpage.

Return to Table of Contents.

Download the PDF Version

Share This Page