12. The Data Cycle System

SOFIA proposals and observation data are managed by the SOFIA Data Cycle System (DCS). The DCS is designed primarily to support Science and Mission Operations activities associated with Guest Investigator (GI) programs for the observatory. From a GI point of view, observatory data flow is cyclic in nature (see Figure 12-1). The GI first develops and submits a proposal to the SOFIA Science Mission Operations (SMO). Once approved, the proposal becomes an observing plan that the GI uses to create and update astronomical observation requests (AORs) that are used in both flight planning and on-aircraft execution. After execution of the observations in-flight, the resulting data are archived and, for Facility Science Instruments (FSIs), automatically processed (pipelined) to produce the final data products, which are then served back to the original GI via a simple user-interface -- thus completing the cycle begun by the original proposal and perhaps driving new proposals. The data products are made available to the astronomical community for archival research purposes after an appropriate proprietary period.

SOFIA data cycle

Figure 12-1: Cartoon of the SOFIA Data Cycle.

The goal of the SOFIA DCS is to maximize the scientific productivity and efficiency of the observatory by providing a suite of easy-to-use tools and infrastructure that are integrated with each other as well as other applications (e.g. Flight Management Infrastructure) at each step in the data cycle.

The DCS Archive is the permanent repository for all raw scientific and housekeeping data accumulated during SOFIA flights and any pipeline products produced thereafter (for example, pipeline reduced data for FSIs). Access to the SOFIA Data Archive is via the DCS web pages. GIs are able to retrieve their data from the archive during the proprietary period using an assigned DCS login username and password. The proprietary period for Cycle 3 will be 1-year from the date of ingestion into the DCS Archive. Once the proprietary period expires, the data becomes public and can be accessed by anyone in the general community.

In order to make use of most of the features of the DCS website, users must register by clicking on the ''Register with DCS'' link on the DCS home page. After the registration form has been submitted, the user will be notified by e-mail when the account has been created. (Users should verify that mail from ''sofia.usra.edu'' is not blocked by any spam filters.) After logging in, a registered user can change his/her profile and password at any time using the View Profile link displayed next to the login name at the top of the DCS pages.

The DCS includes a suite of software tools to archive data and to assist GIs in preparing and submitting proposals:

  • SOFIA Proposal Tool (SPT): SOFIA proposals are created and modified using the SPT, a stand-alone Java application available for a wide variety of platforms. The Download SPT page'' includes links to download the Java application and installation instructions for all supported platforms. Complete help files are included with the application distribution and can be accessed via the Help menu options with the SPT.
  • Time Estimators:In order to assist in the creation of SOFIA observing proposals, the DCS web site provides an online SOFIA Instrument Time Estimator that calculates the total observation time required or signal-to-noise achieved for FLITECAM and FORCAST imaging modes. The DCS also maintains the on-line FLITECAM Grism Exposure Time Calculator and the online FORCAST Grism Exposure Time Calculator. Exposure time calculators for GREAT and EXES are also available, and links may be found on the SITE page.
  • Atmospheric Transmission Estimator (ATRAN): To allow proposers to estimate the effects of atmospheric transmission on potential observations and exposure times with SOFIA instruments, the DCS provides access to a web-based version of the ATRAN program written by Steve Lord. For a specified observing altitude and wavelength range, the web page will run ATRAN and generate an output file and plot of the atmospheric transmission as a function of wavelength. Both the output file and plot can be downloaded to the user's desktop. A brief description of the input parameters and how to run ATRAN is given at the top of the web page.
  • Visibility Tool (VT): The Target Visibility Tool provides the capability to estimate what date, time, and aircraft heading are required to observe an astronomical target. The tool is a Java applet that is accessible on the Visibility Tool page. Details of which operating systems and browsers are compatible with VT are provided on the Visibility Tool page. A stand-alone version of VT is available for download.

    To find the visibility of a given target, the user enters its co-ordinates (J2000 RA and DEC), the estimated latitude and longitude of the observation (which defaults to the SOFIA take-off location of Palmdale, CA), the start date and time, and the duration of interest. The application plots the target elevation for the period of interest. A target is visible to SOFIA if the elevation is in the range 20° to 60°.

    The nominal period of observations for Cycle 3 is 1 March, 2015 to 31 January, 2016. GIs should use VT to ensure that the proposed targets are in the allowed elevation range for visibility. For flights that take off from and return to Palmdale, the southernmost observable declination is -36\arcdeg. However, there will be up to two Southern Hemisphere Deployments during Cycle 3.

    Instructions on how to use VT and help on specific topics are available as part of the tool, using either the ''Help'' pull-down menu or the ''?'' button, both at the top, left of the applet window.

In addition, the DCS provides users with a variety of search options for proposals, data, and observing plans.

  • Proposal Search: All proposals submitted via SPT are stored in the DCS observation planning database and can be accessed from the DCS web pages. Users can retrieve summary data for available proposals by using the Proposal Search page and entering the relevant search criteria for the proposal(s) of interest. Summary information will be displayed for each proposal (appropriate to the permissions of the user).
  • Archive Search: Summary information is available for all public data in the DCS archive. From the Science Archive Search page, MCS Archive Search page, or the Ancillary Files Search page, users can specify search criteria to identify data of interest. Summary info for resulting matches is shown in a table with appropriate links based upon user permissions. Data that are still proprietary are available only to the GIs on the associated observing plan. Hence, \textit{users must log in using a DCS username to view any proprietary data}. Proprietary data can only be seen by the program GIs and selected SMO staff members. From the Summary Information table for science data, users can click through to a list of all data products associated with a particular observation (e.g., raw or pipelined data), or identify data to be downloaded.
  • Observing Plans Search: Once a proposal is approved, it becomes an observing plan, which can be accessed and modified as needed to support science and mission requirements by the proposal GI and selected SMO staff members. Observing plans can be accessed from the planning database using the Search Observing Plans link on the DCS web page. Summary information will be displayed for each observing plan along with links to the Observing Plan Details page. The Observing Plan Details page provides all the information available for a plan and corresponding links to the editor functions to make changes (for users with appropriate permissions).
  • Data Retrieval: Data can be retrieved by identifying desired data in the summary table and then clicking the ''Retrieve Data'' button. In order to retrieve any data, the user must be logged in. Only GIs with appropriate permissions and SMO staff are able to retrieve data that is still within the proprietary period. Once designated for retrieval, the files are staged to a public FTP server and a direct link is e-mailed to the user. All science data products are available in their original format (e.g., FITS). The same retrieval process is used for the MCCS and Ancillary data files.

12.1 Data Rights and Availability

All scientific data from SOFIA observations will be distributed via the SOFIA Data Cycle System's Science Archive. All data will be archived as Level 1 data (raw). Where appropriate, Level 2 (corrected for instrumental and atmospheric effects) and Level 3 (flux calibrated) will also be archived. These data will be accessible to the general community after a proprietary period of twelve months, starting at the ingestion of the calibrated data into the archive.

In order to query the archive and retrieve data, it is necessary to register with the DCS. This can be done by following the appropriate link ("Register With DCS") on the main DCS page. Registration is not required for using the proposal preparation and submission tools.