Data Pipelines

SOFIA's data processing pipelines are now available to the public. This public release will enable our user community to better understand SOFIA data reduction, to customize their own data reduction, and to provide feedback to the pipeline team. The pipeline software, sofia_redux, is available through SOFIA's GitHub repository.

The FORCAST, FIFI-LS, and FLITECAM pipelines are included in the current pipeline release. The HAWC+ and EXES pipelines are being converted to python; after the conversions are complete, they will also be included in our public repository. 

Tutorials:

Below are step-by-step tutorials for running sofia_redux for FIFI-LS data, FORCAST Imaging and Spectroscopy data, and FLITECAM Imaging and Spectroscopy data. These tutorials include instructions for how to download the example data from IRSA and notes for each step of the pipeline. For instructions on how to install sofia_redux, please follow the link above to the GitHub repository.

ATRAN FITS Files:

For optimal telluric correction, FORCAST and FIFI-LS spectroscopic reductions require a library of FITS files, containing model atmospheric transmission spectra, derived from Steve Lord's ATRAN model

Two versions of the model libraries are available for each instrument:

  • FORCAST:
    • Models using a water vapor value based on a standard atmosphere, flight altitude, and telescope elevation (atran_forcast_standard.tgz, 531.3 MB; checksum)
    • Models that are calculated for every combination over a range of zenith water vapor values, telescope elevations, and flight altitudes (atran_forcast_wv.tgz, 37.5 GB; checksum). These files will be used if the 'Optimize ATRAN correction' option is selected when running the pipeline.
  • FIFI-LS:
    • Models using a water vapor value based on a standard atmosphere, flight altitude, and telescope elevation (atran_fifi-ls_standard.tgz, 144 MB; checksum)
    • Models that are calculated for every combination over a range of zenith water vapor values, telescope elevations, and flight altitudes (atran_fifi-ls_wv.tgz, 3.3 GB; checksum). These files will be used if the 'Use WV values' option is selected when running the pipeline.
  • FLITECAM: Models using a water vapor value based on a standard atmosphere, flight altitude, and telescope elevation (atran_flitecam_standard.tgz, 875.2 MB; checksum)

To verify that you've successfully downloaded any of the ATRAN model files, you can run an md5 checksum on the file and compare the results to the contents of the checksum text files provided above. After downloading and unpacking a library, its location can be provided to the pipeline as an optional parameter in the telluric correction step.

Pipeline Users Manuals:

Below is a table containing links to the User Manuals for all versions of the FIFI-LS and FORCAST pipelines. 

Pipeline User Manuals
Pipeline Versions User Manual Revision
FIFI-LS Redux 2.5.0 - 2.5.1 J (PDF)
FIFI-LS Redux 2.4.0 H (PDF)
FIFI-LS Redux 2.3.0 G (PDF)
FIFI-LS Redux 2.2.0 F (PDF)
FIFI-LS Redux 2.0.0 - 2.1.0 E (PDF)
FIFI-LS Redux 1.6.0 - 1.7.0 D (PDF)
FIFI-LS Redux 1.4.0 - 1.5.1 C (PDF)
FIFI-LS Redux 1.3.1 - 1.3.3 B (PDF)
FIFI-LS Redux 1.2.0 - 1.3.0 A (PDF)
FIFI-LS Redux 1.0.0 - 1.1.0 - (PDF)
FORCAST Redux 2.3.0 J (PDF)
FORCAST Redux 2.2.0 - 2.2.1 H (PDF)
FORCAST Redux 2.1.0 G (PDF)
FORCAST Redux 2.0.0 F (PDF)
FORCAST Redux 1.4.0 - 1.5.0 E (PDF)
FORCAST Redux 1.3.0 - 1.3.2 D (PDF)
FORCAST Redux 1.2.0 C (PDF)
FORCAST Redux 1.1.0 - 1.1.3 B (PDF)
FORCAST Redux 1.0.1 - 1.0.7 A (PDF)
FORCAST Redux 1.0.0 - (PDF)
FLITECAM Redux 2.0.0 B (PDF)
FLITECAM Redux 1.1.0 - 1.2.0 A (PDF)
FLITECAM Redux 1.0.0 - 1.0.3 - (PDF)

FAQs

1. Why doesn't the data I reduced match what is in the archive at IRSA?

If the data are not from one of the most recent flight series, it's likely that the data were processed with an older version of the pipeline. To check this, compare the value stored in the PIPEVERS keyword in the primary FITS header in your locally processed file to the one at IRSA (to see that value at IRSA without downloading the file you can click on the 'Show FITS Header' icon in the data tab). It's also possible that the data at IRSA were processed using different options than the data you processed yourself. The list of options used to process the data can be found in the HISTORY keyword in the primary FITS header.

2. What version of the pipline was used to process the data at IRSA?

The version of the pipeline used to process data is stored in the PIPEVERS keyword in the primary FITS header.

Share This Page