SOFIA Redux

Introduction

SOFIA Redux (sofia_redux) contains data processing pipelines and algorithms for instruments on the Stratospheric Observatory for Infrared Astronomy (SOFIA).

SOFIA raw and processed data can be accessed from the SOFIA archive. Archived data may not match the results of data processed with this pipeline software.

SOFIA pipelines are developed internally by the SOFIA Data Center software team, then are published publicly at the SOFIA Redux GitHub project. Contributions and feedback are welcome via the GitHub project.

Getting Started

Installation

Stable release

The sofia_redux package is available via PyPI:

pip install sofia-redux

From source

Obtain the source code for this package from the SOFIA Redux GitHub project, then install it from the source directory:

pip install .

or for development the package can be installed editable:

pip install -e .

Optional Requirements

PyQt6

If sofia_redux is installed via pip, the PyQt5 package, required for the pipeline GUI interface, is not automatically installed as a dependency. To use the GUI tools, install PyQt6 via pip:

pip install PyQt6

or use the [all] extra:

pip install 'sofia-redux[all]'

Please note that there may be some incompatibilities between some versions of PyQt6, some versions of the package dependencies, and some host OS versions.

DS9

Some optional visualization tools in the SOFIA Redux interface also use the ds9samp and regions packages to interface with the external SAOImage DS9 tool. To use these tools, install DS9, then install pyds9 and regions directly via pip:

pip install ds9samp regions

or using the [display] extra:

pip install 'sofia-redux[display]'

On MacOS, you will need to make a ds9 executable available in your PATH environment variable; see the DS9 FAQs for more information.

Reference data

Some pipeline modes require additional reference data for optimal data quality. These files are too large to distribute with the pipeline code, so they are provided separately.

Atmospheric models

For optimal telluric correction, FORCAST, FLITECAM, and FIFI-LS spectroscopic reductions require a library of FITS files, containing model atmospheric transmission spectra, derived from the ATRAN model.

Two versions of the model libraries are available for each instrument, except FLITECAM. ATRAN files parameterized by water vapor are not available for FLITECAM.

The EXES pipeline does not use ATRAN models for telluric correction, but it does attach a reference atmospheric model at a matching altitude and zenith angle to output spectral products, if available. The models used are derived from the Planetary Spectrum Generator (PSG).

  • FORCAST:

    • Approximate models, not accounting for water vapor variation

    • More accurate models, enabling programmatic optimization of the telluric correction

  • FIFI-LS:

    • Approximate models, not accounting for water vapor variation

    • More accurate models, enabling use of water vapor values recorded in the FITS headers for more accurate telluric correction

  • FLITECAM:

    • Approximate models, not accounting for water vapor variation

  • EXES:

    • Approximate models, not accounting for water vapor variation

After downloading and unpacking the library, its location can be provided to the pipeline as an optional parameter in the Calibrate Flux step for FORCAST or FLITECAM, the Telluric Correct step for FIFI-LS, or the Extract Spectra step for EXES.

Standard flux models

In addition to the ATRAN models, a library of standard flux models is required to reduce FORCAST or FLITECAM standard spectra to instrumental response curves. This should be rarely needed for standard scientific reductions, since reference response curves are provided for most data. If needed for re-deriving spectral flux calibrations, the standard model spectra are provided in the source distribution of this package, at sofia_redux/instruments/forcast/data/grism/standard_models or sofia_redux/instruments/flitecam/data/grism/standard_models.

FLITECAM and EXES auxiliary data

The default auxiliary calibration and reference data for FLITECAM and EXES reductions are too large to be included in the software packages provided via PyPI.

These files are provided in full in the source distribution of this package. Since they are required for most data reductions for these instruments, they may also be automatically downloaded as needed for non-source installations. Downloaded calibration files are cached for later use in a ‘.sofia_redux’ directory in the user’s home directory. For offline pipeline reductions, the source installation is recommended.

For FLITECAM, the data provided in this manner includes nonlinearity correction coefficients, spectroscopic order masks, and wavelength calibration files. For EXES, the large data files are bad pixel masks, reset dark files, and nonlinearity correction coefficients.

Troubleshooting

The last working set of installed versions of all dependencies is recorded in the requirements-min.txt file in this package. If errors are encountered in the other listed installation methods, it may be useful to install the frozen versions directly. For example, to install from source create a new Python environment from the sofia_redux package directory:

python -v venv .redux_troubleshoot_venv
source .redux_troubleshoot_venv/bin/activate
python -m pip install -r requirements-min.txt
python -m pip install -e .

Running SOFIA pipelines

Out of the box, Redux provides two primary command-line scripts:

  • redux (sofia_redux.pipeline.sofia.redux_app): An interactive graphical interface (GUI) for the SOFIA pipelines.

  • redux_pipe (sofia_redux.pipeline.sofia.redux_pipe): A command-line interface to the SOFIA pipelines.

Currently, the EXES, FIFI-LS, FLITECAM, FORCAST, and HAWC+ instruments are supported by these pipelines. See the user’s and developer’s manuals for each instrument for full descriptions of all supported observing modes and scientific algorithms.

In general, Redux works by reading in input data, deciding which reduction to run, then running a pre-defined set of reduction steps.

The GUI allows interactive parameter editing, and intermediate product display for each step. To begin a reduction, start the GUI by typing redux, then load in a set of data with the File->Open New Reduction menu.

The command-line interface allows fully automatic pipeline reductions. To begin a reduction with the automatic pipeline, type redux_pipe and provide the file names of the data to reduce on the command line. Non-default parameters can also be provided by specifying a configuration file in INI format. Type redux_pipe -h for a brief help message.

Tutorials

Tutorials are available for data reduction procedures, via the SOFIA website:

Manuals

User’s Manuals:

Developer’s Manuals:

Submodules