Data Analysis and Processing Software
--by Keith Arnaud, GSFC
IntroductionThe basis of the ASCA processing and analysis software system is the use of FITS format files to store all data and calibration information. The first step in the processing is to convert the telemetry data (the so-called FRF) into FITS format. The following three classes of files are created:
- science data
- For most observing modes, this consists of a list of events with attributes
(detector position, time, etc).
- housekeeping data
- All the instrument and spacecraft housekeeping stored in a compressed
- auxiliary data
- Parameters based on orbit and attitude information, e.g. the angle between
the satellite pointing direction and the bright limb of the Earth. This file
also includes selected housekeeping parameters in an uncompressed format for
ease of use.
Further processing and analysis of these files can then be performed using a suite of programs (FTOOLS) written to manipulate FITS data structures. These FTOOLS can run either within IRAF or as stand-alone programs from the operating system. The initial processing of data at GSFC uses FTOOLS run from Unix shell scripts to perform a number of tasks that GOs probably will not want to duplicate. However, individual GOs will want to do some processing and reduction of their data. Since using the FTOOLS requires more knowledge of the detailed structure of the data files than most users will wish to acquire, there is a higher level, interactive, command-driven program (XSELECT) that performs the data reduction functions commonly required. The output from XSELECT then can be used in analysis programs such as those provided by XANADU, IRAF/PROS, or IDL.
- Science data
- The science file contains the science data and the time intervals over
which the data were accumulated. For most observation modes, the science data
consists of a list of events ordered by time. Each event also is labelled by
detector and sky positions, the spectrometer channel, and instrument-specific \parameters. More adventurous observers may have data in SIS FAST mode, where
the Y position is a time coordinate, or GIS MPC mode, where the science data
consist of a series of spectra. The standard science data file follows the
design of the ROSAT Rationalized FITS file.
- Housekeeping data
- The housekeeping file contains the housekeeping parameters stored as a
time-ordered list with each parameter only appearing when its value changes.
An FTOOL can be used to extract from this file the more conventional table
showing the parameter value against time. In practice, very little use has been
found for the housekeeping parameters apart from the small number that are
also included in the auxiliary file. The housekeeping file is distributed for
completeness but most GOs will not want to keep it on disk.
- Auxiliary data
- The auxiliary file contains tabular data giving various parameters against
time. Useful quantities included are the SAA flag, the angle between the
pointing direction and the limb of the bright Earth, the angle between the
pointing direction and the Sun, the pointing stability, the cut-off rigidity,
and the important instrument housekeeping parameters. This file is used to
select the good data times.
Initial ProcessingSome initial data processing is performed at GSFC on all GO datasets. All software used is available for GOs to run at their own institutions if they wish, indeed, most of the programs used are simply FTOOLS. The processing performed is as follows:
- The FRF telemetry file is converted to FITS format science and housekeeping
files. A new science file is started each time there is a significant change
of observation mode.
- The auxiliary file is created.
- The detector and sky positions for each event are calculated based on
calibration data and the attitude history for the observation. The same FTOOL
(ASCALIN) also calculates the PI channel for GIS events by correcting for the
temporal and spatial gain variations.
- A short automatic analysis sequence is run to generate "quick-look" plots.
Data CleaningGOs will have to clean their science data before going on to create the higher level analysis products such as images, light curves, and spectra. This data selection may be done using the program XSELECT and is described in detail in the GOF document An abc of ASCA Data Reduction. We summarize the important features below.
- GIS data selection
- The most important GIS data to reject are those falling outside the central
20 arcminute radius region. This outer ring is poorly calibrated and has a
larger (and variable) background. The section of the image containing the
inflight calibration source also should be excluded. Both these rejections can
be performed in XSELECT using a spatial region filter to produce a new science
data file containing good events. It is also a good idea to reject data close
to the Earth's limb.
- SIS data selection
- Bright Earth contamination is the worst problem with SIS data, so a
selection criterion should be applied based on the angle between the pointing
direction and the bright limb of the Earth. Once the bright Earth contamination
is removed then the CCD "hot" and "flickering" pixels can be identified and
extracted. A command in XSELECT runs an FTOOL that performs this task by
identifying pixels whose count rate is not consistent with those around them.
CCD Hysteresis effects also make it advisable to reject the first few frames
of data immediately after SAA and any transitions between day and night. This
rejection also can be made in a simple way using XSELECT.
Creating Data ProductsOnce a clean set of science data has been created the next step is to create products for use by analysis software. The basic product is an event list. This product can be read into IRAF/PROS or XIMAGE, just like the ROSAT event files. The only caveat on this statement is that SIS data in FAINT mode do not have a single PHA but have instead nine PHA values, one for each of the pixels in the 3x3 event box. An FTOOL called FAINT converts this FAINT mode data into a form with only one PHA per event and this file then can be used. The FAINT task can be run from within XSELECT in a transparent manner. XSELECT also includes a command to accumulate images, light curves or spectra from the clean event files and then to save this accumulation to standard FITS format files. A large range of selection criteria can be used when accumulating these products. For instance, an image can be accumulated by XSELECT then displayed using SAOimage. Regions can be selected and used as filters in XSELECT to create spectra and light curves from the selected part of the image. Analogously, a light curve can be created and displayed. Time ranges can be selected using the cursor and these time ranges used in the creation of other images, light curves or spectra.
Calibration-derived ProductsScientific analysis of the data products requires a number of files that depend on instrument calibrations. These are supplied through a combination of FTOOLS and a calibration database accessible by anonymous ftp. Spectral response files are available from the calibration database for the standard situations and FTOOLS are under developement to produce the response in more complicated circumstances. An FTOOL to create exposure maps is also under developement. We expect that these FTOOLS will continue evolving for the next few months, as the instruments become better understood.
Data Analysis SoftwareNo high-level data analysis software has been written specifically for ASCA, since several multi-mission systems already exist. The following systems are those which have been tested with ASCA data:
- The programs XSPEC, XIMAGE, and XRONOS have been used extensively by the
ASCA team, and can read all the products written by XSELECT.
- The science event file can be read into PROS and imaging and timing
analysis performed. At present spectral analysis of ASCA data is not available.
- A number of ASCA team members have been using IDL for data analysis using
the available astronomy libraries and their own tasks.
Return to the Table of Contents Proceed to the next article