OGIP

OGIP¹ Calibration Memo CAL/GEN/91-001

The HEASARC Calibration Database
(a brief overview)

Ian M. George
Code 668,
NASA/GSFC,
Greenbelt, MD 20771

and

Michael F. Corcoran
Code 662,
NASA/GSFC,
Greenbelt, MD 20771

Version: November 22 2005

SUMMARY

The rôle of the HEASARC concerning the storage and documenting of the calibration data from past, current and future missions is briefly outlined.

LOG OF SIGNIFICANT CHANGES

Release	Sections Changed	Brief Notes
Date

1991 Oct 21		Original Version
1995 Jan 11	All	Made compatible with `LaTeX2HTML` software
1995 Feb 13	Figure 1	Made EPSF and included in text
2004 Apr 1	All	Made compatible with tth
2005 Nov 22	All	Text updated

1 INTRODUCTION
2 THE CALIBRATION DATAFLOW
3 CALIBRATION FILE TYPES
4 CALIBRATION UPDATES
5 CALIBRATION DATABASE STRUCTURE & SUBROUTINE LIBRARIES
6 DOCUMENTATION

1 INTRODUCTION

The availability of the instrument calibration and an understanding of its limitations is fundamental to the scientific analysis of data. In the past, calibration data has generally been made available in a somewhat ad hoc fashion. In most cases it has been a `black box' part of an analysis task, hidden from the user. There is usually little documentation available, nor any record of previous versions of the calibrations.

It is unlikely that the majority of HEASARC ² users will have an intimate understanding of all the detector details of all the instruments from which they obtain data. This is especially true for users of archival data. The transfer of the necessary information such that a user can make sensible judgements concerning their observation data and associated calibration measurements is therefore critical.

The HEASARC Calibration Database (CALDB) serves as an archive for calibration data needed to fully interpret observation data archived at the HEASARC. The CALDB serves the following purposes:

it provides a multi-mission, transportable means of storing and organizing calibration data;
it specifies formats for calibration data files using standard FITS conventions;
it specifies an interface to ingest new calibration data and to retrieve previously ingested calibration data;
it provides an archive for calibration documentation;
it allows users to determine unique, appropriate calibrations based solely on the specifications of the i strumental setup as documented in the observational datasets.

The remainder of this document describes the overall structure of the HEASARC'S CALDB.

2 THE CALIBRATION DATAFLOW

For a given high-energy mission, the CALDB is a joint effort between the instrument teams, the mission Guest Observer Facility (GOF), and the HEASARC. The instrument teams provide the expertise about the behavior of the instruments, the dependencies of this behavior on various environmental conditions, and the reliability of the measurements under various observational conditions. The HEASARC helps the instrument teams package and structure their calibration data so that it is easily accessible by standard data analysis software packages (for example, FTOOLS). A schematic showing the inter-relationship between the various elements within the calibration dataflow are shown in Figure 1. Those elements for which the HEASARC is primarily responsible is shown below the hatched line, while the instrument teams are primarily responsible for the elements above this line.

Figure Figure 1: A schematic representation of the calibration dataflow showing the definitions of, and relationship between, the various elements described in the text. In the case of current and future missions, the responsibility for those elements (including documentation) above the dashed line primarily lie with the hardware teams. In the case of past missions, the HEASARC will be responsible for locating and documenting the required information, although a substantial fraction of the necessary knowledge may already have been lost to posterity. The responsibility for those elements below the line jointly lies within the GOF, HEASARC and software teams.

The "Stage 1" Calibration Software (usually developed and maintained by the hardware teams) combines ground and in-orbit calibration measurements (the Primary Calibration Files, or PCFs) for a given instrument with any necessary theoretical modelling and algorithms to produce the Basic Calibration Files (BCFs). The data within the BCFs is then convolved with further algorithms, and if appropriate, housekeeping data from the satellite, by the "Stage 2" Calibration Software (usually developed and maintained by the mission scientists and/or the GOF) to produce the Calibration Products Files (CPFs). Together, the BCFs and CPFs form the main contents of the `HEASARC Calibration Database' and provide the calibration input to the various Data Analysis Packages.

3 CALIBRATION FILE TYPES

The Basic Calibration Files will ideally contain all the data (excluding any necessary housekeeping information) required to construct the CPFs. In the case of past missions this may, unfortunately, not always be possible. The BCFs will ideally contain calibration information which is both independent of time and in-orbit conditions (in most cases data originating from ground calibration measurements), and information which is expected to vary throughout the mission (mainly from in-orbit measurements). Examples of the type of calibration data included within the BCFs are:

the theoretical effective area of the instrument as a function of energy and detector co-ordinates.
the energy resolution of the detector.
a theoretical parameterization of the instrument point spread function as a function of energy, source intensity and detector co-ordinates.
the change of sensitivity of the instrument as a function of time and detector co-ordinates.
a parameterization of the various components of the instrument background as a function of time and in-orbit conditions.

The Calibration Product Files can be divided into two types: observation-specific and non-observation specific. Non-observation specific CPFs are independent of the conditions pertaining to a specific observation (ie for which housekeeping data is not required). These CPFs are primarily a rearrangement of the information contained within the BCFs suitable for a specific purpose within an individual Data Analysis Package. Observation-specific CPFs require housekeeping data or some input about the conditions of the observation (instrumental temperature, charged-particle rate, magnetic rigidity, high voltage level, etc). These CPFs are not strictly part of the HEASARC Calibration Database as each is associated with specific analysis products file (eg a light curve, spectrum) for a specific observations. These calibration files may be generated dynamically by the analysis software, or in some instances they may form part of the HEASARC Data Products archive.

The number and use of the CPFs for a specific instrument may increase as experience dictates. Examples of the type of calibration data included within the CPFs are:

the instrument `exposure map' describing the time each detector pixel spent unobscurred by any instrument support structure etc during a given exposure.
the detector response matrix describing the conversion between pulse height and energy as a function of detector co-ordinates during a given exposure.
a specific parameterization of the instrument point spread function as a function of energy, detector co-ordinates etc.

4 CALIBRATION UPDATES

The CALDB will be updated whenever necessary, for example to update calibration data which explicitly depend on time, or as new understanding of the behavior of instrumentation on-orbit is acquired. To preserve an historical record, previous versions of the calibration data (including erroneous datasets) will be retained. This allows researchers to compare the effects of improved calibrations with previous analyses. Updates to the CALDB are described in the document "How to Manage a Calibration Database" Active missions can communicate with the HEASARC CALDB to update the mission's calibration database and to communicate the updates to researchers - see the document "Automated Delivery of Calibration Data to the CALDB". These documents and others are available from the CALDB Documentation Library website, http://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/caldb_doc.html.

5 CALIBRATION DATABASE STRUCTURE & SUBROUTINE LIBRARIES

All the BCF and CPF data files within the CALDB will be in standard FITS format. All files are accessible on-line via anonymous ftp or for supported missions via the world-wide web. Users can also access appropriate CALDB data via remote access, in which appropriate calibration files are downloaded transparently.

Formats for many CALDB common calibration files have been defined by the HEASARC in consultation with individual missions; these formats are documented on the Developer section of the CALDB Documentation Library website. These formats are intended to take full advantage of the similarities between different individual instruments of similar type (like proportional counters or CCDs). However, it is recognized that at some level, the type and format of the calibration data required and available may differ between instruments. This is particularly the case for archival missions for which it may be impossible to always present the calibration in an identical form as some of the information may no longer be available.

The location of the main CALDB directory on disk is defined by an environment variable $CALDB. Data files for a given mission are located in subdirectories of this top-level directory. For a given mission data is stored within subdirectories under the directory path:

$CALDB/data/<mission>/<instrument>/<class>

where < mission > is a lower-case string (without spaces) naming the mission (for example einstein, rosat, suzaku) and < instrument > is a lower case string (without spaces) which gives the name of the mission, and < class > is either pcf, bcf or cpf. Standard strings for mission and instrument are listed in the document "Standard Strings for Mission, Instrument, Filter, Detector & Grating Names for OGIP FITS files", available from the CALDB documentation page.

To facilitate accesss to the CALDB a subroutine library (in ANSI FORTRAN or C) is maintained by the HEASARC. The subroutines employ the standard FITSIO package (Pence 1991) for FITS file input and output. This library is used by the Stage 2 Calibration Software and the HEASARC supported Data Analysis Packages to return required calibration data needed by the Data Analysis Packages (eg the effective area of the instrument at a specific time, energy and detector position). The source code for this library is also available for download at the FTOOLS webpage for any user wishing to develop their own Stage 2 Calibration Software and/or Data Analysis Package.

6 DOCUMENTATION

Documentation describing the HEASARC CALDB, the CALDB as implemented for specific missions, FITS data file formats, means to access HEASARC calibration data is available from the CALDB docmentation library. General information about the HEASARC CALDB, including recent news and updates, is available from the CALDB website.

USEFUL LINKS TO OTHER HTML PAGES

The following useful links are available (in the HTML version of this document only):

Footnotes:

¹ Office of Guest Investigator Programs

²High Energy Astrophysics Science Archive Research Center

File translated from T_EX by T_TH, version 3.67.
On 26 Nov 2005, 10:26.