OGIP Calibration Memo CAL/GEN/91-001

The HEASARC Calibration Database
(a brief overview)

Ian M. George
Code 668,
Greenbelt, MD20771

Version: 1995 Mar 29


The rôle of the HEASARC concerning the storage and documenting of the calibration data from past, current and future missions is briefly outlined.


Release Sections Changed Brief Notes
1991 Oct 21 Original Version
1995 Jan 11 All Made compatible with LaTeX2HTML software
1995 Feb 13 Figure 1 Made EPSF and included in text
2004 Apr 1 All Made compatible with tth


The availability of the instrument calibration and an understanding of its limitations is fundamental to the scientific analysis of data. In the past, calibration data has generally been made available in a somewhat ad hoc fashion. In most cases it has been a `black box' part of an analysis task, hidden from the user. There is usually little documentation available, nor any record of previous versions of the calibrations.

It is unlikely that the majority of HEASARC users will have an intimate understanding of all the detector details of all the instruments from which they obtain data. The transfer of the necessary information such that a user can make sensible judgements concerning their dataset and associated calibration measurements is therefore critical.

The HEASARC intends to locate, document and store on-line a full and up-to-date set of calibration data for each instrument for which scientific data is contained within the archive. The HEASARC will define the standard types and formats for the calibration data files, and develop a subroutine library to access them. The overall motivation is to present users and programmers with calibration data available for each instrument in a standard, flexible and user-friendly format, along with information concerning its use, importance and quality.


A schematic showing the inter-relationship between the various elements within the calibration dataflow are shown in Figure 1. Those elements for which the HEASARC is primarily responsible are shown hatched.

The Stage 1 Calibration Software (developed and maintained by the hardware teams) combines ground and in-orbit calibration measurements for a given instrument with any necessary theoretical modelling and algorithms to produce the Basic Calibration Files (BCFs). The data within the BCFs is then convolved with further algorithms, and if appropriate housekeeping data from the satellite, by the Stage 2 Calibration Software (developed and maintained by the relevant project) to produce the Calibration Products Files (CPFs). Together, the BCFs and CPFs form the contents of the `HEASARC Calibration Database' and provide the calibration input to the various Data Analysis Packages.


The Basic Calibration Files will ideally contain all the data (excluding any necessary housekeeping information) required to construct the CPFs. In the case of past missions this may, unfortunately, not always be possible. The BCFs will contain calibration information which is both independent of time and in-orbit conditions (in most cases data originating from ground calibration measurements), and information which is expected to vary throughout the mission (mainly from in-orbit measurements).
Examples of the type of calibration data included within the BCFs are:
- the theoretical effective area of the instrument as a function of energy and detector co-ordinates.
- the energy resolution of the detector.
- a theoretical parameterization of the instrument point spread function as a function of energy, source intensity and detector co-ordinates.
- the change of sensitivity of the instrument as a function of time and detector co-ordinates.
- a parameterization of the various components of the instrument background as a function of time and in-orbit conditions.

The Calibration Product Files can be divided into two types. The first is those CPFs that are independent of a specific observation (ie for which housekeeping data is not required). These CPFs are primarily a rearrangement of the information contained within the BCFs suitable for a specific purpose within an individual Data Analysis Package. The second type of CPF is that for which housekeeping data is required and hence observation specific. These CPFs are not strictly part of the HEASARC Calibration Database as each is associated with specific analysis products file (eg a light curve, spectrum). They will therefore form part of the HEASARC Products Database.

The number and use of the CPFs for a specific instrument may increase as experience dictates.
Examples of the type of calibration data included within the CPFs are:
- the instrument `exposure map' describing the time each detector pixel spent unobscurred by any instrument support structure etc during a given exposure.
- the detector response matrix describing the conversion between pulse height and energy as a function of detector co-ordinates during a given exposure.
- a specific parameterization of the instrument point spread function as a function of energy, detector co-ordinates etc.


In the case of current and future missions, the Calibration Databases will be `live' in the sense that it will be continually updated as necessary. Periodic updates to those BCFs which contain data which explicitly depend on time will be obviously be necessary. As the mission progresses, the appropriate files will thus be extended as each new set of calibration measurements becomes available. Updates to the BCFs will also obviously be necessary in the event of an error. However in all cases previous (including erroneous) versions of the calibration data will be retained within the same file to ease direct comparison of results. The format of the BCFs and CPFs will therefore be such that updates are easily incorporated. In the case of a current mission, the project/GOF will automatically inform Guest Observers of calibrations updates, and during the propriety data period re-release appropriate BCFs and (reprocessed) CPFs. In the case of a past mission, all efforts will be extended to include earlier calibrations into the appropriate databases, although this task will be given a relatively low priority. Notification of calibration updates and a brief description of their impact will be made available on a bulletin board and within future issues of Legacy to enable users to assess their impact on previous results.


All the data files within the HEASARC Calibration Database will be in standard FITS format and stored on-line. Users will therefore be able to browse the various calibration data available and extract data. The detailed format of individual files will be defined by the HEASARC in discussion with the various projects currently distributing calibration data. It is intended to take full advantage of the similarities between different individual instruments of similar type, and hence make the format of the corresponding calibration files identical wherever possible. However, it is recognized that at some level, the type and format of the calibration data required and available may differ between instruments. This is particularly the case for past missions for which it may be impossible to always present the calibration in an identical form as some of the information may no longer be available. Full documentation of the format of each calibration file for each instrument will of course be provided by the HEASARC(see section 6 below).

To facilitate accesss to the Calibration Database a subroutine library will be developed (in ANSI FORTRAN or C) and maintained by the HEASARCThe subroutines will employ the FITSIO package (Pence 1991) and be used by the Stage 2 Calibration Software and the HEASARC supported Data Analysis Packages to return all the calibration results required by the Data Analysis Packages (eg the effective area of the instrument at a specific time, energy and detector position). The source code for this library will also be made available for user wishing to develop their own Stage 2 Calibration Software and/or Data Analysis Package.


The existence of several updates to the calibration data within the BCFs introduces an added complexity to the construction of the CPFs. By default, the Stage 2 Calibration Software will use the calibration data within the BCFs deemed at the time of processing to be the most appropriate. The resultant CPFs will be sent to the GO in the case of a current mission, and in the absence of later updates, transferred to theHEASARC data archives once the propriety data period expires. However, the Stage 2 Calibration Software will also be sophisticated enough to allow a user to interactively display and choose between the various alternate calibration measurements avaliable. Hence a user may therefore customise their own CPFs. The problems, changes and effects of the various calibrations will be well documented to enable the user to make optimum use of the information and software.


Substantial hard-copy & on-line documentation will be provided, and in the case of current and future missions may form part of the Observer's Guide. It's main purpose is to serve as a guide to HEASARC users & staff as to the status of the calibration data of a given instrument, the origin and quality of the data, the detailed contents of the calibration files, and a recipe for the use of the calibration data. The detailed format of each of the calibration files will also be included, as will notes on relevant experiences, problems and worries associated with the calibration of the science data. The guide should be written at a level appropriate for a user with only limited experience of X-ray astronomy, and hence be accessible to users from the general astronomical community. Copies will be made available to the community on request as appropriate. A reduced form of the guide will also be kept on-line in the HEASARC Documents Database.


Work is currently underway locating, reformatting and documenting the Einstein IPC, EXOSAT LE/CMA and ROSAT PSPC calibration data. A draft definition of the formats will be circulated to interested parties in the near future for comments. The subroutine library will simultaneously be developed. Thereafter work will commence on the other datasets for which the has responsibilities, starting with the EXOSAT ME, Einstein HRI and ROSAT HRI. It is anticipated that ASCA (formerly ASTRO-D) data will be distributed in the standard format from the start. The EXOSAT LE Calibration Guide is in preparation and will be used as a guide to the production of the documentation of the other instruments. Anybody wishing to contribute original calibration data or products and experiences are encouraged to contact the author.



Figure 1: A schematic representation of the calibration dataflow showing the definitions of, and relationship between, the various elements described in the text. In the case of current and future missions, the responsibility for those elements (including documentation) above the dashed line primarily lie with the hardware teams. In the case of past missions, the HEASARC will be responsible for locating and documenting the required information, although a substantial fraction of the necessary knowledge may already have been lost to posterity. The responsibility for those elements below the line jointly lies within the GOF, HEASARC and software teams.


The following useful links are available (in the HTML version of this document only):

File translated from TEX by TTH, version 3.13.
On 20 Apr 2004, 14:45.