OGIP Calibration Memo CAL/GEN/93-006

The Organization of the HEASARC Calibration Database

Ian M George & Ron S Zellar
Code 668,
NASA/GSFC,
Greenbelt,
MD 20771



Version: 1995 Jul 17






SUMMARY

The location, organization and contents of the HEASARC calibration database on-line at NASA/GSFC is summarized.

LOG OF SIGNIFICANT CHANGES

Release Sections Changed Brief Notes
Date
1993 Apr 30 All First Public Version
1995 Feb 15 All Made compatible with LaTeX2HTML software
2004 Apr 01 All Made compatible with tth

Contents

1  OVERVIEW
    1.1  Location
    1.2  Basic Structure
    1.3  Relation to Other Areas of the HEASARC anonymous FTP account
    1.4  Access
        1.4.1  Anonymous ftp to the Legacy machine
2  CALIBRATION DATA FILES
    2.1  BCF & CPF Datasets
        2.1.1  Symbolic Links to Other Areas
    2.2  PCFs
        2.2.1  Symbolic Links to Other Areas
    2.3  Non-FITS files
3  CALIBRATION SOFTWARE
    3.1  CALTOOLS & CALLIB
        3.1.1  Symbolic Links to Other Areas
4  CALIBRATION DOCUMENTATION

1  OVERVIEW

1.1  Location

The HEASARC calibration database (CALDB) is physically located on the legacy machine within the LHEA at NASA/GSFC:

within the caldb directory tree. The entire caldb directory tree is available to remote users via anonymous ftp (see Section 1.4.1). Within the HEASARC, the caldb has been remote mounted on all other machines1 and the symbolic links /caldb & /CALDB created to point to the above top level directory.

1.2  Basic Structure

The contents of the /caldb area is sub-divided between the following sub-directory trees2:

All directories and files within the /caldb tree are freely accessible to remote users, and each is discussed in more detail in the following sections.

1.3  Relation to Other Areas of the HEASARC anonymous FTP account

During a reorganization of the HEASARC anonymous FTP account on legacy carried out in 1993 Feb, it was decided to relationalize the structure seen by remote users in the hope that this would facilitate their search for desired files. The account contains the following sub-directories:

with the structure described in some detail in Drake (1993). It can be seen that there are two routes by which a user can access the calibration data: via /mission/calib_data, or via /caldb/mission. It is considered likely that most users will prefer to use the former route, collecting calibration data along with other information specific to a given mission/instrument. However users concerned only with calibration data (eg those maintaining their own, local copy of the CALDB may prefer to use the latter route. It should be stressed however that both routes access the same physical files, and the two access routes are (invisibly) connected via a symbolic link (see Section 2). A similar choice of routes also exists for some parts of the calibration software and documentation trees. These are also discussed in Sections 3 & 4 respectively.

1.4  Access

As described in Section 1.1, at the present time the HEASARC CALDB exists onftp://legacy.gsfc.nasa.gov/caldb.

1.4.1  Anonymous ftp to the Legacy machine

Users of machines on the Internet network may use File Transfer Protocol (ftp), to list the contents and/or copy file from the HEASARC CALDB.

To initiate ftp from the command line, type the following (boldface) commands in a terminal window at the prompt (comments appear in italics):
ftp legacy.gsfc.nasa.gov (or ftp 128.183.8.233)
then
ftp> user anonymous
Password: type your e-mail address as the password
You will now be in the top level directory3 of the HEASARC anonymous ftp account on legacy and a message to this effect, along with any important bulletins, should be displayed. A general overview of the account can be found in Drake (1993). More detailed on-line information, instructions etc concerning the ftp account can be found within the ASCII README file within this (and many other) directories.

The following list of basic ftp commands may be of use to the inexperienced ftp user:
ftp> help                 (or ftp> ?) displays list of ftp commands
ftp> help lcd displays help for ftp command lcd
ftp> cd caldb/docs move to the caldb/docs directory
ftp> cd ../ move back up one directory
ftp> ls brief list of the contents of the current legacy directory
ftp> dir detailed list of the contents of the current legacy directory
ftp> pwd display the present working directory on legacy
ftp> binary set binary transfer type (required for FITS files etc)
ftp> mget * copies all files in the current legacy directory to your area
ftp> ascii set ASCII transfer type (required for LATEXdocuments, postscript files etc)
ftp> get README copies the README from current Legacy dir to local dir
ftp> lcd /home change local directory to /home on user's home machine
ftp> quit quits from ftp & returns control to local operating system A variety of other useful commands are available within ftp, a number of which depend on which version of ftp is installed on the local machine.

2  CALIBRATION DATA FILES

The HEASARC CALDB classifies files as `Primary Calibration Files' `Basic Calibration Files' and `Calibration Product Files' (PCFs, BCFs & CPFs respectively; see also CAL/GEN/91-001). PCFs are 'raw' ground and in-orbit calibration datasets not of immediate interest to most users as they are not directly required for (all but the most specialized) scientific data analysis tasks. BCFs contain the lowest level calibration datasets potentially required by downstrem software, and can be considered the `atomic units' of the instrument calibration. CPFs contain `convolutions' of the information stored within BCFs customized for a specific analysis task and/or scientific observation.

2.1  BCF & CPF Datasets

All BCF & CPF calibration files are organized using the scheme

/caldb/data/mission/inst
where mission & inst are the OGIP-standard names for the mission and instrument. For internal management purposes, a further division into inst/bcf and inst/cpf sub-directories is made in most cases. Miscellaneous, non-instrument specific calibration datasets, including general spacecraft housekeeping information etc, can be found in the /caldb/data/mission/mis sub-directory.

Besides the calibration datasets themselves, each sub-directory of /caldb/data/mission contains:

2.1.1  Symbolic Links to Other Areas

As mentioned in Section 1.3, there are 2 routes via which ftp users can access BCF & CPF calibration datasets:

Both routes access the same physical files, and the two access routes are connected via a symbolic link calib_data in the /mission directory to the /caldb/data/mission directory4 Thus ftp users in (for example) /rosat who change directory to calib_data (ftp> cd calib_data) will actually set their current working directory to /caldb/data/rosat directory. A message to this effect will appear on their screen, along with instructions that at any time they may return to the main /rosat ftp area by changing directory to rosat (ftp> cd /rosat).

2.2  PCFs

Due the the large data volume, the PCFs from all missions are given their own parallel directory tree:

/caldb/data/pcf/mission
where mission in the standard OGIP name for the mission. Each directory is further divided into the following sub-directories:

2.2.1  Symbolic Links to Other Areas

The calibration datasets resulting from the observation of astronomical sources (eg the Crab) are available in the /mission/data/inst directory tree (see Section 1.3). Thus cal_obs will be a symbolic link to this directory. It should be noted that NO corresponding symbolic link from the /mission/data/inst area back to /caldb/data/pcf/mission/inst will be provided. Users wishing to return will therefore be required to use the full path when changing directory back.

2.3  Non-FITS files

For the convenience of users, at the present time the HEASARC CALDB does contain a small number of calibration datasets in formats other than FITS. In almost all cases, FITS versions of these datasets (using a standard OGIP file format) also exist within the caldb. Users should note that the permanent existence, quality etc of these non-FITS files is in NO WAY guarenteed. In most cases such files represent calibration datasets in an old format. Thus such files are stored in a old sub-directory of the .../inst/bcf or .../inst/cpf directories. ASCII README files usually exist explaining the formats etc of the datasets, and giving the name and location of the corresponding FITS versions of the file(s).

3  CALIBRATION SOFTWARE

3.1  CALTOOLS & CALLIB

The bulk of the calibration-related software publically available from the HEASARC will be in the form of ftools forming the caltools sub-package. All routines are written in ANSI FORTRAN or C, and are able to be executed both as standalone tasks on all OGIP-supported platforms/operating systems, or from within IRAF environment. A list of all currently available and planned caltools tasks is given in CAL/SW/93-004.

Users may also be interested in the callib subroutine library (in a directory of this name parallel to the caltools package) containing commonly used utilities, and in particular, routines for reading and writing calibration datasets adhering to OGIP-standard file formats. A detailed listing of the contents of callib can be found in CAL/SW/93-005.

Details on the installation and maintanence of the HEASARC caldb software collection at remote nodes can be found in CAL/GEN/92-015.

3.1.1  Symbolic Links to Other Areas

The /caldb/software/caltools file is a symbolic link to the /software/ftools/caltools area within the main HEASARC software collection. Similarly /caldb/software/callib is a symbolic link to the /software/ftools/callib area.

It should be noted that NO corresponding symbolic links from these directories back to /caldb/software are provided. Users wishing to return will therefore be required to use the full path of the disired directory when changing back.

4  CALIBRATION DOCUMENTATION

The /caldb/docs directory is sub-divided into

In most cases both the LaTeX source code, Postscript & HTML versions of each document are available.

The best starting point for finding calibration-related documentation is via the World-wide Web via the URL:
/docs/heasarc/caldb/caldb\_doc.html






REFERENCES

Drake, S., & O'neel, B., 1993. Legacy, 3, 53.

George, I.M., 1992. Legacy, 1, 56. (CAL/GEN/91-001)

George, I.M. & Zellar, R., 1993. In preparation. (CAL/GEN/92-015)

George, I.M., Zellar, R. & Yusaf, R, 1993a. In preparation. (CAL/SW/93-004)

George, I.M., Zellar, R. & Yusaf, R, 1993b. In preparation. (CAL/SW/93-005)

Pence, W., 1992. Legacy, 1, 14.

USEFUL LINKS TO OTHER HTML PAGES

The following useful links are available (in the HTML version of this document):


Footnotes:

1Currently this excludes the LHEA Vaxes

2We generally use unix-styledirectory paths throughout this memo for simplicity.

3This will be the +/+directory for ftp users, and the +/FTP+directory for non-ftp users.

4NOTE This symbolic link ONLY works for ftp users. Users not logged onto +legacy+via ftp unfortunately have to specify the path to the relevant caldb area either using:




File translated from TEX by TTH, version 3.13.
On 4 May 2004, 12:01.