OGIP Calibration Memo CAL/GEN/93-006
The Organization of the HEASARC Calibration Database
Ian M George
&
Ron S Zellar
Code 668,
NASA/GSFC,
Greenbelt,
MD 20771
Version: 1995 Jul 17
|
Release | Sections Changed | Brief Notes |
Date | ||
1993 Apr 30 | All | First Public Version |
1995 Feb 15 | All | Made compatible with LaTeX2HTML software |
2004 Apr 01 | All | Made compatible with tth |
The
HEASARC calibration database (CALDB)
is physically located on the
legacy machine within the
LHEA
at
NASA/GSFC:
The contents of the /caldb area is sub-divided between
the following sub-directory trees2:
During a reorganization of the HEASARC anonymous FTP account
on legacy carried out in 1993 Feb, it was decided to
relationalize the structure seen by remote users in the hope that
this would facilitate their search for desired files.
The account contains the following sub-directories:
As described in Section 1.1, at the present time the
HEASARC CALDB
exists onftp://legacy.gsfc.nasa.gov/caldb.
Users of machines on the Internet network may use File Transfer Protocol
(ftp), to list the contents and/or copy file from the
HEASARC CALDB.
To initiate ftp from the command line, type the following (boldface) commands in a terminal window at the prompt (comments appear in italics):
The following list of basic ftp commands may be of use to the
inexperienced ftp user:
The
HEASARC CALDB
classifies files as `Primary Calibration Files'
`Basic Calibration Files' and `Calibration Product Files'
(PCFs, BCFs & CPFs respectively; see also
CAL/GEN/91-001).
PCFs are 'raw' ground and in-orbit calibration datasets not of
immediate interest to most users as they are not directly required for
(all but the most specialized) scientific data analysis tasks.
BCFs contain the lowest level calibration datasets potentially required
by downstrem software, and can be considered the `atomic units' of the
instrument calibration.
CPFs contain `convolutions' of the information stored within BCFs
customized for a specific analysis task and/or scientific observation.
All BCF & CPF calibration files are organized using the scheme Contents
1 OVERVIEW
1.1 Location
1.2 Basic Structure
1.3 Relation to Other Areas of the HEASARC anonymous FTP account
1.4 Access
1.4.1 Anonymous ftp to the Legacy machine
2 CALIBRATION DATA FILES
2.1 BCF & CPF Datasets
2.1.1 Symbolic Links to Other Areas
2.2 PCFs
2.2.1 Symbolic Links to Other Areas
2.3 Non-FITS files
3 CALIBRATION SOFTWARE
3.1 CALTOOLS & CALLIB
3.1.1 Symbolic Links to Other Areas
4 CALIBRATION DOCUMENTATION
1 OVERVIEW
1.1 Location
within the caldb directory tree.
The entire caldb directory tree is available to remote users
via anonymous ftp (see Section 1.4.1).
Within the
HEASARC,
the caldb has been remote mounted on all other
machines1
and the symbolic links /caldb & /CALDB created to point to
the above top level directory.
1.2 Basic Structure
All directories and files within the /caldb tree are
freely accessible to remote users, and each is discussed in more detail
in the following sections.
1.3 Relation to Other Areas of the HEASARC anonymous FTP account
with the structure described in some detail in Drake (1993).
It can be seen that there are two routes by which a user can access
the calibration data: via /mission/calib_data,
or via
/caldb/mission.
It is considered likely that most users will prefer to use the former
route, collecting calibration data along with other information specific
to a given mission/instrument.
However users concerned only with
calibration data (eg those maintaining their own, local copy of the
CALDB
may prefer to use the latter route.
It should be stressed however that both routes access the same physical
files, and the two access routes are (invisibly) connected via
a symbolic link (see Section 2).
A similar choice of routes also exists for
some parts of the calibration software and documentation trees. These are
also discussed in Sections 3 &
4 respectively.
A generic mission sub-directory tree therefore consists of:
1.4 Access
1.4.1 Anonymous ftp to the Legacy machine
ftp legacy.gsfc.nasa.gov
(or ftp 128.183.8.233)
then
ftp> user anonymous
Password:
type your e-mail address as the password
You will now be in the top level directory3 of the HEASARC anonymous ftp
account on legacy and a message to this effect, along with any
important bulletins, should be displayed.
A general overview of the account can be found in Drake (1993). More
detailed on-line information, instructions etc concerning the
ftp account can be found within the ASCII README file within this
(and many other) directories.
ftp> help (or ftp> ?)
displays list of ftp commands
ftp> help lcd
displays help for ftp command lcd
ftp> cd caldb/docs
move to the caldb/docs directory
ftp> cd ../
move back up one directory
ftp> ls
brief list of the contents of the current legacy directory
ftp> dir
detailed list of the contents of the current legacy directory
ftp> pwd
display the present working directory on legacy
ftp> binary
set binary transfer type (required for FITS files etc)
ftp> mget *
copies all files in the current legacy directory to your area
ftp> ascii
set ASCII transfer type (required for LATEXdocuments, postscript
files etc)
ftp> get README
copies the README from current Legacy dir to
local dir
ftp> lcd /home
change local directory to /home on user's home machine
ftp> quit
quits from ftp & returns control to local operating system
A variety of other useful commands are available within ftp, a number of
which depend on which version of ftp is installed on the local machine.
2 CALIBRATION DATA FILES
2.1 BCF & CPF Datasets
Besides the calibration datasets themselves, each sub-directory of /caldb/data/mission contains:
As mentioned in Section 1.3, there are 2 routes via which ftp users can access BCF & CPF calibration datasets:
Due the the large data volume, the PCFs from all missions are given their
own parallel directory tree:
The calibration datasets resulting from the observation of astronomical sources (eg the Crab) are available in the /mission/data/inst directory tree (see Section 1.3). Thus cal_obs will be a symbolic link to this directory. It should be noted that NO corresponding symbolic link from the /mission/data/inst area back to /caldb/data/pcf/mission/inst will be provided. Users wishing to return will therefore be required to use the full path when changing directory back.
For the convenience of users, at the present time the HEASARC CALDB does contain a small number of calibration datasets in formats other than FITS. In almost all cases, FITS versions of these datasets (using a standard OGIP file format) also exist within the caldb. Users should note that the permanent existence, quality etc of these non-FITS files is in NO WAY guarenteed. In most cases such files represent calibration datasets in an old format. Thus such files are stored in a old sub-directory of the .../inst/bcf or .../inst/cpf directories. ASCII README files usually exist explaining the formats etc of the datasets, and giving the name and location of the corresponding FITS versions of the file(s).
The bulk of the calibration-related software publically available from the HEASARC will be in the form of ftools forming the caltools sub-package. All routines are written in ANSI FORTRAN or C, and are able to be executed both as standalone tasks on all OGIP-supported platforms/operating systems, or from within IRAF environment. A list of all currently available and planned caltools tasks is given in CAL/SW/93-004.
Users may also be interested in the callib subroutine library (in a directory of this name parallel to the caltools package) containing commonly used utilities, and in particular, routines for reading and writing calibration datasets adhering to OGIP-standard file formats. A detailed listing of the contents of callib can be found in CAL/SW/93-005.
Details on the installation and maintanence of the HEASARC caldb software collection at remote nodes can be found in CAL/GEN/92-015.
The /caldb/software/caltools file is a symbolic link to the /software/ftools/caltools area within the main HEASARC software collection. Similarly /caldb/software/callib is a symbolic link to the /software/ftools/callib area.
It should be noted that NO corresponding symbolic links from these directories back to /caldb/software are provided. Users wishing to return will therefore be required to use the full path of the disired directory when changing back.
The /caldb/docs directory is sub-divided into
The best starting point for finding calibration-related documentation
is via the World-wide Web via the URL:
/docs/heasarc/caldb/caldb\_doc.html
Drake, S., & O'neel, B., 1993. Legacy, 3, 53.
George, I.M., 1992. Legacy, 1, 56. (CAL/GEN/91-001)
George, I.M. & Zellar, R., 1993. In preparation. (CAL/GEN/92-015)
George, I.M., Zellar, R. & Yusaf, R, 1993a. In preparation. (CAL/SW/93-004)
George, I.M., Zellar, R. & Yusaf, R, 1993b. In preparation. (CAL/SW/93-005)
Pence, W., 1992. Legacy, 1, 14.
The following useful links are available (in the HTML version of this document):
1Currently this excludes the LHEA Vaxes
2We generally use unix-styledirectory paths throughout this memo for simplicity.
3This will be the +/+directory for ftp users, and the +/FTP+directory for non-ftp users.
4NOTE This symbolic link ONLY works for ftp users. Users not logged onto +legacy+via ftp unfortunately have to specify the path to the relevant caldb area either using: