The Compton Gamma Ray Observatory was launched in April 1991 as the second of
NASA's Great Observatories to look at the universe in the high energy regime
from 20 keV to 20 Gev. In July 1992, the Compton Observatory Archive was
opened to allow public access to gamma-ray data from the Compton Observatory
mission. This article describes the current and future contents of this
archive, how astronomers may access the data, and the catalog structures which
are used to index the data. The remainder of this introduction will briefly
describe the overall mission to help define the terms used in this paper. EGRET The Energetic Gamma Ray Experiment Telescope, a spark-chamber
detector which images roughly a steradian field of view at energies of 30 MeV
to 30 GeV. The spatial resolution for individual photons is strongly energy
dependent but is better than a degree for the highest energy photons.
All four experiments in the Compton Observatory operate simultaneously and
continuously except for passages through the South Atlantic Anomaly. The
imaging instruments, EGRET and COMPTEL, are turned off when they would be
observing only the Earth. The EGRET and COMPTEL fields of view are aligned
with the spacecraft's Z-axis and the OSSE scintillators can be oriented along
an approximately 180 degree arc in the X-Z plane. A typical observation period
of the observatory consists of a fixed pointing of the spacecraft for one or
more weeks with the OSSE experiment looking at a primary source when possible
and then slewing to a secondary source when the primary is occulted by the
Earth. When OSSE is looking at a source, the scintillators are nodded on and
off the source by a few degrees in a pattern set by the observer to determine
the instrumental and sky background. During an observation, the EGRET and
COMPTEL instruments may have some nominal target in the center of the field of
view, but many other sources may also be contained within the field.
The overall mission comprises a number of phases. A one month verification
phase followed the launch where a number of targets were observed briefly. In
Phase 1, which will last from May 1991 through November 1992, the COMPTEL and
EGRET instruments are making an all-sky survey. Data from this period was
proprietary to the instrument teams. In subsequent phases, an increasing
percentage of observing time will be allocated to Guest Investigator proposals.
In all cases, the data products are proprietary to the PI's or GI's for a
period of one year after they are produced in a usable form.
Routine data products are produced by the Principal Investigator (PI) teams for
their own data and that of the Guest Investigators (GIs). The products are
available for analysis for one year by the PI or GI under whose program the
observation was taken. Then, as the data become non-proprietary, they are sent
to the Compton Observatory Science Support Center (SSC) where they are
archived. Each instrument team also maintains an archive of data from its own
Characteristics of Compton Data
This section describes the general characteristics of the data available from
each instrument. For some of the instruments, only the low-level data products
have been defined and the discussion focuses on these.
All data archived in the Compton Observatory Archive is available in FITS
format. Often this is an encapsulation of the PI-developed internal formats.
The FITS binary tables extension is used extensively. The columns are
described in the FITS headers, and the numeric format of the data uses the IEEE
standards and the big-endian byte order as required by FITS. To accommodate
archival researchers who may wish to use the software systems developed by the
PI teams, software exists to convert these FITS files back to the original PI
Three different classes of data are available from the BATSE instrument:
Continuous, Pulsar data and Burst data. As the name indicates, continuous data
are collected continuously for the BATSE instrument. The low level data
comprise fairly high time resolution data in four discriminator channels, and
high energy resolution data with a temporal resolution of several minutes.
Associated high level products may include sources detected through Earth
occultation measurements and bursts which did not meet the trigger criteria.
Pulsar data are high energy resolution data which are folded on-board to a
specified period. Burst data are collected immediately before, during, and
after a gamma ray burst, solar flare or other BATSE trigger. Low level burst
data comprise more than a dozen datatypes where there is a general tradeoff
between temporal and spectral resolution of each type.
The primary low-level product of the OSSE experiment are the spectra summed
over the nodding interval. OSSE typically nods on and off target in two-minute
intervals, so these Spectral DataBase (SDB) products consist of two-minute
summed spectra. Most analysis will start with these, but several other
products are available for study. The Telemetry Scalar File (TSF) contains
information which allows studies with time resolutions smaller than the nodding
time. The OSSE Pulsar data detects individual events. Other products give
shield rates and other information useful for specialized investigations.
The lowest level COMPTEL data which will be available in the archive comprises
individual photon events for COMPTEL. For each event a time, position
(p), angle , energy and other
will be provided. An individual COMPTEL photon is not well-localized on the
sky. Rather, for a given photon, what is known is that original location was
from an annulus, where the center and diameter of the annulus are given by
p and . The width of the
annulus is a degree or two.
Higher level products, including skymaps, spectra and light curves generated
using sophisticated maximum entropy techniques, will also be archived.
The lowest level of EGRET data also comprise photon events. However, unlike
COMPTEL these have more conventional uncertainties in their location and are
amenable to more direct methods for generating maps and other higher level
products. The EGRET instrument is almost noise-free, i.e., virtually all EGRET
gamma-ray events in the observatory archive will represent real cosmic photons.
While a significant fraction of the triggers in the instrument are
non-gamma-ray events or Earth albedo gammas, these are marked early in the
EGRET standard processing. Maps and spectra, light curves and catalogs of
individual sources will also be placed in the archive.
Schedule of Deliveries
The pipelines for data delivery to the archive have just begun to flow. Below,
a tentative schedule of when we currently anticipate the first data of each
type is given.
How to access the archive
BATSE Burst July 1992
BATSE Pulsar December 1992
BATSE Continuous December 1992
OSSE SDB October 1992
Other OSSE November 1992
EGRET December 1992
COMPTEL December 1992
The Compton Observatory Archive is open to all astronomers. We prefer
electronic data retrieval, but when very large quantities of data are to be
retrieved, or for users who do not have electronic connections to the data
archive, we shall support archive retrieval to physical media on a limited
Nodes and accounts
The archive system is currently located on the node enemy of the SSC
cluster. A guest account, gof, is available for initial browsing and
occasional use of the archive. Users who anticipate frequent or heavy usage of
the archive may wish to obtain a personal account on enemy to
facilitate their work. This node is a Unix machine but is accessible through
both DECnet (SET HOST ENEMY) and TCP/IP (telnet
To use the gof account simply login as gof. No password is
required. This captive account will present you with a menu of possible
options. Simply select the Archive Data Selector option.
The Data Selector
Users who wish to retrieve data log into the archive machine as described
above, and start the Data Selector task. The Data Selector allows the user to
quickly select archive data according to a variety of selection criteria
including instrument, target, target type, position, time of observation and
The Data Selector is an INGRES forms application. The user enters data in the
appropriate fields, and a query is made of the Observation Catalog. The
Observation Catalog, described in detail below, maintains an index of the
archive and describes all of the data currently available.
Once a user has selected a set of observations of interest, he or she may elect
to retrieve the associated filesets. A fileset is simply a group of one or
more files which is retrieved as a unit. When using the gof account,
files that are retrieved from the archive are retrieved into directories on
enemy which are accessible through anonymous FTP or proxy DECnet
copies. The archival researcher can use either route to copy the files back to
his or her home computer. Users with accounts on the SSC cluster can retrieve
data directly to their own areas.
Data will also be retrievable through the HEASARC Browse system. While the
low-level data currently in the archive is not easily comparable with the
high-level data products available from other missions, e.g., Einstein and
EXOSAT, future products should be much more compatible.
Data retrieval is actually performed using the Generic Retrieve/Archive
Software Protocol (Grasp). Grasp is a protocol which the SSC has developed to
attempt to isolate the data retrieval from the underlying details of the
archive implementation. Grasp defines a small set of relatively simple
functions that the underlying archive will perform and provides the user with a
uniform interface to any of the underlying archives.
We envisage that the underlying data may reside on several different physical
archives both simultaneously and serially during the lifetime of the Compton
Observatory. Grasp allows us to isolate our dependencies on this heterogeneous
and evolving hardware.
The Physical Archive
Data in the Compton Observatory Archive is currently stored in two distinct
physical archives, a rewritable magneto-optical disk jukebox with a capacity of
92 GB, and the NSSDC NDADS archive. Generally we anticipate faster response
from the local jukebox, but the capacity of the NDADS is far greater. When
available, data will be retrieved from the local jukebox but the system will
automatically retrieve from the NDADS when necessary.
For the first year or two of the archive we will keep all data received in our
local jukebox. As the total volume of data begins to exceed the capacity of
our local jukebox only the data most frequently used will be stored locally.
The NSSDC NDADS archive will be used as a backup for the local system and to
store data which will not fit in our local system. A few data types will be
stored only in NDADS, e.g., telemetry. Also, the NSSDC has requested that data
be stored in the NDADS in both the native PI formats as well as in FITS when
those data are delivered to the SSC in the native format.
The Observation Catalog
The Observation Catalog is a relational database which indexes and describes
the contents of the archive. It is this catalog with which a user actually has
the greatest interaction when retrieving data.
The overall organization of the catalog is illustrated in Figure 1 where the
relationships between the various catalog tables is given. These tables form
two clusters, one about the OBSERVATION table and one about the FILESET table.
This first cluster contains information about the actions that the satellite
has taken and the scientific programs they are intended to support. The
OBSERVATION relation itself contains timing, target and program information
about the observations of the satellite. The two pointing tables, POINT and
OSSEPOINT, describe the specific spacecraft attitudes, and the PROGRAM relation
gives information about accepted Compton Observatory observing programs.
The cluster around the FILESET relation describes the data in the archive.
FILESET directly indexes the archive, with one entry for each archived fileset.
The LEVELINFO and TYPEINFO relations describe the types of filesets. The
FILELOG relation maintains a record of when filesets have been retrieved. The
FILEINFO and INFOTEXT relations allow comments to be made about filesets, e.g.,
warnings about flaws in the data or data processing. The FILESET_FILES,
KEYWORDS and FIXEDFIELDS tables help control the flow of data into the archive,
and the update of catalog fields to reflect new archive data.
The connection between these two clusters is the DATA relation which describes
the filesets used in a given observation. A fileset may be used in several
observations and conversely, an observation may generate many filesets.
Archive Data in the HEASARC
So far, all Compton Observatory data that has been released into the archive is
very low-level and quite instrument specific. These datasets will be visible
from within Browse but they will not be accessible visually the way other
datasets are, e.g., one cannot display spectra or images of the data. As
higher level products are produced and placed in the archive such visual
browsing of the Compton Observatory products will be supported.
The SSC has also begun to look at making Compton Observatory data compatible
with the formats used in popular astronomical packages such as XANADU and IRAF.
The idiosyncrasies of gamma-ray data make this non-trivial, but we expect to
introduce capabilities in this area in 1993. We anticipate that this effort
will be especially effective for the high-level Compton data products. The SSC
is working with other groups in the OGIP to try to make these products
compatible with those of other missions such as ROSAT and ASTRO-D.
Data analysis of archival data
While the SSC directly supports data in the archive only in FITS formats,
converters to reformat the data to the original Principal Investigator formats
are available for all datatypes. Thus PI software can be used to analyze all
archival data. Archival researchers can analyze their data at the PI team
sites or at the SSC. Generally it is advisable for archival investigators who
are just beginning work with Compton Observatory data to discuss their
research efforts with the Instrument Specialists and PI teams for the
instruments they are working on.
PI software tools are being collected at the SSC for use by Guest and archival
investigators. Currently the BATSE Spectral Analysis Software (BSAS), the OSSE
IGORE system, and some EGRET software has been brought up on the SSC cluster.
Any archival investigator may request an account to use this software.
Archival investigators may also wish to discuss how to port appropriate
software to their home sites with the PI teams.
As discussed above, the SSC is working with the HEASARC to make Compton
Observatory data usable within more general software packages. We expect that
some capabilities to do spectral analysis within XSPEC will be available this
winter, and additional functionality will be developed over the coming year.
Documents which may be used to get further information on topics discussed
above are described below. All documents described as SSC internal document
are available by anonymous FTP on the enemy and grossc nodes
of the SSC network.
The Compton GRO Mission
See various articles in The Compton Observatory Science Workshop, NASA
Conference Publication 3137, 1992.
Data produced by the Compton Observatory
These are defined in the Compton Observatory Project Data Management
Plan. Revision due 1992.
Some data formats for particular instruments
For low-level BATSE data: GRO BATSE Flight Software User's Manual,
The GRASP Interface
For OSSE SDB data: Spectral Data Base Version 7 Data Structure
Description, by Mark Strickman, NRL Document 0926-159 Rev 3.00, 1992.
This interface is defined in The Generic Archive/Retrieve Software
Protocol, SSC internal document.
The Observation Database
The Observation Database, including the mechanisms used for maintenance and
update, as well as a more complete description of the Data Selection, is
described in The Compton Observatory Observation Catalog, SSC internal
Proceed to the next article
Return to the previous article
Select another article
HEASARC Home |
Last modified: Monday, 19-Jun-2006 11:40:52 EDT
HEASARC support for unencrypted FTP access ended on September 20, 2019. Please see this notice for details.