NOTICE:
This Legacy journal article was published in Volume 5, November 1994, and has not been
updated since publication. Please use the search facility above to find regularly-updated information about
this topic elsewhere on the HEASARC site.
|
The ASCA Archive
N.E. White (HEASARC, ASCA), E.V. Gotthelf (ASCA),
& N. Oliverson (NSSDC)
The ASCA archive opened on 15 November 1994 and is accessed via the online
service of the High Energy Astrophysics Science Archive Research Center,
HEASARC. This article describes the archive contents and how to retrieve the
data. The analysis of ASCA data is fully described in the document "An abc
Guide to ASCA Analysis" which is available in the anonymous ftp area on
heasarc.gsfc.nasa.gov under asca/docs.
1 The ASCA Archive
The following propriertary periods apply to the ASCA data:
- PV data taken between April 1993 and approximately December 1993 is made
public 18 months from the observation date.
- US and joint US/Jp data (i.e. with a US PPI) that was delivered prior May 2
1994 will be public 1 year after that date.
- US and joint US/Jp data delivered after May 16 1994 will be public 1 year + 2
weeks from dispatch to the PI.
- Jp and joint Jp/US data (with a Japanese PPI) will be delivered 18 months + 2
weeks from dispatch to the PI.
A list of the currently available data, and release dates for future data is
available from the BROWSE database (http://heasarc.gsfc.nasa.gov/W3Browse.
, database ASCAPUBLIC).
A text dump of the archive contents is also available by anonymous FTP at
heasarc.gsfc.nasa.gov under asca/docs/ascapublic.txt.
The data will initially be released in 1 month batches, around the 15th of each
month, with the first batch released on 15 November 1994. As the archive
population becomes routine, the releases will be made more often, on a weekly
basis.
1.1 Accessing the ASCA data
Public ASCA data may be retrieved in any one of the following ways:
- By anonymous FTP at heasarc.gsfc.nasa.gov under the asca/data/[sequence
number] directories (see Section 3, below).
- By logging into heasarc.gsfc.nasa.gov, username XRAY, BROWSE database
ASCALOG. Data products can be located and extracted via the BROWSE
database interface (Section 3)
- By the World Wide Web at http://heasarc.gsfc.nasa.gov/W3Browse/.
Select the mission ASCA and then the database ascapublic. Data
products can be located and extracted via this interface (Section 3).
- By an e-mail request to the NASA's NSSDC archive service (Section 4),
Address: ndadsa.gsfc.nasa.gov, Subject: REQUEST ASCA DATA_TYPE, Message:
SEQUENCE_NUMBER.
In all cases the same basic file structure (described in Section 2, below) is
used to store the data.
Intrepid explorers of the ASCA public archive should note that the PV data,
especially those taken in the first few months, are not always optimized for
scientific investigation of the targeted object. Many observations were made
for instrument calibration and contain frequent mode changes and non-standard
parameter setting for which instrument responses are not available. The
observation modes and instrument settings during the latter PV and AO phase are
more stable. If in doubt, please ask the ASCA GOF.
2 Archive File Structure
ASCA data is most conveniently stored (and labeled) by the 8-digit SEQUENCE
NUMBER which is assigned to each pointed observation and generally consists of
a day's worth of consecutive data.
The data for a particular observation sequence is quite large (up to 200 Mbytes
or more) and thus the data is divided into six manageable sub-directories
(TELEM, AUX, RAW, SCREENED, UNSCREENED, and PRODUCT). For a typical analysis
not all files are necessary. The following is summary of each sub-directory and
what is needed for a typical analysis:
- TELEM: Contains the telemetry (FRF) files which makes up each sequence
(typically one file, occasionally more). These are by far the largest files and
these data are NOT required; they are here for completeness.
- RAW: Contains the fundamental FITS format event list files, with one file per
mode change per instrument. These are the basic data, which are required for a
complete analysis of the data. You'll want this if you plan to start the
analysis from scratch, using ascascreen and tkascascreen. But before
downloading this relatively large number of files, its wise to take a quick
look at one of the processed products.
- AUX: Contains the calibration and filter files. The filter files (extension
.mkf) are necessary to time filtering the data. To make any rescreening of the
data the .mkf file must be retrieved. For barycentric timing correction the
frf.orbit file is required. The other files contain calibration information
used during the processing and are typically not required for a standard
analysis.
- UNSCREENED: Contains the modally merged event list files for each instrument,
but not screened to remove bad data (no time filter). This is a convenient
starting point for basic data analysis, especially if you want to rescreen the
data using different cleaning criteria.
- SCREENED: Contains the modally merged event list files - screened using
current standard filter criteria. These files are useful for quick analysis to
extract standard spectra, images, and, light curves.
- PRODUCTS: contains the product catalog, information, and contents files. A
good getting started point with basic observation catalog information,
pre-digested quick look images, and summary plots.
The individual files for all four instruments (SIS0 + SIS1 + GIS2 + GIS3), for
a given observation sequence, are stored together under [seq #]/[sub_dir].
This structure is automatically recognized by the Xselect v1.2 and higher. It
is recommended that this directory structure be duplicated at the home
institution. Most of the files have been compressed (*.Z extension) using the
UNIX compress utility for ease of storage and transfer and should be
uncompressed before starting an Xselect session.
The individual ASCA data files are archived in FITS format and two basic naming
conventions are used to identify them:
a) The Telemetry FRF (First Reduction Files) convention:
root: ft930408_0413_0603 = ft[OBS-DATE]_[START(UT)]_[STOP(UT)]
b) AD (Modally merged files):
root: ad10004010 = ad[SEQUENCE-NUMBER]
c) Appendage (added on for both a and b):
s100312h = [s1] [003] [12] [h]
= [instrument] [index] [inst. mode] [bitrate]
= [s0/s1/g2/g3] [0-999] [code] [l/m/h]
||
Some common modes: 01 = SIS FAINT
02 = SIS BRIGHT
03 = SIS FAST
70 = GIS PH
For the FT files, the index parameter orders the files by time. The AD files
are indexed by number of events; for a given set, files with the lowest numbers
have the largest number of events.
d) Extensions
The following extensions are commonly used to denote different types of
files:
- .fits indicate raw FITS files containing photon event data.
- .unf are unscreened FITS events files.
- .evt are screened FITS events files.
- .img are FITS images in sky coordinates.
- .detimg are FITS images in detector coordinates.
- .gif are gif format images. The file names are appended with _detimg or _img
to indicate if the images are sky or detector coordinates.
- .mkf are the "makefilter" filter files, used to screen the data.
- .txt contain ASCII files.
- .log are the output from the processing logs.
- .ps are postscript output from the quicklook analysis.
- .cat are the observation catalogs, which list all the modes and detector
configurations. These can be used via xselect to choose the files of
interest.
3 HEASARC Archive Access
ASCA data archived at the High Energy Astrophysics Science Archive (HEASARC) is
available directly via anonymous FTP at heasarc.gsfc.nasa.gov under
asca/data/rev1/[seq_num]. A database interface is also available in the same
account by logging into legacy and typing browse and selecting the
ASCALOG database. A World Wide Web (WWW) interface to browse on legacy
is also available.
3.1 Anonymous FTP
Data can retrieve by anonymous FTP in the usual way (% = UNIX
prompt}:
%ftp heasarc.gsfc.nasa.gov
(username: anonymous, password: your_e-mail_address)
%cd asca/data/rev1/seq_num/sub_dir
%binary
%[prompt]
%[m]get file_name[s]
%quit
To find the sequence number of an observation, use BROWSE (see below) or the
file in asca/docs/ascapublic.txt.
Example 1: Getting the cleaned products.
mget asca/data/rev1/60000000/product/*.gif <-- get all the gif images
mget asca/data/rev1/60000000/product/*.evt <-- get the merged cleaned
event list data
mget asca/data/rev1/60000000/product/*.img <-- get all the sky FITS
images
Example 2: Getting all the screened event files
mget asca/data/rev1/60000000/screened/* <-- get all the screened event
files plus the observation
catalogs.
Example 3: Getting all the unscreened event files
mget asca/data/rev1/60000000/unscreened/* <-- get all the unscreened event
files, and the observation
catalogs
mget asca/data/rev1/60000000/aux/*.mkf.Z <-- get the mkf
files
Example 4: Getting the raw data
mget asca/data/rev1/60000000/raw/*s0*h.fits.Z <-- get all the raw sis0
high bit rate data
mget asca/data/rev1/60000000/raw/*g2*m.fits.Z <-- get all the raw gis2
medium bit rate data
mget asca/data/rev1/60000000/aux/*.mkf.Z <-- get all the mkf files
3.2 The HEASARC Browse
The HEASARC BROWSE facility can be used to find out which data are available,
and also to download the data. Browse can be accessed either via remote login
or via the WWW.
3.2.2 World Wide Web
To access the WWW browse interface go to
http://heasarc.gsfc.nasa.gov/W3Browse/
click on ASCA, and select Archival Data. Then simply type the name or
coordinates of the object. The hyperlinks then lead to the data, if its
available. Data can be downloaded either file by file, or as a tar file. The
gifs can be displayed, to give a quick look at the data.
3.3 NDADS Access
ASCA data are being archived at the National Space Science Data Center on
optical disk (WORM) platters. The ASCA/NDADS archive can be accessed by
submitting e-mail request or via the World Wide Web (WWW) to NDADS. Requests
for ASCA data on tape will also be supported.
3.3.1 ASCA Data on NDADS
ASCA data in the NSSDC Data Archive and Distribution System, NDADS are
organized by the 8-digit observation sequence number and sub-directory
structure described above. The sub-directories are referred to here as
"data-types" and are archived on NDADS in unix tar containers. When untar-ed,
the appropriate directory structure is reproduced [seq_num/data-type]. Each tar
file also contains the original bill-of-lading (BOL) file delivered to NSSDC
with each data-type by the ASCA project. The BOL file contains a list of the
individual files within each data-type and sequence number and several other
fields used to ingest the ASCA data to NDADS (see data-type `LOG' below). All
BOL files are in ascii format. In addition, some processing logs and tape
contents lists have also been archived in ASCII format.
Also archived on NDADS are six logs (listings) of the files within each tar
container. These logs are derived from the original BOL's from the project and
contain the following fields; original file name delivered by the project,
compressed byte count, decompressed byte count, original release date as
supplied by the ASCA project, an archive flag used to ingest the data, and the
object name. These logs are archived as ASCII files separate from the tar
container files.
3.3.2 NDADS E-Mail Requests
ASCA data can be requested electronically through the Automated Retrieval Mail
System (ARMS). For general information on NDADS send the following e-mail
message (nothing needs to be entered in the body of the message):
To: NDADSA::ARCHIVES (or archives@ndadsa.gsfc.nasa.gov)
Subject: HELP
To request specific information on the NDADS ASCA archive send the following
e-mail message (nothing needs to be entered in the body of the message):
To: NDADSA::ARCHIVES
Subject: HOLDINGS ASCA
NDADS requests for ASCA data should include the NDADS entry-ids (8-digit
observation sequence numbers), the data-type, and the project name (ASCA). A
listing of the ASCA/NDADS data-type's and sample file names are given in Table
4.1. An e-mail request to NDADS for data should use the following syntax:
To: NDADSA::ARCHIVES
Subject: REQUEST Project_Name Data_Type
In the body of the mail message list the sequence number(s) you wish to
request. A request for a particular object may also be submitted instead of
the 8-digit observation sequence number. If a particular object was observed
multiple times then more than one observation sequence will be returned.
For example, a request for the "SCREENED" tar file for sequence number 12345678
would look like:
To: NDADSA::ARCHIVES (or archives@ndadsa.gsfc.nasa.gov)
Subject: REQUEST ASCA SCREENED
Then in the body of the e-mail message enter:
12345678
<end of message>
More than one sequence number can be requested in a single archive request.
However, multiple requests must be made to retrieve more than one data-type.
3.3.3 Downloading Requested Data
After receipt of the e-mail message sent by the user, ARMS (Automated Retrieval
Mail System) will extract the data to the NDADS staging area or send the
requested data to your home computer if remote data transfer was requested.
ARMS sends an e-mail message notifying the user that the data are available and
includes a list of all the files retrieved. The archive user can then retrieve
the files via anonymous ftp from ndadsa.gsfc.nasa.gov.
The basic steps are:
a) connect to NDADSA via anonymous ftp. Give your e-mail address as the
password.
b) change directory to data_dist.asca
c) check the directory to make sure the expected files are there.
d) to ensure the proper transmission of the data, tar files should be sent in
binary mode and the BOL files should be sent in ASCII mode.
e) get the desired files.
3.3.4 Remote transfer of Data
ASCA data requests can be transferred directly to the requesters home computer.
This method avoids the need to place the data on the NDADS staging disk, but
requires the requester to supply remote node information. This feature is
available either through the e-mail interface or the WWW. To request data be
ftp-ed to your home computer send the following e-mail message:
To: NDADSA::ARCHIVES
Subject: REQUEST Project_Name Data_Type [Data_format [Remote_access]]
The syntax for the remote access information is:
Node_Name|User_Name|Password|Destination_Directory
Note that when requesting remote node data transfer, the Data_format field must
also be specified. Initially, for ASCA, the Data_format field should be set to
"None". Further details are contained in the NDADS help information
and/or via the WWW.
3.3.5 NDADS World Wide Web Interface
The ASCA data on NDADS and related information can be accessed via the WWW
using Mosaic. Requests for ASCA data utilize the Mosaic forms capability.
The URL for the ASCA NSSDC page is:
http://nssdca.gsfc.nasa.gov/asca/asca_nssdc.html
3.3.6 Restoring the Files on Your (UNIX) Computer
Place the ASCA tar files in the directory you wish. Then cd to this
directory and enter:
tar xvf asca_SequenceNum_data-type.tar
e.g.:
tar xvf asca_12345678_screened.tar
This will create a subdirectory named by the sequence number and under this
directory another subdirectory named by the `data-type' which, in turn,
contains all the component files. To decompress all the files cd to
the data-type subdirectory, e.g.:
cd 12345678
cd screened
uncompress *.Z
3.3.7 Tape Requests
Individuals who need ASCA data on magnetic tape should submit their requests to
the NSSDC/CRUSO office (address below). Requesters can receive the data on
4mm, 8mm or 9-track tape. Note that there is a significant charge for off-line
media but that this charge may be waived for NASA-funded researchers. Contact
the CRUSO office for the latest price lists. Note that off-line requests for
data will take substantially more time than direct requests to NDADS, so if
possible, requesters are urged to use one of the electronic request methods.
3.3.8 VMS Requests
ASCA researchers using the VMS operating system who experience problems with
detarring and/or decompressing the ASCA tar files should contact Dr. Nancy
Oliversen at the address below for assistance.
Table 1 ASCA/NDADS Data-types
NDADS
Data-type Component files Sample file names
AUX ASCA_nnnnnnnn_AUX.tar ASCA_12345678_AUX.tar
PRODUCT ASCA_nnnnnnnn_PRODUCT.tar ASCA_12345678_PRODUCT.tar
RAW ASCA_nnnnnnnn_RAW.tar ASCA_12345678_RAW.tar
SCREENED ASCA_nnnnnnnn_SCREENED.tar ASCA_12345678_SCREENED.tar
TELEM ASCA_nnnnnnnn_TELEM.tar ASCA_12345678_TELEM.tar
UNSCREENED ASCA_nnnnnnnn_UNSCREENED.tar ASCA_12345678_UNSCREENED.tar
LOG ascii log of files in each tar container.
ASCA_nnnnnnnn_RAW.BOL ASCA_12345678_RAW.BOL
ASCA_nnnnnnnn_TELEM.BOL ASCA_12345678_TELEM.BOL
ASCA_nnnnnnnn_SCREENED.BOL ASCA_12345678_SCREENED.BOL
ASCA_nnnnnnnn_UNSCREENED.BOL ASCA_12345678_UNSCREENED.BOL
ASCA_nnnnnnnn_AUX.BOL ASCA_12345678_AUX.BOL
ASCA_nnnnnnnn_PRODUCT.BOL ASCA_12345678_PRODUCT.BOL
where nnnnnnnnn = 8 digit observation sequence number
4 Contacts
If you have any questions, comments, suggestions or problems using the ASCA
archive please send e-mail to ascahelp@athena.gsfc.nasa.gov, or contact Eric
Gotthelf directly at 301-286-1587.
Users are welcome to visit the ASCA GOF to analyze both ASCA archival and guest
data. To do so please contact Karen Smale at ksmale@lheavx.gsfc.nasa.gov or
301-286-7612 to book the facility.
For information regarding the ASCA NDADS archive and obtaining tape copies of
data contact Dr. Nancy Oliversen (noliversen@nssdca.gsfc.nasa.gov or
301-286-1598). Tape copies can also be directly requested from NSSDC/CRUSO via
request@nssdca.gsfc.nasa.gov or 301-286-6695.
5 Acknowledgements
The creation of the ASCA archive has been a joint effort by the ASCA GOF in the
Laboratory for High Energy Astrophysics, and the ASCA Processing Team in the
Astrophysics Data Facility (ADF). We would like to thank Richard Fink, Don
Jennings, Ed Seufert, Ron Watson and Rich Pisarski for their efforts to make
sure the archive was released on time. We also thank the other ASCA GOF members
for their contributions and, of course, the entire ASCA team in Japan and the
US for making this mission so successful.
Proceed to the next article
Return to the previous article
Select another article
HEASARC Home |
Observatories |
Archive |
Calibration |
Software |
Tools |
Students/Teachers/Public
Last modified: Wednesday, 20-Oct-2021 10:49:49 EDT
The Astrophysics Science Division (ASD) at NASA's Goddard Space Flight Center (GSFC) seeks a creative, innovative individual with strong teamwork and leadership skills to serve as Director of the High Energy Astrophysics Science Archive Research Center (HEASARC). This will be a permanent civil servant position. + Learn more.
|