next up previous contents
Next: 4. Suzaku Data Analysis Up: Suzaku ABC Guide Previous: 2. Software   Contents

Subsections


3. Suzaku Data Specifics and Conventions

This chapter describes the contents of Suzaku observation data set, including the directory structure, data files, and the format of those files. The Suzaku data structure is similar to previous X-ray missions, with small variations.

3.1 Directory and Data File Structure

The standard Suzaku ``pipeline processing'' products (encrypted for proprietary data) are available from the GSFC HEASARC archive, as explained at
http://suzaku.gsfc.nasa.gov/docs/suzaku/aehp_archive.html. Users can also access the data at the ISAS DARTS site (for Japanese and European-based observers). Standard data formatting and calibration are carried out in the pipeline processing, and all Suzaku users should start scientific data analysis from the pipeline processing products.

3.1.1 Retrieving the data

This section is relevant for US PIs only

When the data are processed, the PI of the observation will receive an e-mail from the Suzaku GOF at GSFC giving the FTP location to access and download the data. For more information on the format of the location (presently
ftp://legacy.gsfc.nasa.gov/suzaku/data/obs/M/NNNNNNNNN where M is a number indicating the type of target and NNNNNNNNN the sequence number of the data), please access the guide to the Suzaku archive at
http://suzaku.gsfc.nasa.gov/docs/suzaku/aehp_archive.html.

There are two options available for the download: FTP and wget.
To retrieve the data via FTP type: 

 ftp legacy.gsfc.nasa.gov 
 login:  anonymous
 password : your_email_address@your_domain_address
 ftp> cd suzaku/data/obs/M
 ftp> binary
 ftp> get NNNNNNNNN.tar.gz
 ftp> quit
To retrieve the data via wget3.1, type: 
wget --passive-ftp -q -nH --cut-dirs=5 -r 10 -c -N -np \
--retr-symlinks ftp_address_received
where the ftp_address_received is the location mentioned above:
ftp://legacy.gsfc.nasa.gov/suzaku/data/obs/M/NNNNNNNNN.

Once retrieved, the need to be decrypted using either PGP or GPG software and a perl script available at the website http://suzaku.gsfc.nasa.gov/docs/cookbook/decrypt_data.pl. General information on how to decrypt the data is available at:
http://suzaku.gsfc.nasa.gov/docs/cookbook/decrypt.html.

The decryption keys for Suzaku data are always 34 characters long and sometimes include special characters. We therefore recommend against specifying the key on the command line. Also, with gpg, this process will leave both the encrypted and decrypted versions of the files in your data directory. You therefore need to make sure you have adequate (original $\times$2) disk space. Finally, glitches during download can prevent decryption. If an initial attempt fails, re-downloading the data set may be all that is required to successfully decrypt the data.

3.1.2 Organization of the data

All Suzaku data (including ground calibration and test data) have unique 9-digit sequence numbers (e.g. 900000450) which is used as the name of the top level directory. Under this directory are a series of sub-directories, each of which carries a particular kind of data file, as explained below. All the data files are in the standard FITS format, although some output products are in Postscript, HTML, GIF or simple ASCII3.2. The subdirectories are:
auxil
Auxiliary files not associated with a particular instrument, such as the spacecraft attitude (file named aeNNNNNNNNN.att - see Section 3.2 for an explanation of the name structure) and the orbit file (file named aeNNNNNNNNN.orb). The most important of these is the ``filter file'' (with the suffix ``mkf''), in which various satellite and instrumental parameters to be used for data screening are recorded as a function of time.
log
Log files from the pipeline processing.
hxd
Data from the Hard X-ray Detector (HXD).
xis
Data from the X-ray Imaging Spectrometers (XIS).

Within each of the two instrumental directories (hxd, xis) there are four subdirectories:

hk
Instrumental housekeeping files containing information such as voltages, temperatures and other detector-specific data.
event_uf
Second FITS Files (SFF); these are unfiltered events files derived from the First FITS Files (FFF). FFF are effectively the telemetry data converted into FITS format
event_cl
Cleaned events in this directory have gone through the standard cuts (grades, SAA and such) and they are in principle directly useful for analysis. However, users can re-run these cleaning processes (see Chapters 6 and 7 for more on the standard cuts applied).
products
Output products from the pipeline, such as GIF images of the data and automatically generated lightcurves.

The filename conventions in each of these directories are instrument dependent, as described in the next section.


3.2 Filenames

The filenames (except for some log files) use the following general convention:
aeXXXXXXXXXiii_N_mmmmmmmm_ll.ext.gz
where
ae
is short for Astro-E2 the initial name of Suzaku.
XXXXXXXXX
is the observation sequence number and is identical to the directory name.
iii
is the instrument specification. This string is set as follows: hxd=HXD, xi[0-3]=XIS-[0-3]. xis is used for files common to all the XIS units. This string can be omitted in files under the auxil and log directories.
N
ranges from 0 to 9 and indicates the RPT file number. The original telemetry file is divided into RPT files and more than one RPT can contribute to one observation. The value of 0 is used when the science file combines data from different RPT or if there is only one RPT file that contributes to that sequence. This number can be omitted in files under the auxil and log directories.
mmmmmmmm
is the file identifier. The string distinguishes between files from the same instrument.
ll
indicates the file level. For event files, the string can be ``uf'' or ``cl'' to indicate ``unfiltered'' or ``cleaned'' event files. It also can be ``bg,'' ``sk,'' ``sr,'' ``gso,'' ``pin,'' ``wel'' (products directory for both the XIS and HXD) or ``wam'' ( hk directory for the HXD). The string can be omitted.
ext
is the file extension. Currently can take the values: ``evt'' (event files), ``gti'' (good time interval), ``hk'' (house keeping), ``ghf'' (gain history file), ``ght'' (gain history table), ``lc'' (light curve), ``pi'' (pulse invariant), ``html,'' ``log,'' ``com,'' ``att'' (attitude file), ``cat,'' ``ehk,'' ``mkf,'' ``orb,'' ``tim,'' ``img,'' and ``gif.''

For more informations on file names of the products of the pipeline processing, please refer to the documentation that can be found at
http://suzaku.gsfc.nasa.gov/docs/suzaku/aehp_data_analysis.html.


3.3 Suzaku Coordinates

The XIS is an imaging instrument (unlike the HXD), and the coordinate values in XIS files indicate the pixel center positions. The XIS coordinate systems are described below:
Sky coordinates
``X'' and ``Y'' are used to describe the sky positions of the events relative to a celestial reference point. The tangential projection is used, and north is defined up (increasing Y), and east is left (decreasing X). ``X'' and ``Y'' columns are computed using attitude information.
Focal plane coordinates
These are the event locations on the focal plane, which is common to the four (there are four XIS detectors) imaging instruments. ``FOCX'' and ``FOCY'' event file columns are used. The FOC coordinates differ from the Sky images in that the satellite attitude is not considered in the former. FOC images of the four instruments should match, as instrument misalignments are already taken into account.
Detector coordinates
These give the physical positions of the pixels within each sensor. Misalignments between the sensors are not taken into account. The DET X and Y values take 1 to 1024 for XIS. The XIS DETX/Y pixels correspond to the actual 1024x1024 CCD pixels, and the DETX/Y pixel size is the same as the CCD physical pixel size. The DET images will give correct sky images of the objects (not mirrored images), except that attitude wobbling is not taken into account. Note that X-ray images focused by the mirrors and detected by the focal plane instruments will be the mirror images, which have to be flipped to be the actual images of celestial objects. Thus, the original look-down images are flipped (and rotated if necessary) so that the satellite +Y-axis direction will be the DETY direction.
ACT and RAW coordinates
The ACT coordinates are used to tell actual pixel locations on the chip. Each XIS chip is composed of the four segments, and the RAW coordinates are the pixel locations on each segment . Note that the XIS-0 and XIS-3 installations on the baseplate are aligned, whereas XIS-1 and XIS-2 are 90 degrees rotated relative to them, in opposite directions respectively. Therefore the relation between ACT and DET coordinates is dependent on each XIS sensor.3.3


Table 3.1: Types of coordinates and coordinate related variables and their possible values
Type   Type Minimum Maximum Origin Unit
Sky X/Y Integer 1 1536 768.5 $0.0174'$
  ROLL Real 0.0 360.0 - degree
FOC X/Y Integer 1 1536 768.5 $0.0174'$
DET(XIS)   Integer 1 1024 512.5 0.024 mm
ACT X/Y Integer 0 1023 - -
SEGMENT   Integer 0 3 - -
RAWX(XIS)   Integer 0 255 - -
RAWY(XIS)   Integer 0 1023 - -


3.4 Photon Energies and Pulse-heights

All Suzaku instruments are energy-sensitive, and each event has a measured ``Pulse Height Amplitude'' (PHA). The PHA may be both position- and time-varying, depending upon the instrument. Therefore, a calculated ``PHA Invariant'' (PI) value is also determined using the PHA in combination with the instrumental calibration and gain drift. In all cases, the PI columns should be used to extract energy spectra, or to produce energy-band selected images or light curves. For reference, the approximate relationship between ``true'' X-ray energy E and the event PI is shown below for each instrument. The exact relationship between energy and PI is given in the second extension of the instrument response matrix file, or ``RMF.''

XIS
The PI column name is ``PI'', which takes values from 0 to 4095. The PI vs. energy relationship is the following: E [eV] = $3.65\times$PI [channel].
HXD
The ``PI_SLOW'' column (as opposed to ``PI_FAST''), which takes values from 0 to 4095, should be used for GSO spectral analysis. The PI vs. energy relationship is the following: E [keV] =2$\times$(PI_SLOW + 0.5). For PIN spectral analysis, the ``PI_PIN'' column which takes values from 0 to 255, should be used. The value in this column is copied from the PI column of the triggered PIN, which is one of the PI_PIN0, PI_PIN1, PI_PIN2 or PI_PIN3. The PI vs. energy relationship is the following: E [keV] =0.375$\times$(PI_PIN + 1.0).

3.5 Timing Information

The Suzaku event arrival time is represented by the ``Suzaku time,'' which is defined as the elapsed time in seconds from the beginning of the year 2000 (January 1st, 00:00:00.000) in UTC (when TAI is 32 seconds ahead). There will always be a constant offset between TT and Suzaku time, and this is reflected in the time-related keywords. These and other systems of time are documented at:
http://heasarc.gsfc.nasa.gov/docs/xte/abc/time_tutorial.html.

The event time resolution of each detector as follows:

XIS
In the Normal observation modes (5x5, 3x3 or 2x2) without a Window option, the time resolution is 8 sec, corresponding to a single frame exposure. The event time assigned is the midpoint of the exposure frame. When the Window option is used, depending on its size, the time resolution will be 4 s (1/2 Window), 2 s (1/4 Window), or 1 s (1/8 Window). In Timing mode, the time resolution is 7.8125 ms, regardless of the number of lines to be combined (either 64, 128 or 256). Users should note that when combining a small number of lines, there could be a noticeable amount of cross-talk between one time bin and the next, due to the wings of the PSF. For example, 64 lines is only about 1.2 arcmin, so a fraction of the source counts will fall on the neighboring groups of 64 lines, and so be mis-time-tagged by +/-N times 7.8125 ms. For this reason, it may be safer to always use a grouping of 256 lines.

HXD
Nominal time resolution is $61\mu$s, which corresponds to the HXD_WPU_CLK_RATE_HK parameter = 1 (Fine). A higher time resolution, $30.5\mu$s is possible by commands, in which case HXD_WPU_CLK_RATE will be 2 (Super-Fine), although this is not user-selectable at this time.

3.6 Suzaku Telemetry

3.6.1 Data rates

The telemetry rate determines the data transfer rate from the on­board instruments to the Data Recorder. Being limited by the data storage and downlink capacity, the highest data rates may not be used all the time3.4. Basically, a combination of the following three telemetry rates will be used for observations; High rate (262 kbps), Medium rate (131 kbps), or Low rate (33 kbps)3.5. Among the 10 Gbit raw data per day, 4 Gbits will be taken between the contacts (contact passes) with High and Medium bitrates, and 6 Gbits will be taken after the contacts (remote pass) using Medium and Low bitrates.

3.6.2 Allocations

Although the maximum Data Recorder recording rate is limited by the telemetry rate for each bitrate, allocation of the telemetry to various instruments is variable. The XIS and HXD telemetry limits will be dependent on the bitrates.

3.6.3 Telemetry Limits

XIS
The approximate XIS telemetry limits (events/s for four XIS combined) for different bitrates and observational modes will be the following:


Table 3.2: Telemetry limits (in events/s) in different XIS modes
  5x5 3x3 2x2 Timing
Super-High 985 1971 3942 9381
High 475 949 1899 4528
Medium 221 441 883 2114
Low 29 58 116 292


XIS events are compressed on-board and actual telemetry limits may vary within $\sim \pm40$% depending on the PHA values. Note that different XIS sensors may be operated using different modes and telemetry allocations.

HXD
The approximate HXD Well telemetry limits will be the following (in counts/s): Super-High=1150, High=550, Medium=250, and Low=30. This is based on the assumption that HXD will take 30% of the telemetry. Note that the Crab rate in the HXD is $\sim 200$cts/s.

3.7 xselect Default Parameters

XSELECT behavior for each mission is determined by the mission database file, usually located at $FTOOLS/bin/xselect.mdb3.6. The Suzaku entries in the mission database files enable the following:


Footnotes

...wget3.1
wget is available at: http://www.gnu.org/software/wget/wget.html
... ASCII3.2
In the early stage of the mission, some calibration files may be ASCII files but these will eventually be converted into FITS format.
... sensor.3.3
Conversion from the RAW to ACT coordinates is not straightforward, because of the particular order of the pixel read-out and possible use of the Window option.
... time3.4
The amount of the data taken per day is mainly limited by the capacity of the Data Recorder (6 Gbits) and the downlink rate at Uchinoura Space Center (2 Gbits/ground contact). There will be 5 ground contacts per day separated by 90 minutes, so it is expected that usually 10 Gbits/day raw data will be taken.
... kbps)3.5
In addition, there is Super-High rate (524 kbps), which may not be allowed for general observations.
... \$FTOOLS/bin/xselect.mdb3.6
Users may specify their own mission database file with an environmental parameter XSELECT_MDB.
... WMAP3.7
WMAP is the part of the detector image from which the energy spectrum has been extracted, and will be used to create spectral responses by downstream FTOOLS.
... UNITID3.8
For each HXD event, UNITID and DET_TYPE tells the Well unit-ID and the detector type. UNITID takes a value in the range of 0 to 15 corresponding to the 16 Well units. DET_TYPE = 0 corresponds to GSO, and 1 to 4 correspond to PIN0 to 3 respectively.
next up previous contents
Next: 4. Suzaku Data Analysis Up: Suzaku ABC Guide Previous: 2. Software   Contents
Michael Arida 2008-01-11