3. Suzaku Data Specifics and Conventions

This chapter describes the contents of Suzaku observation data sets, including the directory structure, data files, and the format of those files. The Suzaku data structure is similar to previous X-ray missions, with small variations.

3.1 Directory and Data File Structure

3.1.1 Retrieving the Data

3.1.1.1 Public Data Access

Public Suzaku data sets can be accessed through the HEASARC Browse interface at GSFC which can be found at http://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/w3browse.pl. They can also be retrieved using wget or FTP.

For more details also see the Suzaku Archive GOF web page at
http://heasarc.gsfc.nasa.gov/docs/suzaku/aehp_archive.html. The wget or public FTP download works identical to the proprietary data access (see next section), without the need for decrypting the downloaded files.

In addition users can access the data at the ISAS DARTS site (mainly intended for Japanese and European-based observers).

3.1.1.2 Proprietary Data Access - US PI Case

When the data are processed, the PI of the observation will receive an e-mail from the Suzaku GOF at GSFC giving the FTP location to access and download the data. For more information on the format of the location (presently
ftp://heasarc.gsfc.nasa.gov/suzaku/data/obs/M/NNNNNNNNN where M is a number indicating the type of target and NNNNNNNNN the sequence number of the data), please access the guide to the Suzaku archive at
http://heasarc.gsfc.nasa.gov/docs/suzaku/aehp_archive.html.

We recommend to use wget to retrieve the data^3.1:

wget --passive-ftp -q -nH --cut-dirs=5 -r 10 -c -N -np \
--retr-symlinks ftp_address_received

where the ftp_address_received is the location mentioned above:
ftp://heasarc.gsfc.nasa.gov/suzaku/data/obs/M/NNNNNNNNN/
Note that the ``/'' at the end of the command is required.

Alternatively, this can be done using the following FTP commands:

 ftp heasarc.gsfc.nasa.gov
 login:  anonymous
 password : your_email_address@your_domain_address
 ftp> cd suzaku/data/obs/M
 ftp> binary
 ftp> get NNNNNNNNN.tar.gz
 ftp> quit

Once retrieved, they need to be decrypted using either PGP or GPG software and a perl script available at the website http://heasarc.gsfc.nasa.gov/docs/cookbook/decrypt_data.pl. General information on how to decrypt the data is available at:
http://heasarc.gsfc.nasa.gov/docs/cookbook/decrypt.html.

The decryption keys for Suzaku data are 32 or 34 characters long and sometimes include special characters. We therefore recommend against specifying the key on the command line. Also, with gpg, this process will leave both the encrypted and decrypted versions of the files in your data directory. You therefore need to make sure you have adequate (original $\times$2) disk space. Finally, glitches during download can prevent decryption. If an initial attempt fails, re-downloading the data set may be all that is required to successfully decrypt the data.

3.1.2 Data Organization

All Suzaku data (including ground calibration and test data) have unique 9-digit sequence numbers (e.g. 900000450) which is used as the name of the top level directory. Under this directory are a series of sub-directories, each of which carries a particular kind of data file, as explained below. All the data files are in the standard FITS format, although some output products are in Postscript, HTML, GIF or simple ASCII. The subdirectories are:

auxil

Auxiliary files not associated with a particular instrument, such as the spacecraft attitude (file named aeNNNNNNNNN.att - see Section 3.2 for an explanation of the name structure) and the orbit file (file named aeNNNNNNNNN.orb). The most important of these is the ``filter file'' (with the suffix ``mkf''), in which various satellite and instrumental parameters to be used for data screening are recorded as a function of time.

log

Log files from the pipeline processing.

hxd

Data from the Hard X-ray Detector (HXD).

xis

Data from the X-ray Imaging Spectrometers (XIS).

Within each of the two instrumental directories (hxd, xis) there are four subdirectories:

hk

Instrumental housekeeping files containing information such as voltages, temperatures and other detector-specific data.

event_uf

Second FITS Files (SFF); these are unfiltered events files derived from the First FITS Files (FFF). FFF are effectively the telemetry data converted into FITS format.

event_cl

Cleaned events in this directory have gone through the standard cuts (grades, SAA and such) and they are in principle directly useful for analysis. However, users can re-run these cleaning processes (see chapters 6 and 7 for more on the standard cuts applied).

products

Output products from the pipeline, such as GIF images of the data and automatically generated lightcurves.

The filename conventions in each of these directories are instrument dependent, as described in the next section.

3.2 Filenames

The filenames (except for some log files) use the following general convention:

aeNNNNNNNNNiii_n_mmmmmmmm_ll.ext.gz

where

ae

is short for Astro-E2, the initial name of Suzaku.

NNNNNNNNN

is the observation sequence number and is identical to the directory name.

iii

is the instrument specification. This string is set as follows: hxd=HXD, xi[0-3]=XIS-[0-3]. xis is used for files common to all the XIS units. This string can be omitted in files under the auxil and log directories.

n

ranges from 0 to 9 and indicates the RPT file number. The original telemetry file is divided into RPT files and more than one RPT can contribute to one observation. The value of 0 is used when the science file combines data from different RPT or if there is only one RPT file that contributes to that sequence. This number can be omitted in files under the auxil and log directories.

mmmmmmmm

is the file identifier. The string distinguishes between files from the same instrument.

ll

indicates the file level. For event files, the string can be ``uf'' or ``cl'' to indicate ``unfiltered'' or ``cleaned'' event files. It also can be ``bg,'' ``sk,'' ``sr,'' ``gso,'' ``pin,'' ``wel'' (products directory for both the XIS and HXD) or ``wam'' ( hk directory for the HXD). The string can be omitted.

ext

is the file extension. Currently it can take the values: ``evt'' (event files), ``gti'' (good time interval), ``hk'' (house keeping), ``ghf'' (gain history file), ``ght'' (gain history table), ``lc'' (light curve), ``pi'' (pulse invariant), ``html,'' ``log,'' ``com,'' ``att'' (attitude file), ``cat,'' ``ehk,'' ``mkf,'' ``orb,'' ``tim,'' ``img,'' and ``gif.''

For more information on file names of the products of the pipeline processing, please refer to the documentation that can be found at
http://heasarc.gsfc.nasa.gov/docs/suzaku/aehp_data_analysis.html.

3.3 Suzaku Coordinates

The XIS is an imaging instrument (unlike the HXD), and the coordinate values in XIS files indicate the pixel center positions. The XIS coordinate systems are described below:

Sky coordinates

``X'' and ``Y'' are used to describe the sky positions of the events relative to a celestial reference point. The tangential projection is used, and north is defined up (increasing Y), and east is left (decreasing X). ``X'' and ``Y'' columns are computed using attitude information.

Focal plane coordinates

These are the event locations on the focal plane, which is common to the four (there are four XIS detectors) imaging instruments. ``FOCX'' and ``FOCY'' event file columns are used. The FOC coordinates differ from the Sky images in that the satellite attitude is not considered in the former. FOC images of the four instruments should match, as instrument misalignments are already taken into account.

Detector coordinates

These give the physical positions of the pixels within each sensor. Misalignments between the sensors are not taken into account. The DET X and Y values take 1 to 1024 for XIS. The XIS DETX/Y pixels correspond to the actual 1024x1024 CCD pixels, and the DETX/Y pixel size is the same as the CCD physical pixel size. The DET images will give correct sky images of the objects (not mirrored images), except that attitude wobbling is not taken into account. Note that X-ray images focused by the mirrors and detected by the focal plane instruments will be the mirror images, which have to be flipped to be the actual images of celestial objects. Thus, the original look-down images are flipped (and rotated if necessary) so that the satellite +Y-axis direction will be the DETY direction.

ACT and RAW coordinates

The ACT coordinates are used to tell actual pixel locations on the chip. Each XIS chip is composed of the four segments, and the RAW coordinates are the pixel locations on each segment. Note that the XIS-0 and XIS-3 installations on the baseplate are aligned, whereas XIS-1 and XIS-2 are 90 degrees rotated relative to them, in opposite directions respectively. Therefore the relation between ACT and DET coordinates is dependent on each XIS sensor.^3.2

Table 3.1: Types of coordinates and coordinate related variables and their possible values.

Type		Type	Minimum	Maximum	Origin	Unit
Sky	X/Y	Integer	1	1536	768.5	$0.0174'$
	ROLL	Real	0.0	360.0	-	degree
FOC	X/Y	Integer	1	1536	768.5	$0.0174'$
DET(XIS)		Integer	1	1024	512.5	0.024mm
ACT	X/Y	Integer	0	1023	-	-
SEGMENT		Integer	0	3	-	-
RAWX(XIS)		Integer	0	255	-	-
RAWY(XIS)		Integer	0	1023	-	-

3.4 Photon Energies and Pulse Heights

All Suzaku instruments are energy-sensitive, and each event has a measured ``Pulse Height Amplitude'' (PHA). The PHA may be both position- and time-varying, depending upon the instrument. Therefore, a calculated ``PHA Invariant'' (PI) value is also determined using the PHA in combination with the instrumental calibration and gain drift. In all cases, the PI columns should be used to extract energy spectra, or to produce energy-band selected images or light curves. For reference, the approximate relationship between ``true'' X-ray energy E and the event PI is shown below for each instrument. The exact relationship between energy and PI is given in the second extension of the instrument response matrix file, or ``RMF.''

XIS

The PI column name is ``PI'', which takes values from 0 to 4095. The PI vs. energy relationship is the following: $E$ [eV] = $3.65\times$PI [channel].

HXD

The ``PI_SLOW'' column (as opposed to ``PI_FAST''), which takes values from 0 to 511, should be used for GSO spectral analysis. The PI vs. energy relationship is the following: $E$ [keV] =2$\times$(PI_SLOW + 1.0), where $E$ is the center of the bin. For PIN spectral analysis, the ``PI_PIN'' column which takes values from 0 to 255, should be used. The value in this column is copied from the PI column of the triggered PIN, which is one of the PI_PIN0, PI_PIN1, PI_PIN2 or PI_PIN3. The PI vs. energy relationship is the following: $E$ [keV] =0.375$\times$(PI_PIN + 1.0), again $E$ is the center of the bin.

3.5 Timing Information

The Suzaku event arrival time is represented by the ``Suzaku time'', which is defined as the elapsed time in seconds from the beginning of the year 2000 (January 1st, 00:00:00.000) in UTC (when TAI is 32 seconds ahead). There will always be a constant offset between TT and Suzaku time, and this is reflected in the time-related keywords. These and other systems of time are documented at:
http://heasarc.gsfc.nasa.gov/docs/xte/abc/time_tutorial.html.

The event time resolution of each detector as follows:

XIS

In the Normal observation modes (5x5, 3x3, or 2x2) without a Window option, the time resolution is 8sec, corresponding to a single frame exposure. The event time assigned is the midpoint of the exposure frame. When the Window option is used, depending on its size, the time resolution will be 2s (1/4 Window), or 1s (1/8 Window). In Timing mode, the time resolution is 7.8125ms, regardless of the number of lines to be combined (either 64, 128, or 256). Users should note that when combining a small number of lines, there could be a noticeable amount of cross-talk between one time bin and the next, due to the wings of the PSF. For example, 64 lines is only about 1.2arcmin, so a fraction of the source counts will fall on the neighboring groups of 64 lines, and so be mis-time-tagged by +/-N times 7.8125ms. For this reason, it may be safer to always use a grouping of 256 lines.

HXD

The nominal time resolution is $61 \mu$s, which corresponds to the HXD_WPU_CLK_RATE_HK parameter = 1 (Fine). A higher time resolution, $30.5 \mu$s is possible within the requirements of satellite operations, in which case HXD_WPU_CLK_RATE_HK will be 2 (Super-Fine), although this is not user-selectable at this time.

3.6 Suzaku Telemetry

3.6.1 Data Rates

The telemetry rate determines the data transfer rate from the on�board instruments to the Data Recorder. Being limited by the data storage and downlink capacity, the highest data rates may not be used all the time^3.3. Basically, a combination of the following three telemetry rates will be used for observations: High rate (261kbps), Medium rate (131kbps), or Low rate (33kbps) ^3.4. Among the 10 Gbit raw data per day, 4 Gbits will be taken between the contacts (contact passes) with High and Medium bitrates, and 6 Gbits will be taken after the contacts (remote pass) using Medium and Low bitrates.

3.6.2 Allocations

Although the maximum Data Recorder recording rate is limited by the telemetry rate for each bitrate, allocation of the telemetry to various instruments is variable. The XIS and HXD telemetry limits depend on the bitrates.

3.6.3 Telemetry Limits

XIS

The approximate XIS telemetry limits (events/s per XIS unit) - as in use since the loss of XIS2 in 2006 November - for different bitrates and editing modes is given in Table 3.2.

Table 3.2: Telemetry limits (in events/s per XIS unit) - as in use since the loss of XIS2 in 2006 November - for different XIS editing modes.

	Telem. limit	5x5	3x3	FI 2x2 / BI 3x3	Timing
	[kbps/XIS]	s$^{-1}$	s$^{-1}$	s$^{-1}$/s$^{-1}$	s$^{-1}$
Super-High	144	138.24	291.03	341.33/485.05	1382.40
High	144	138.24	291.03	341.33/485.05	1382.40
Medium (weekday)	60	57.60	121.26	142.22/202.11	576.00
Medium (weekend)	25	24.00	50.53	59.26/84.21	240.00
Low (weekday)	15	14.40	30.32	35.56/50.53	144.00
Low (weekend)	1	0.96	2.02	2.37/0.76	9.60

These numbers do not include overheads (telemetry header and HK) and background event rates (FI$\sim$10/s/sensor; BI$\sim$20/s/sensor). Allow for a 10% margin.
Modification of telemetry saturation limit by window/burst options:
1. No window $+m$s burst option - telemetry limit increases $8/m$ times.
2. $1/n$ window $+m$s burst option - telemetry limit increases $8/m/n$ times.
3. Area discrimination will further increase the telemetry limit.

XIS events are compressed on-board and actual telemetry limits may vary within $\sim \pm40$% depending on the PHA values. Note that different XIS sensors may be operated using different modes and telemetry allocations.

HXD

The approximate HXD Well telemetry limits will be the following (in counts/s): Super-High=1150, High=550, Medium=250, and Low=30. This is based on the assumption that HXD will take 30% of the telemetry. Note that the Crab rate in the HXD is $\sim 200$cts/s.

3.7 XSELECT Default Parameters

The XSELECT behavior for each mission is determined by the mission database file, usually located at $FTOOLS/bin/xselect.mdb^3.5. The Suzaku entries in the mission database files enable the following:

• Common for all the instruments:
  • Default light curve bin size is 16sec.

  • ``extractor'' is used to extract products.

  • WMAP^3.6 is created as the spectral file header.

  • Default image coordinates are Sky coordinates (X and Y).

  • Default WMAP coordinates are Detector coordinates (DETX and DETY).

  • Event file has one of the following names; ae*xis0*.evt, ae*xis1*.evt, ae*xis2*.evt, ae*xis3*.evt, or ae*hxd wel*.*.

  • The filter file has the name ae*mkf*, and is in the directory ../../auxil relative to the event file directories^3.7

• XIS:
  • Default image binning is 8.

  • Default WMAP binning is 8.

  • ``RAWX'' and ``RAWY'' coordinates are set to ``ACTX'' and ``ACTY'', so the ``set image raw'' command creates ACT coordinate images.

  • Pixels in the WMAP outside of the selected region will have the value ``-1''

  • Spawns ``grppha'' when saving a spectral file, three speed options - ``fast'', ``medium'', ``slow'' - are available. ``Medium'', the default, results in the following channel binning: 1$\times$ from channel 0 to 699, 2$\times$ from 700 to 2695, and 4$\times$ from 2696 to 4095. Channels 0-81 and 1645-2047 are flagged as ``bad'' (original PI channels 0-81 and 3290-4095).

  • Spawns ``xisrmfgen'', ``xissimarfgen'', ``marfrmf'', and, optionally, ``rbnrmf'' when saving a spectral response file, with the three speed options mentioned above. Again, ``Medium'' is the default.

• HXD:
  • The default energy column to make energy spectra is ``PI_PIN'' for a PIN event file and ``PI_SLOW'' for a GSO event file.

  • The UNITID event file column is used in lieu of standard X, Y, RAWX, RAWY and DETX of imaging instruments, so that the ``sky'' or ``raw'' images will be a pseudo-diagonal image of UNITID^3.8.

  • The PIN_ID event file column (defined only for DET_TYPE = 1) is used in lieu of DETY, so that the WMAP is created with UNITID vs. PIN_ID, which is useful when creating ARFs and RMFs for the PIN.

  • No binning for ``image'' and WMAP.

  • PIN and GSO spectra are saved with the original number of channels, i.e., 256 and 512, respectively.

Footnotes

... data^3.1

wget is available at: http://www.gnu.org/software/wget/wget.html

... sensor.^3.2

Conversion from the RAW to ACT coordinates is not straightforward, because of the particular order of the pixel read-out and possible use of the Window option.

... time^3.3

The amount of the data taken per day is mainly limited by the capacity of the Data Recorder (6 Gbits) and the downlink rate at Uchinoura Space Center (2 Gbits/ground contact). There will be 5 ground contacts per day separated by 90 minutes, so not more than 10 Gbits/day raw data can be taken, with typical daily sizes being considerably smaller than this.

... (33 kbps)^3.4

In addition, there is a Super-High rate (524 kbps) which is not used for general observations.

... \$FTOOLS/bin/xselect.mdb^3.5

Users may specify their own mission database file with an environmental parameter XSELECT_MDB.

... WMAP^3.6

WMAP is the part of the detector image from which the energy spectrum has been extracted, and will be used to create spectral responses by downstream FTOOLS.

... directories^3.7

For Suzaku FTOOLS Version 13 and earlier the filter file has to be uncompressed or the file name has to be explicitly specified.

... UNITID^3.8

For each HXD event, UNITID and DET_TYPE tell the Well unit-ID and the detector type, respectively. UNITID takes a value in the range of 0 to 15 corresponding to the 16 Well units. DET_TYPE = 0 corresponds to the GSO, and DET_TYPE=1 to the PIN.