next up previous contents
Next: 5. XRISM Data Analysis Up: xrism abc Previous: 3. Analyses Tools and   Contents

Subsections


4. XRISM Data Specifics and Conventions

This chapter describes the XRISM observation datasets, including the directory structure, data files, and their formats. The data format and structure are similar to some X-ray observatory mission data (e.g., ASCA, Suzaku) with minor variations.


4.1 Data Organization

Unique 9-digit sequence numbers are used to organize the XRISM observations. A sequence number is assigned for data obtained during an observation and during the slew from the previous observation. The initial data processing at JAXA and NASA, hereafter pipeline, collects all relevant data of an observation, calibrates and screens the data, makes standard analysis for quick looks, and stores all products in a package with a sequence number. NASA and JAXA have their own data archives open to the public through the following URLs.

NASA: https://heasarc.gsfc.nasa.gov/FTP/xrism/data/obs/
JAXA: https://data.darts.isas.jaxa.jp/pub/xrism/data/obs/
Users can directly access these URLs to download any packages, but they can find the desired datasets more easily through the archival portal sites (see Section 5.2).


4.2 Directory and Data File Structure

A XRISM data package is stored under a directory with its sequence number name. There are four directories under the top directory.

auxil
contains auxiliary files not associated with a particular instrument, such as the attitude, orbit, filter, and satellite housekeeping files. These files help assess the satellite condition during the observation. Users may need to explicitly specify some of these files for running downstream software (see Section 5.7),
log
contains log files of the pipeline processing,
resolve
contains Resolve related data files, and
xtend
contains Xtend related data files.

Each of the Resolve and Xtend directories has the following four sub-directories.

hk
contains instrument housekeeping data: voltages, temperatures, and other instrument-specific information,
event_uf
contains all X-ray event data down-linked from the observatory and calibrated, so-called unfiltered event files,
event_cl
contains X-ray event data filtered from unfiltered event data using the standard screening criteria, so-called cleaned event files, and
products
contains quick-look analysis products (images, light curves & spectra) processed with the pipeline. These products are not suitable for science data analysis.

All science and HK files are in FITS format, while processing logs are in plain text.


4.3 Filenames

Users may use the following files in downstream software commands.

attitude auxil/xaOBSID .att
filter auxil/xaOBSID .ehk
orbit auxil/xaOBSID .orb
unfiltered event (resolve or xtend)/event_uf/xaOBSID iii_P0mmmmmm_uf.evt
cleaned event (resolve or xtend)/event_cl/xaOBSID iii_P0mmmmmm_cl.evt

where

xa
short term of xarm, the start-up name of the XRISM project,
OBSID
9-digit sequence number,
iii
instrument name abbreviation - rsl: Resolve, xtd: Xtend, gen: all instruments,
P
observation mode - p: pointing, s: slew, a: all (p+s),
m$\times N$
instrument mode. ($N$ =6 for Resolve, =8 for Xtend)

The instrument mode for the Resolve datasets describes the X-ray filter used during the Resolve observation. The former 3 digits of the instrument mode for the Xtend data describe the CCD window/burst mode used for the Xtend observation.

Resolve
Sub-string X-ray Filter
px0000 Undefined
px1000 OPEN
px2000 Al/Polyimide
px3000 Neutral Density (ND)
px4000 Be
px5000 Fe 55 calibration source
Xtend
Sub-string CCD Window/Burst Mode
300 all four CCDs in full window mode
311 CCD1 & CCD2 in 1/8 window mode
312 CCD1 & CCD2 in full window + 0.1 sec burst mode
313 CCD1 & CCD2 in 1/8 window + 0.1 sec burst mode
320 CCD3 & CCD4 in full window mode

All files are compressed in gzip, so they have extra .gz extensions. FTOOLS software reads gzipped files, so users do not need to unzip them.


4.4 Unfiltered or Screened?

The pipeline produces two types of event files for each instrument: the 'unfiltered' file (*_uf.evt) under the event_uf directory and the 'cleaned' file (*_cl.evt) under the event_cl directory. The unfiltered files contain all events telemetered from the observatory; the events are calibrated but not screened. The initial pipeline processing at the SDC or the xapipeline reprocessing tool feeds these unfiltered files, filters events that satisfy specific conditions (e.g., event type, grades) or time intervals (e.g., outside of South Atlantic Anomaly) and outputs 'cleaned' event files. The initial pipeline processing and a xapipeline reprocessing with the default setup apply the minimal set of filters to unfiltered data for screening, and the resulting cleaned event files can be used for any type of standard analysis, including bright sources. The minimal screening criteria are defined in Table 5.1 for the Resolve data and in Table 5.2 for the Xtend data. General users should always start from cleaned event files for science analysis. Users can adjust the screening conditions to optimize for their scientific goals following the guidance of Section 6.4 for Resolve and Section 7.4 for Xtend.


4.5 Important Reference Values


4.5.1 Image Coordinates

The pipeline uses various coordinate systems (RAW, ACT, DET, FOC, SKY) to map detected event locations on the detectors to celestial coordinates. Event files store all coordinate values for each event, but general users need only the following coordinates for science analysis. See also Fig. 5.1 for Resolve and Fig. 6.2 for Xtend in POG.

RA/DEC:
the celestial right ascension and declination coordinate system in $J2000$ (e.g., FK5, ICRS). XRISM event files do not store these coordinates for each event, but HEASoft tools can derive them from SKY coordinates using the relation described below.
SKY:
the X/Y coordinates with the same orientation as the RADEC coordinate system — declination ($\delta$) increases in the +Y direction and Right Ascension ($\alpha$) increases in the -X direction — incremented every $\sim$1.77 arcsec. Their origin is defined for each dataset in the HEADER keywords, using RADEC reference coordinates, ($\alpha$, $\delta$) = (TCRVL31, TCRVL32) for Xtend and (TCRVL41, TCRVL42) for Resolve , with the same values between the instruments for the same observation. A detector pixel is normally mapped onto multiple SKY pixels, so the software picks a SKY pixel for each event based on its weighted positional probability and stores the values in the X/Y columns of the event FITS files. Standard image viewers such as ds9 and Ximage can automatically convert the SKY coordinates to the RADEC coordinate system.
DET:
the coordinate system mapped individually on the Resolve and Xtend detector planes, looking up toward the sky through the X-ray mirror (XMA). The Resolve DET coordinate system puts the 6$\times$6 pixel array in the positive quadrant, with the missing corner pixel for calibration at (1, 1), and increments the coordinates by 1 with every pixel. The Xtend DET coordinate system originates at the corner of the CCD array on CCD_ID=1, having the x- and y-axes of the vertical and horizontal CCD pixel directions of CCD_ID=1. The four CCDs do not perfectly align, so the DETX/Y axes are not strictly parallel to the CCD rows or columns of the other CCDs (CCD_IDs = 0, 2, 3). The event FITS files store these values in the DETX/DETY columns.

Users can convert coordinates from one coordinate system to another by using the generic FTOOLS command coordpnt, with information from the detector configuration and the nominal satellite attitude. Note that the satellite attitude wobbles during observations, so that conversions between a detector coordinate and a celestial coordinate are for the mean satellite attitude although the fluctuation should be small ($\lesssim$15 arcsec). Here is an example of converting DET coordinates (730.1, 727.5) of the Xtend ObsID: 000126000 data (see Chapter 5 for the dataset) to the RA/DEC coordinate system. 

term> punlearn coordpnt
term> coordpnt input="730.1,727.5" outfile="NONE" telescop="XRISM" 
instrume="XTEND" teldeffile=CALDB startsys=DET stopsys=RADEC 
ra=81.2584940856445 dec=-69.641223343144 roll=171.054859779373
(output) coordpnt: OUTX OUTY=   81.25215339   -69.63990500

The ra/dec/roll option values use the RA_NOM/DEC_NOM/PA_NOM keyword values, found in the header of FITS data files (e.g., cleaned event file). The tool can also feed a region file and output the conversion to another region file.


4.5.2 Pulse Height

Each event has a PI (pulse invariant) value that is proportional to the incident photon energy. This value is calculated from the sensor's pulse outputs (pulse height value: PHA for Resolve, PHAS for Xtend) and adjusted to the linear energy scale of the incident photon using calibration information. Xselect produces a PI histogram (event counts for each PI channel) from the selected events for spectral analysis. Xspec fits the histogram with physical models using the corresponding detector response. Each instrument applies its own Energy-PI relation, which is:

$\hspace{1.5cm}\texttt{PI[Resolve]} = \texttt{E(keV)}\times2000 $
$\hspace*{1.5cm}\texttt{PI[Xtend]}~~~= \texttt{E(keV)}~/~0.006 $


4.5.3 Detection Time

Each event dataset records the detection time in the TIME keyword, which is described in the XRISM Mission Elapsed Time (MET). The MET origin, 2019-01-01 00:00:00 UTC, is stored in the MJDREFI and MJDREFF keywords of all XRISM FITS file headers, so HEASoft or OGIP-compliant timing software can convert the mission time to a human-friendly time system. Users can also convert XRISM time to various time systems on the xTime HEASARC web page. Note that all time-related values in the XRISM FITS file headers are stored in the TT (Terrestrial Time) system (TIMESYS = TT). This time system is 69.184 sec ahead of UTC at 2019-01-01 00:00:00 UTC, and therefore MJDREF (=MJDREFI+MJDREFF) in the FITS header has MJD for 2019-01-01 00:01:09.184.


4.6 XSELECT Default Parameters

By default, XSELECT (see Section 3.1.4) uses the following setups for XRISM data analyses4.1. Users can change them during XSELECT sessions after loading event data.


4.7 Important Values in the Event Files


Table 4.1: Resolve Event Columns for Calibration, Screening, Filtering, and Analysis
Column Description Range
ITYPE/TYPE Resolution Grade 0$-$7/Xx
    0/Hp: High-resolution Primary  
    1/Mp: Mid-resolution Primary  
    2/Ms: Mid-resolution Secondary  
    3/Lp: Low-resolution Primary  
    4/Ls: Low-resolution Secondary  
    5/Bl: Baseline event (diagnostic)  
    6/El: Lost event $^{\dagger}$  
    7/Rj: rejected events  
PIXEL Pixel number 0$-$35
RISE_TIME Event pulse time from the baseline to the peak 0$-$255
PI Pulse Invariant energy channel 0$-$59999
STATUS[$N$] 16 bit Event Flag (14 in use) b0: no
    [1]: outside of all-pixel GTIs b1: yes
    [2]: outside of individual-pixel GTIs  
    [3]: coincidence with anti-co event  
    [4]: coincidence with event on any pixel except 12  
    [5]: coincidence with pixel 12 event  
    [6]: [5] & passed energy test for absorption of electron ejected  
        from 12  
    [7]: candidate electrical crosstalk event or its source  
    [8]: [7] & largest PHA of coincident group  
    [9]: during pulse of MXS direct source  
    [10]: during afterglow of MXS direct source  
    [11]: during pulse of MXS indirect source  
    [12]: during afterglow of MXS indirect source  
    [13]: event likely contaminated by untriggered electrical crosstalk  
    [14]: [13] & largest PHA of coincident group  
ITYPE/TYPE provides a numerical/adjectival representation of the event grade. It is also
conventionally called GRADE.
$^{\dagger}$The events the onboard software is unable to process within the available computational
time.



Table 4.2: Xtend Event Columns for Calibration, Screening, Filtering, and Analysis
Column Description Range
GRADE 5$\times$5 pixel pulse height patterns 0$-$11
    0: Single $^{\dagger}$  
    1: Corner split  
    2: Vertical split $^{\dagger}$  
    3: Left split $^{\dagger}$  
    4: Right split $^{\dagger}$  
    5: Vertical, left, or right split with a corner  
    6: 2$\times$2 box split with a low corner pulse-height $^{\dagger}$  
    7: 3 pixels vertical or horizontal split with an outer 5$\times$5 pixel  
        above thres.  
    8: unused  
    9: 3 pixels vertical or horizontal split with no outer 5x5 pixel  
        above thres.  
   10: GRADE 2, 4, 5, or 6 with an outer 5x5 pixel above thres.  
   11: 2$\times$2 box split with a high corner pulse-height  
PI Pulse Invariant energy channel 0$-$4095
STATUS[$N$] 48 bit Event Flag (37 in use) b0: no
    [1]: Bad event based on all other status flags b1: yes
    [2]: Inside the calibration source region  
    [3]: Outside of CCD$^{*}$  
    [4]: Outside of window$^{*}$  
    [5]: Outside of the area discrimination$^{*}$  
    [6]: Charge Injection (CI) row$^{*}$  
    [7]: Bad pixel from CALDB$^{*}$  
    [8]: Bad column from CALDB$^{*}$  
    [9]: Hot pixel from pre-pipeline$^{*}$  
    [10]: Flickering pixel detected by searchflickpix$^{*}$  
    [11]: CCD boundary$^{*}$  
    [12]: Window boundary$^{*}$  
    [13]: Segment boundary  
    [14]: Area discrimination boundary  
    [15]: At least one 3x3 surrounding pixel has bad status  
    [16]: CI trailing row$^{*}$  
    [17]: CI preceding row$^{*}$  
    [18]: Preceding/following of bad column$^{*}$  
    [19]: Neighbor of bad/hot pixel and bad column$^{*}$  
    [20]: Neighbor of flickering pixel$^{*}$  



Table 4.2: Continue
Column Description Range
STATUS[$N$]   [21]: Neighbor of preceding/following of bad column STATUS b0: no
    [22]: Neighbor of CCD/window boundary b1: yes
    [23]: Neighbor of segment boundary  
    [24]: 3x3 info is present but 5x5 is absent  
    [25]: 3x3 is absent$^{*}$  
    [26]: PHAS[0] $<$ event threshold$^{*}$  
    [27]: Video temperature is out of range$^{*}$  
    [28]: Lack of video temp HK at time close to the event$^{*}$  
    [29]: Correction value is negative  
    [30]: Null value by correction process$^{*}$  
    [31]: 1st trailing row of the CI rows  
    [32]: 1st preceding row of the CI rows  
    [33]: 2nd trailing row of the CI rows  
    [34]: 2nd preceding row of the CI rows  
    [35]: 3rd trailing row of the CI rows  
    [36]: 3rd preceding row of the CI rows  
    [37]: Cosmic ray echo pixel  
$^{\dagger}$Grades corresponding to X-ray (as opposed to cosmic ray) events.
$^{*}$Bad pixel events removed with the standard screening.


next up previous contents
Next: 5. XRISM Data Analysis Up: xrism abc Previous: 3. Analyses Tools and   Contents