9. An OM Data Processing and Analysis Primer

As with EPIC and RGS datasets, many files are associated with an OM dataset. The INDEX.HTM file, and links therein, are viewable with a web browser and will help you navigate the dataset. The different types of files are discussed in Chapter 5.2; however, since the OM is somewhat different from the other instruments on-board XMM-Newton, we will discuss them in more detail in later sections.

The OM can operate in IMAGING, FAST, and GRISM mode. Each of these modes has dedicated commands to reprocess the data: omichain, omfchain, and omgchain. These are Perl scripts which each call several procedures sequentially that are used to prepare the data for processing, make and apply flatfield images, and detect sources. The tasks omichain and omfchain also calculate the instrumental magnitudes of sources, find the position of the sources (in equatorial coordinates), and produce a sky image; omgchain produces a spectrum. If you run these chains, it is helpful to inspect the sas_log file to get a detailed list of the performed tasks. These chains rely on filters specified by the user; if no arguments are given, they run on all the files present in the ODF directory. Due to the long file names and the large number of input parameters, users are urged to simply use the chains and not run the chains' individual tasks one at a time.

Most OM data are obtained in IMAGING mode. If they were obtained in the FAST mode, there will be an additional event list file corresponding to the Fast window (*FAE.FIT). Reprocessing of data taken in FAST mode is discussed in §9.3. Reprocessing OM GRISM data is discussed in §9.4.

At this point, it is assumed that you have downloaded the data from the HEASARC archive onto a Hera server, standard or anonymous Hera is running (see §4.2), you have prepared the data for processing (see Chapter 6), and the working directory PROC has been made. For example data, we will use observations of the Lockman Hole (Obs ID 0123700101; the same as for the EPIC walk-through) for Image mode, Mkn 421 (Obs ID 0411081601) for Fast mode, and BPM 16274 (Obs ID 0125320801) for Grism mode, though any dataset with the appropriate mode will suffice.

9.1 OM Artifacts and General Information

Before proceeding with the pipeline, it is appropriate to discuss the artifacts that often affect OM images. These can affect the accuracy of a measurement by, for example, increasing the background level. Some of these can be seen in Fig. 9.1.

$\bullet$

Stray light - background celestial light is reflected by the OM detector housing onto the center on the OM field of view, producing a circular area of high background. This can also produce looping structures and long streaks.

$\bullet$

Modulo 8 noise - In the raw images, a modulo 8 pattern arises from imperfections in the event centroiding algorithm in the OM electronics. This is removed during image processing.

$\bullet$

Smoke rings - light from bright sources is reflected from the entrance window back on the detector, producing faint rings located radially away from the center of the field of view.

$\bullet$

Out-of-time events - sources with count rates of several tens of counts/sec show a strip of events along the readout direction, corresponding to photons that arrived while the detector was being read out.

Further, artifacts also can contaminate grism data. Due to this mode's complexity, users are urged to be very careful when working with grism data, and should refer to the SOC's website on this topic.

Users should also keep in mind some differences between OM data and X-ray data. Unlike EPIC and RGS, there are no good time intervals (GTIs) in OM data; an entire exposure is either kept or rejected. Also, OM exposures only provide direct energy information when in grism mode, and the flat field response of the detector is assumed to be unity.

For detailed descriptions of PP data nomenclature, file contents, and which tasks can be used to view them, see Tables 5.2 and 5.3.

If you simply want a quick look at your data, sky images and source lists are in *SIMAGE*.FTZ and *SWSRLI*.FTZ, respectively. Further, there are low resolution sky images for each filter; they follow the nomenclature:

PjjjjjjkkkkOMX000RSIMAGbb000.QQQ

jjjjjj - Proposal number
kkkk - Observation ID
b - Filter keyword: B, V, U, M (UVM2), L (UVW1) and S (UVW2)
QQQ - File type (e.g., PNG, FTZ)

So for example, P0123700101OMX000RSIMAGV000.FTZ is the final low resolution sky image in the V filter. To see what files have been summed to make the final image, search for the keyword XPROC0 in the FITS header. For our example image, this would be

XPROC0  = 'ommosaic imagesets=''product/P0123700101OMS004SIMAGE1000.FIT produc&'
CONTINUE  't/P0123700101OMS415SIMAGE1000.FIT product/P0123700101OMS416SIMAGE10&'
CONTINUE  '00.FIT product/P0123700101OMS417SIMAGE1000.FIT product/P0123700101O&'
CONTINUE  'MS418SIMAGE1000.FIT'' mosaicedset=product/P0123700101OMX000RSIMAGV0&'
CONTINUE  '00.FIT exposuremap=no exposure=1000 # (ommosaic-1.11.7) [xmmsas_200&'
CONTINUE  '61026_1802-6.6.0]'

The source list file (*SWSRLI*.FTZ) also contains useful information for the user; the column names are listed in Table 9.1.

The source list file (*SWSRLI*.FTZ) also contains useful information for the user; the column names are listed in Table 9.1.

Table 9.1: Some of the important columns in the source list file.

Column name	Contents
SRCNUM	Source number
RA	RA of the detected source
DEC	Dec of the detected source
POSERR	Positional uncertainty
RATE	extracted count rate
RATE_ERR	error estimate on the count rate
SIGNIFICANCE	Significance of the detection (in $\sigma$)
MAG	Brightness of the source in magnitude
MAGERR	uncertainty on the magnitude

9.2 Imaging Mode

9.2.1 Rerunning the Pipeline

Please note that calling any of the repipelining tasks will initiate processing on all OM data of that particular mode; currently, only omichain will accept parameters to limit processing to a specific filter or exposure.

To rerun the pipeline on all exposures and filters, in the new Command Window you made at the end of §6, run the task:

omichain

This produces numerous files, including images and regions for each exposure and each filter. If we are interested in the sources detected in the mosaicked, V band image we could run omichain with the appropriate flags by typing

omichain filters=V processmosaicedimages=yes omdetectnsigma=2.0
$$ omdetectminsignificance=3.0

where

filters - list of filters to be processed
processmosaicedimages - process the mosaicked sky images?
omdetectnsigma - number of $\sigma$ above background required for a pixel to be considered part of a source
omdetectminsignificance - minimum significance of a source to be included in the source list file

The output files can be used immediately for analysis, though users are strongly urged to examine the output for consistancy first (see §9.2.2). The chains apply all necessary corrections, so no further processing or filtering needs to be done. Please note that the chains do not produce output files with exactly the same names as those in the PPS directory (they also produce some files which are not included in the PPS directory at all.) Table 9.2 lists the file ID equivalences between repipelined and PPS files.

Table 9.2: File ID equivalences between repipelined and PPS OM files.

Repipelined	PPS Name	Description
Name
EVLIST	none	FAST mode events list
FIMAG_	FIMAG_	combined full-frame image
FLAFLD	none	flatfield
FSIMAG	FSIMAG	combined full-frame sky image
HSIMAG	HSIMAG	full-frame HIRES sky image mosaic
IMAGE_	IMAGE_	image from any filter or GRISM
IMAGE_	IMAGEF	FAST mode image
LSIMAG	LSIMAG	full-frame LORES sky image mosaic
OBSMLI	OBSMLI	combined observation source list
REGION	SWSREG	sources region file
REGION	SFSREG	FAST mode sources region file
REGION	SGSREG	GRISM ds9 regions
RIMAGE	GIMAGE	GRISM rotated image
RSIMAG	RSIMAG	default mode sky mosaic
SIMAGE	SIMAGE	sky aligned image
SIMAGE	SIMAGF	FAST mode sky aligned image
SIMAGE	none	GRISM sky aligned image
SPCREG	SPCREG	GRISM ds9 spectrum regions
SPECLI	SPECLI	GRISM specra list
SPECTR	SPECTR	source extracted spectra
SUMMAR	SUMMAR	observation summary
SWSRLI	SWSRLI	sources list
SWSRLI	SFSRLI	FAST mode sources list
SWSRLI	SGSRLI	GRISM sources list
TIMESR	TIMESR	FAST mode source timeseries
TSHPLT	TSHPLT	tracking history plot
TSTRTS	TSTRTS	tracking star timeseries

9.2.2 Verifying the Output

While the output from the chains is ready for analysis, OM does have some peculiarities, as discussed in §9.1. While these usually have only an aesthetic effect, they can also affect source brightness measurements, since they increase the background. In light of this, users are strongly encouraged to verify the consistency of the data prior to analysis. There are a few ways to do this. Users can examine the combined source list with fv, which will let them see if interesting sources have been detected in all the filters where they are visible. Users can also overlay the image source list on to the sky image with implot by downloading the relevant files to the user's local machine. This can also be done with ds9 or gaia by using slconv to change source lists into region files and downloading the relevant files to your local machine. The task slconv allows users to set the regions radii in arcseconds to a constant value or scale them to header keywords, such as RATE. By default, ds9 region files have suffixes of .reg; gaia region files have suffixes of .gaia.

In the following example, we make a region file from the source list for the mosaicked, V-band sky image by typing

slconv srclisttab=P0123700101OMS000RSISWSV.FIT radiusexpression=5
$$ outfileprefix=Vband_mosaic outputstyle=ds9

where

srclisttab - source list file name
radiusexpression - constant or expression (possibly involving keywords) used
$$ to determine the radii of the plotted circles
outputstyle - output format; either ds9 or gaia
outfileprefix - prefix of output file name

The image and region files can be easily downloaded to a local machine for examination by dragging-and-dropping into a target directory. The mosaicked, V-band sky image, P0123700101OMS000RSIMAGV.FIT, with the region file from slconv overlayed, is shown in Fig. 9.1. There clearly are spurious detections; these can be removed by hand, or by rerunning omichain with a different background-level threshold or source significance.

Figure 9.1: A mosaicked, V-band sky image and corresponding region file viewed with ds9. Spurious detections are clearly present and can be removed either by hand or by rerunning omichain with the appropriate flags.

9.3 Fast Mode

9.3.1 Rerunning the Pipeline

The repipelining task for OM data taken in fast mode is omfchain. It produces images of the detected sources, extracts events related to the sources and the background, and extracts the corresponding light curves. At present, unlike omichain, omfchain does not allow for keywords to specify filters or exposures; calling this task will process all fast mode data.

To run the pipeline on fast mode data, type

omfchain

There are two types of output files: those that start with f are intermediate images or time series files; those that start with p are products. The output files are are described in Table 9.2.

To demonstrate some of these output files, we have rerun the pipeline on the example dataset and downloaded the output to a local machine. The processed image in sky-coordinates from one exposure, P0411081601OMS006SIMAGE1000.FIT, is shown in Fig. 9.2 (left). The background-subtracted light curve produced automatically by the task, F0411081601OMS006TIMESR1000.PS, is shown in Fig. 9.2 (right).

9.3.2 Verifying the Output

A good first step in checking the output is to examine the light curve plot for both the source and background, making sure they are reasonable: no isolated, unusually high (or low) values, and no frequent drop-outs. Users should also check the image with fv or ds9 in the Fast mode window to see if the source is near an edge. If it is, it's a good idea to examine the light curves from diffent exposures to verify that they are consistent from exposure to exposure (while keeping in mind any intrinsic source variability). If the image is blurred or unusual in any way, users should check the tracking history file to verify the tracking was reliable.

Figure 9.2: Left: The processed FAST mode sky image. Right: the light curve produced automatically by omfchain.

9.4 Grism Analysis

9.4.1 Rerunning the Pipeline

The repipelining task for OM data taken in grism mode is omgchain. It produces images of the detected sources and background, extracts source spectra and region files, and makes source lists and postscript and PDF plots. At present, unlike omichain, omgchain does not allow for keywords to specify filters or exposures; calling this task will process all grism mode data.

To run the pipeline on grism mode data, type

omgchain

There are two types of output files: those that start with g are intermediate or auxiliary files and source lists; those that start with p are products. The output files are are described in Table 9.2.

To demonstrate some of these output files, we have rerun the pipeline on the example dataset and downloaded the output to a local machine. The processed image, rotated to align with the columns of the image (p0125320801OMS005RIMAGE0000.FIT), is shown in Fig. 9.3 (left). Two region files are overlayed: p0125320801OMS005REGION0001.ASC, which corresponds to the sources detected in this rotated image (green), and p0125320801OMS005SPCREG0001.ASC, which corresponds to the sources in the spectra list file (red) and indicates the locations of the zero and first orders. The task omgchain automatically extracted the spectrum of the red region (p0125320801OMS005SPECTR0000.FIT); this is shown in Fig. 9.3 (right).

Figure 9.3: Left: The repipelined, rotated image with regions overlayed. Right: the spectrum extracted from the source (red region).

9.4.2 Verifying the Output

The correct correlation of zero and first orders is crucial for grism analysis. Users should inspect the rotated image with fv or ds9 and verify the identification of the orders by overlaying the *SPCREG* region file, as shown in Fig. 9.3 (left); the *SPECLI* file also contains this information. If users are interested in all source detections, the region file can also be overlayed and the full source list examined. Users should also examine the spectra plots automatically produced by omgchain, for both the source and background, making sure they are reasonable. For improved source detection, the parameter nsigma can be changed.