11. OM Data Processing (Command Line and GUI)

As with EPIC and RGS datasets, many files are associated with an OM dataset. The INDEX.HTM file, and links therein, are viewable with a web browser and will help you navigate the dataset. The different types of files are discussed in Chapter 3.2; however, since the OM is somewhat different from the other instruments on-board XMM-Newton, we will discuss them in more detail in later sections.

The OM can operate in Imaging, Fast, and Grism mode. Each of these modes has dedicated commands to reprocess the data: omichain, omfchain, and omgchain. These are metatasks that each call several procedures that are used to prepare the data for processing, make and apply flatfield images, and detect sources. The tasks omichain and omfchain also calculate the instrumental magnitudes of sources, find the position of the sources (in equatorial coordinates), and produce a sky image; omgchain produces a spectrum. Due to the long file names and the large number of input parameters, users are urged to simply use the chains and not run the chains' individual tasks one at a time.

The chains apply all necessary corrections, so no further processing or filtering needs to be done. Please note that the chains do not produce output files with exactly the same names as those in the PPS directory (they also produce some files which are not included in the PPS directory at all.) Table 11.1 lists the file ID equivalences between repipelined and PPS files for the three modes.

Table 11.1: File ID equivalences between repipelined and PPS OM files.

Repipelined	PPS Name	Description
Name
TSHPLT	TSHPLT	OM tracking history plot
TSTRTS	TSTRTS	OM tracking star time series
IMAGE_	IMAGE_	OM OSW image (any filter or grism)
IMAGE_	IMAGEF	OM fast mode OSW image
SIMAGE	SIMAGE	OM OSW sky aligned image
SIMAGE	SIMAGF	OM fast mode OSW sky aligned image
REGION	SWSREG	OM OSW sources region file
REGION	SFSREG	OM fast mode OSW sources region file
SWSRLI	SWSRLI	OM OSW sources list
SWSRLI	SFSRLI	OM fast mode OSW sources list
TIMESR	TIMESR	OM fast mode OSW source time series
OBSMLI	OBSMLI	OM combined observation source list
FIMAG_	FIMAG_	OM combined full-frame image
FSIMAG	FSIMAG	OM combined full-frame sky image
HSIMAG	HSIMAG	OM full-frame HIRES sky image mosaic
LSIMAG	LSIMAG	OM full-frame LORES sky image mosaic
RSIMAG	RSIMAG	OM default mode sky image mosaic
USIMAG	USIMAG	OM user windows sky image mosaic
RSISWS	RSISWS	OM default mode source list from image mosaic
HSISWS	HSISWS	OM full-frame HIRES source list from image mosaic
LSISWS	LSISWS	OM full-frame LORES source list from image mosaic
USISWS	USISWS	OM user windows source list from image mosaic
OBSMOS	OBSMOS	OM mosaic merged sources list
OBSMER	none	OM final combined sources list: OBSMLI+OBSMOS
RIMAGE	GIMAGE	OM grism rotated image
SIMAGE	SIMAGE	OM grism OSW sky aligned image
SPECLI	SPECLI	OM grism spectra list
REGION	SGSREG	OM grism DS9 regions
SPCREG	SPCREG	OM grism DS9 spectrum regions
SPECTR	SPECTR	OM source extracted spectra
SWSRLI	SGSRLI	OM grism OSW sources list

As always, it is recommended to keep all reprocessed data in its own directory! SAS places output files in whichever directory it is in when a task is called. Throughout this Guide, it is assumed that the Pipeline Processed data are in the PPS directory, the ODF data (with upper case file names, and uncompressed) are in the directory ODF, and the analysis is taking place in the PROC directory. It is also assumed that the data were prepared for processing (see §6). For example data, we will use observations of the Lockman Hole (Obs ID 0123700101; the same as for the EPIC walk-through) for Image mode, Mkn 421 (Obs ID 0411081601) for Fast mode, and BPM 16274 (Obs ID 0125320801) for Grism mode, though any dataset with the appropriate mode will suffice.

11.1 OM Artifacts and General Information

Before proceeding with the pipeline, it is appropriate to discuss the artifacts that often affect OM images. These can affect the accuracy of a measurement by, for example, increasing the background level. Some of these can be seen in Fig. 11.1.

$\bullet$	Stray light - background celestial light is reflected by the OM detector housing onto the center on the OM field of view, producing a circular area of high background. This can also produce looping structures and long streaks.
$\bullet$	Modulo 8 noise - In the raw images, a modulo 8 pattern arises from imperfections in the event centroiding algorithm in the OM electronics. This is removed during image processing.
$\bullet$	Smoke rings - light from bright sources is reflected from the entrance window back on the detector, producing faint rings located radially away from the center of the field of view.
$\bullet$	Out-of-time events - sources with count rates of several tens of counts/sec show a strip of events along the readout direction, corresponding to photons that arrived while the detector was being read out.

Further, artifacts also can contaminate grism data. Due to this mode's complexity, users are urged to be very careful when working with grism data, and should refer to the SOC's website on this topic.

Users should also keep in mind some differences between OM data and X-ray data. Unlike EPIC and RGS, there are no good time intervals (GTIs) in OM data; an entire exposure is either kept or rejected. Also, OM exposures only provide direct energy information when in grism mode, and the flat field response of the detector is assumed to be unity.

For detailed descriptions of PP data nomenclature, file contents, and which tasks can be used to view them, see Tables 3.2 and 3.3.

If you simply want a quick look at your data, sky images and source lists are in *SIMAGE*.FTZ and *SWSRLI*.FTZ, respectively. Further, there are low resolution sky images for each filter; they follow the nomenclature:

 PjjjjjjkkkkOMX000USIMAGbb000.QQQ

jjjjjj - Proposal number kkkk - Observation ID b - Filter keyword: B, V, U, M (UVM2), L (UVW1) and S (UVW2) QQQ - File type (e.g., PNG, FTZ)

So for example, P0123700101OMX000USIMAGV000.FTZ is the final low resolution sky image in the V filter. To see what files have been summed to make the final image, search for the keyword XPROC0 in the FITS header. For our example image, this would be

XPROC0  = 'ommosaic imagesets=”product/P0123700101OMX000USIMAGV000.FTZ produc&'
CONTINUE  't/P0123700101OMS415SIMAGE1000.FIT product/P0123700101OMS416SIMAGE10&'
CONTINUE  '00.FIT product/P0123700101OMS417SIMAGE1000.FIT product/P0123700101O&'
CONTINUE  'MS418SIMAGE1000.FIT” mosaicedset=product/P0123700101OMX000RSIMAGV0&'
CONTINUE  '00.FIT exposuremap=no exposure=1000 # (ommosaic-2.10) [xm&msas_2019&'
CONTINUE  '0401_1820-18.0.0]'

The source list file (*SWSRLI*.FTZ) also contains useful information for the user; the column names are listed in Table 11.2.

Table 11.2: Some of the important columns in the source list file.

Column name	Contents
SRCNUM	Source number
RA	RA of the detected source
DEC	Dec of the detected source
POSERR	Positional uncertainty
RATE	extracted count rate
RATE_ERR	error estimate on the count rate
SIGNIFICANCE	Significance of the detection (in $\sigma$)
MAG	Brightness of the source in magnitude
MAGERR	uncertainty on the magnitude

11.2 Imaging Mode

11.2.1 Rerunning the Pipeline

To rerun the pipeline on all exposures and filters from the command line, type

 omichain

Alternatively, to run the pipeline from the SAS GUI,

1)	Call omichain.
2)	In the task pop-up window, click “Run”.

This produces numerous files, including images and regions for each exposure and each filter. If we are interested in the sources detected in the mosaicked V band image, we could run omichain with the appropriate flags from the command line typing:

 omichain filters=V processmosaicedimages=yes

where

 filters - list of filters to be processed
processmosaicedimages - process the mosaicked sky images?

Alternatively, we can do it with the SAS GUI:

1)	Call omichain.
2)	In the pop-up window, in the “0” tab, next to “filters”, enter V.
3)	Click “Run”.

11.2.2 Verifying the Output

While the output from the chains is ready for analysis, OM does have some peculiarities, as discussed in §11.1. These can possibly affect source brightness measurements, since they increase the background. In light of this, users are strongly encouraged to verify the consistency of the data prior to analysis. There are a few ways to do this. Users can examine the combined source list with fv, which will let them see if interesting sources have been detected in all the filters where they are visible. Users can also overlay the image source list on to the sky image with implot. This can also be done with ds9 or gaia by using slconv to change source lists into region files.

The task slconv allows users to set the regions radii in arcseconds to a constant value or scale them to header keywords, such as RATE. By default, ds9 region files have suffixes of .reg; gaia region files have suffixes of .gaia. In the example below, we make a region file from the source list for the mosaicked, V-band sky image.

To make a ds9 region file from a source list from the command line, type:

 slconv srclisttab=P0123700101OMS000RSISWSV.FIT radiusexpression=5
$$ outfileprefix=Vband_mosaic outputstyle=ds9

where

 srclisttab - source list file name
radiusexpression - constant or expression (possibly involving keywords) used
$$ to determine the radii of the plotted circles in arcseconds
outputstyle - output format; either ds9 or gaia
outfileprefix - prefix of output file name

At present, this task cannot be run in the GUI.

The mosaicked, V-band sky image, P0123700101OMS000RSIMAGV.FIT, with the region file from slconv overlayed, is shown in Fig. 11.1. There clearly are spurious detections; these can be removed by hand, or by rerunning omichain with a different background-level threshold or source significance. These are given by the parameters omdetectnsigma and omdetectminsignificance, respectively. So, if we wanted to rerun omichain for this example with the minimum significance of a source to be included in the source-list file set to 5 $\sigma$ from the default value of 3, we would enter:

 omichain filters=V processmosaicedimages=yes omdetectminsignificance=5

To do it with the GUI,

1)	Call omichain.
2)	In the pop-up window, in the “0” tab, in the box next to filters, enter V. Next to omdetectminsignificance, enter 5. In the “1” tab, set processmosaicedimages to yes.
3)	Click “Run”.

Figure 11.1: A mosaicked, V-band sky image and corresponding region file viewed with ds9. Spurious detections are clearly present and can be removed either by hand or by rerunning omichain with the appropriate flags. Some artifacts are also visible.

It is also a good idea to look at the tracking over the course of the observation. The spacecraft's tracking correction history over each OM exposure is plotted in the file that follows the naming convention *TSHPLT*.PDF. An example is shown in Figure 11.2.

Figure 11.2: The tracking history of a Lockman Hole OM observation.

11.2.3 Making a PHA File

It is possible to make an OM “spectrum” which shows the flux in a given filter at the effective wavelength or energy of that filter. It can be analyzed by itself or together with that from RGS or EPIC in XSpec, Sherpa, or other familiar software package, but we will first need to transform the OM data into OGIP II format. To do this, we must identify the RA and Dec of the source that we are interested in. This can be done by displaying the EPIC event file in ds9 and loading the OM region file to make sure there is something there; see Figure 11.3. It can be seen that the sources are identified with circles of various colors, with the colors indicating the quality flags. The flags have colors as defined in Table 11.3, with integer values in the *OMCOMBOBSMLI0000.FIT file giving more detailed information. The integer values are the sum of the quality flags for the source, so for example, if a source has an integer quality value = 3, then it is affected by one or more bad pixels and a read-out streak. Sources are numbered, so it is easy to find the correct entry in the OM source list.

Table 11.3: Color-coding of quality flags.

Reason	Color	Integer Value
Bad pixel	red	1
Read-out streak	magenta	2
Smoke ring	yellow	4
Source on star-spike	white	8
Mod-8 pattern	black	16
Source within central enhancement	red	32
Source lies near to a bright source	black	64
Near an edge	blue	128
point-source within extended source	blue	256
Weird source	red	512
Multiple exposure values within photometry aperture	red	1024

For our example, we will consider the OM source that corresponds to the X-ray source that we made an EPIC spectrum for. Loading the EPIC data mos1_filt_time.fits and the combined image source region file P0123700101OMCOMBOBSMLI0000.reg, we see that we need source number 192, which has RA=163.165152 and Dec=57.408701. It has quality flag warnings in the UVW1, U, V, and white bands that indicate that it is near a read-out streak.

Now we can do the transform:

 om2pha srclist=P0123700101OMCOMBOBSMLI0000.FIT ra=163.165152 dec=57.408701 \
$$ output=om_src_pi.fits

where

 srclist - source list file name
ra - source's right ascension in degrees
dec - source's declination in degrees
output - output file name

To do it with the GUI,

1)	Call om2pha.
2)	In the pop-up window, in the box next to srclist, enter P0123700101OMCOMBOBSMLI0000.FIT. Next to ra, enter 163.165152. Next to dec, enter 57.408701. In the box next to output, enter the name of the output file, om_src_pi.fits.
3)	Click “Run”.

The output file will have information in the header that directs the analysis software to look for canned response files. These are available for download here:
https://heasarc.gsfc.nasa.gov/FTP/xmm/data/responses/OM/

or here:
https://sasdev-xmm.esac.esa.int/pub/ccf/constituents/extras/responses/OM/

We will download and decompress “om_effarea_v2.0.tgz”, which contains the responses for all the OM filters, to make sure we will have what we need:

 tar xvfz om_effarea_v2.0.tgz

No further processing is required. From here, we can proceed straight to analyzing our spectrum in XSpec or Sherpa.

Figure 11.3: The V-band region file overlaid on the X-ray image. The X-ray source we are interested in was detected in OM, and is source #192.

11.3 Fast Mode

11.3.1 Rerunning the Pipeline

The repipelining task for OM data taken in fast mode is omfchain. It produces images of the detected sources, extracts events related to the sources and the background, and extracts the corresponding light curves. At present, unlike omichain, omfchain does not allow for keywords to specify filters or exposures; calling this task will process all fast mode data.

To run the pipeline on fast mode data on the command line, type

 omfchain

Alternatively, omfchain can be run from the GUI:

1)	Call omfchain.
2)	In the task pop-up window, click “Run”.

11.3.2 Verifying the Output

There are two types of output files: those that start with F are intermediate images or time series files; those that start with P are products. A good first step in checking the output is to examine the light curve plot for both the source and background, making sure they are reasonable: no isolated, unusually high (or low) values, and no frequent drop-outs. The light curves for the background-subtracted source and the background are automatically produced for each observation. An example (P0411081601OMS006TIMESR1000.PDF) is shown in Fig. 11.4 (right).

The background light curve in Figure 11.4 (right) is constant because omfchain runs with the parameter bkgfromimage=yes by default, so that the background light curve is found by using the imaging-mode data, instead of the fast-mode window. This is preferable for even only moderately bright sources (count rate $>$ 0.6 ct/s), as the fast-mode window is small - only 22$\times$23 pixels - and any background measurement that uses it will likely be contaminated with source photons. This is less of a concern if the source is faint, in which case the background can by found from data in the fast-mode window by typing:

 omfchain bkgfromimage=no

To do this with the GUI,

1)	Call omfchain.
2)	In the “1” tab, set bkgfromimage to no.
3)	Click “Run”.

We should also check the image of the window with the extraction area overplotted to see if the source is near an edge and if there are other sources in the window. We can use slconv to get the source extraction region and plot it in ds9 (see Figure 11.4 (left)).

 slconv srclisttab=P0411081601OMS006SWSRLI1000.FIT radiusexpression=5 \
$$ outfileprefix=om_source
ds9 P0411081601OMS006SIMAGE1000.FIT &

As noted earlier, at present, slconv cannot be used with the GUI.

If the source is near an edge, it's a good idea to examine the light curves from different exposures (P*TIMESR*PDF) to verify that they are consistent from exposure to exposure, while keeping in mind any intrinsic source variability.

If there is another source in the window, the light curve and background flux could be inaccurate. The size and position of the source and background extraction areas can be modified by rerunning omfchain with different values for the srcradius, bkginner, and bkgouter parameters.

Figure 11.4: Left: The processed Fast mode image and extraction area. Right: the light curve produced automatically by omfchain.

11.4 Grism Analysis

11.4.1 Rerunning the Pipeline

The repipelining task for OM data taken in grism mode is omgchain. It produces images of the detected sources and background, extracts source spectra and region files, and makes source lists and postscript and PDF plots. At present, unlike omichain, omgchain does not allow for keywords to specify filters or exposures; calling this task will process all grism mode data.

To run the pipeline on fast mode data on the command line, type

 omgchain

Alternatively, omgchain can be run from the GUI:

1)	Call omgchain.
2)	In the task pop-up window, click “Run”.

There are two types of output files: those that start with g are intermediate or auxiliary files and source lists; those that start with p are products.

To demonstrate some of these output files, we have rerun the pipeline on the example dataset. The processed image, rotated to align with the columns of the image (p0125320801OMS005RIMAGE0000.FIT), is shown in Fig. 11.5 (left). Two region files are overlayed: p0125320801OMS005REGION0001.ASC, which corresponds to the sources detected in this rotated image (green), and p0125320801OMS005SPCREG0001.ASC, which corresponds to the sources in the spectra list file (red) and indicates the locations of the zero and first orders. The task omgchain automatically extracted the spectrum of the red region (p0125320801OMS005SPECTR0000.FIT); this is shown in Fig. 11.5 (right).

Figure 11.5: Top: The repipelined, rotated image with regions overlayed. The source is in the red region. Bottom: the spectrum and count rate extracted from the source displayed in fv.

11.4.2 Verifying the Output

The correct correlation of zero and first orders is crucial for grism analysis. Users should inspect the rotated image with fv or ds9 and verify the identification of the orders by overlaying the *SPCREG* region file, as shown in Fig. 11.5 (left); the *SPECLI* file also contains this informtaion. If users are interested in all source detections, the region file can also be overlayed and the full source list examined. Users should also examine the spectra plots automatically produced by omgchain, for both the source and background, making sure they are reasonable. For improved source detection, the parameter nsigma can be changed.

11.4.3 Making a PHA File

OM Grism mode data can be cast in a format to allow it to be analyzed alongside RGS or EPIC spectral data in XSpec or Sherpa. There are no canned response files like for Image mode, but it is easy to make the response file at the same time as the spectrum file. For our example observation, we would enter

 omgrismresp inputspec=P0810242101OMS027SPECTR0000.FIT outputspec=grism_spec.fits \
$$ responsefile=grism_resp.fits

To do this with the GUI,

1)	Call omgrismresp.
2)	In the task pop-up window, in the box next to inputspec, enter the input spectrum file, P0810242101OMS027SPECTR0000.FIT. Next to outputspec, enter the output spectrum file, grism_spec.fits. Next to responsefile, enter the output resonse file, grism_resp.fits.
3)	Click “Run”.

The output response file is not an RMF or ARF, but rather the product of them: response file = RMF $\times$ ARF. Xspec will recognize this, but other software packages may not. If needed, the response file can be split into its RMF and ARF components with ftools:

 ftrsp2rmfarf infile=grism_resp.fits rmffile=grism_rmf.fits arffile=grism_arf.fits

These can be used with the grism's spectrum in the same way that X-ray RMF and ARF files are used with an X-ray spectrum.