11. An RGS Data Processing and Analysis Primer, GUI

Many files are associated with an RGS dataset, and it is easy to be overwhelmed. The INDEX.HTM file, and links therein, are viewable with a web browser and will help you navigate the dataset. The different types of files are discussed in more detail in Chapter 1.

As ever, it is strongly recommended that you keep all reprocessed data in its own directory! SAS places output files in whichever directory it is in when a task is called. Throughout this primer, it is assumed that the Pipleline Processed data are in the PPS directory, the ODF data (with upper case file names, and uncompressed) are in the directory ODF, the reprocessing and analysis is taking place in the PROC directory, and the CCF data are in the directory CCF.

If you have just received your data from the SOC, it has been processed with the most recent version of SAS, and you should not need to repipeline it (though no harm is done if you do); you need only to gunzip the files and prepare the data for processing (see §5. However, it is very likely that you will want to filter your data; in this case, you will need to reprocess it in order to determine the appropriate filters. Therefore, we recommend that you rerun the pipeline regardless of the age of your dataset.

But if you decide that reprocessing is unnecessary, you need only to gunzip the files and rename event files for easier handling. For example, for the RGS1 event list,

cp PPS/PiiiiiijjkkR1lEVENLInmmm.FTZ PROC/r1_evt.fits

where

iiiiiijjkk - observation number
l - scheduled (S) or unscheduled (U) obseravtion
n - spectral order number
mmm - source number

As noted in Tables 3.2 and 3.3 you can view images of your data. While the zipped FITS files may need to be unzipped before display in ds9 (depending on the version of ds9), they can be displayed when zipped using fv. As usual, there are some HTML products to help you inspect the data. These have file names of the form:

PPiiiiiijjkkAAAAAA000_0.HTM

where

iiiiiijjkk - Observation number
jj - observation ID - target number in proposal
kk - observation ID - observation number for target
AAAAAA - Group ID (see Table 3.2)

You will find a variety of RGS-specific files in XMM-Newton data sets. Generally there are two of each because there are two RGS instruments. Table 3.3 lists typical file names, their purpose, the file format, and a list of tools that will enable the user to inspect their data. Remember that the INDEX.HTM file will help you navigate.

Various analysis procedures are demonstrated using the Mkn 421 dataset, ObsID 0153950701, which definitely needs to be repipelined. The following procedures are applicable to all XMM-Newton datasets, so it is not required that you use this particular dataset; any observation should be sufficient.

11.1 Rerun the Pipeline on the Command Line

We assume that the data was prepared and environment variables were set according to §5, the GUI has been invoked (see §5.3), and we are in our working directory, ``PROC''.

A window will appear with two panels. The upper one will contain task names, what group they belong to (such as utility, epic, timing, calibration, etc.) and a short description of each. The lower one will contain information about environment variables, and as tasks are invoked, feedback from the tasks.

From the upper window, select rgsproc. As with all SAS tasks in the GUI, double-clicking the task will bring up pop-up windows that will allow you to change the parameters. For rgsproc,

1)

In the ``global'' tab, verify that ``orders'' is set to 1 2 and ``spectrumbinning'' is set to lambda. In the ``angles'' tab, verify that ``withmlambdacolumn'' is set to yes. In the ``spectra'' tab, under the ``rgsspectrum'' sub-tab, verify that ``bkgcorrect'' is set to no.

2)

Click ``Run''.

Note the last keyword, spectrumbinning. If you want to merge data from the same orders in RGS1 and RGS2, keep it at the default value lambda. If you want to merge data from the same instrument, with different orders, set it to beta. Merging spectra is discussed in §11.7.

This takes several minutes, and outputs 12 files per RGS, plus 3 general use FITS files. At this point, renaming files to something easy to type is a good idea:

ln -s *R1*EVENLI*FIT r1_evt1.fits
ln -s *R2*EVENLI*FIT r2_evt1.fits

11.1.1 Potentially useful tips for using the pipeline

The pipeline task, rgsproc, is very flexible and can address potential pitfalls for RGS users. In §11.1, we used a simple set of parameters with the task; if this is sufficient for your data (and it should be for most), feel free to skip to later sections, where data filters are discussed. In the following subsections, we will look at the cases of a nearby bright optical source, a nearby bright X-ray source, and a user-defined source.

11.1.2 A Nearby Bright Optical Source

With certain pointing angles, zeroth-order optical light may be reflected off the telescope optics and cast onto the RGS CCD detectors. If this falls on an extraction region, the current energy calibration will require a wavelength-dependent zero-offset. Stray light can be detected on RGS DIAGNOSTIC images taken before, during and after the observation. Please note that this will not work in every case. If a source is very bright, the diagnostic data that this relies on may not have been downloaded from the telescope in order to save bandwidth. Also, the RGS target itself cannot be the source of optical photons, as the spectrum's zero-order falls far from the RGS chip array. This test, and the offset correction, are not performed on the data before delivery.

To check for stray light and apply the appropriate offsets,

1)

Call rgsproc.

2)

In the ``global'' tab, verify that ``orders'' is set to 1 2 and ``spectrumbinning'' is set to lambda. In the ``angles'' tab, verify that ``withmlambdacolumn'' is set to yes. In the ``spectra'' tab, under the ``rgsspectrum'' sub-tab, verify that ``bkgcorrect'' is set to no.

3)

In the ``events'' tab, under the ``rgsoffsetcalc'' sub-tab, set ``calcoffsets'' to yes and ``withoffsethistogram'' to no.

4)

Click ``Run''.

11.1.3 A Nearby Bright X-ray Source

In the example above, it is assumed that the field around the source contains sky only. Provided a bright background source is well-separated from the target in the cross-dispersion direction, a mask can be created that excludes it from the background region. Here the source has been identified in the EPIC images and its coordinates have been taken from the EPIC source list which is included among the pipeline products. The bright neighboring object is found to be the third source listed in the sources file. The first source is the target:

1)

Call rgsproc.

2)

In the ``global'' tab, verify that ``orders'' is set to 1 2 and ``spectrumbinning'' is set to lambda. In the ``angles'' tab, verify that ``withmlambdacolumn'' is set to yes. In the ``spectra'' tab, under the ``rgsspectrum'' sub-tab, verify that ``bkgcorrect'' is set to no.

3)

In the ``events'' tab, under the ``rgssources'' sub-tab, set ``withepicset'' to yes and ``epicset'' to the name of the EPIC source list as generated by emldetect or eboxdetect. In the ``spectra'' tab, under the ``rgsregions'' sub-tab, set ``exclsrcsexpr'' to INDEX==1&&INDEX==3.

4)

Click ``Run''.

11.1.4 User-defined Source Coordinates

If the true coordinates of an object are not included in the EPIC source list or the science proposal, the user can define the coordinates of a new source.

1)

Call rgsproc.

2)

In the ``global'' tab, verify that ``orders'' is set to 1 2 and ``spectrumbinning'' is set to lambda. In the ``angles'' tab, verify that ``withmlambdacolumn'' is set to yes. In the ``spectra'' tab, under the ``rgsspectrum'' sub-tab, verify that ``bkgcorrect'' is set to no.

3)

In the ``events'' tab, under the ``rgssources'' sub-tab, set ``withsrc'' to yes. Be sure that the box beneath ``withsrc'' is toggled to radec. Next to ``srcra'', enter the RA in decimal degrees, 166.113808 and next to ``srcdec'', enter the declination (again in decimal degrees), +38.208833. Next to ``srclabel'', enter Mkn421.

4)

Click ``Run''.

Since the event files are current, we can proceed with some simple analysis demonstrations, which will allow us to generate filters. Remember that all tasks should be called from the window where SAS was initiated, and that tasks place output files in whatever directory you are in when they are called.

11.2 An Introduction to xmmselect

The task xmmselect is used for many procedures in the GUI. Like all tasks, it can easily be invoked by starting to type the name and pressing enter when it is highlighted.

When xmmselect is invoked a dialog box will first appear requesting a file name. You can either use the browser button or just type the file name in the entry area, ``r1_evts1.fits:EVENTS'' in this case. To use the browser, select the file folder icon; this will bring up a second window for the file selection. Choose ``r1_evts1.fits'', then the ``EVENTS'' extension in the right-hand column, and click ``OK''. The directory window will then disappear and you can click ``Run'' on the selection window.

When the file name has been submitted the xmmselect GUI (see Figure 11.1) will appear. (Later, when loading an event file that has been filtered, the filters that have been applied to that point will appear in the selection expression area.)

Figure 11.1: The xmmselect GUI.

11.3 Create and Display an Image

Two commonly-made plots are those showing PI vs. BETA_CORR (also known as ``banana plots'') and XDSP_CORR vs. BETA_CORR. To make a banana plot, invoke xmmselect in the SAS GUI and load the event file r1_evt1.fits, as discussed above. Then,

1)

Check the square boxes to the left of the BETA_CORR and PI entries to set the X and Y values in the image.

2)

Click on the "Image" button near the bottom of the page. This brings up the evselect GUI.

3)

Click on the "Image" tab in the evselect GUI. In the imageset box, enter the name of the output file. We will use pi_bc.fits.

4)

Click "Run".

Different binnings and other selections can be invoked by accessing the ``Image'' tab at the top of the GUI. The default settings are reasonable, however, for a basic image. The resultant image is automatically displayed using ds9. Similarly, plots can be made comparing BETA_CORR to XDSP_CORR. These two example plots can be seen in Figure 11.2.

Figure 11.2: Plots of PI vs. BETA_CORR (left) and XDSP vs. BETA_CORR (right).

11.4 Create and Display a Light Curve

The background is assessed through examination of the light curve. We will extract a region, CCD9, that is most susceptible to proton events and generally records the least source events due to its location close to the optical axis. Also, to avoid confusing solar flares for source variability, a region filter that removes the source from the final event list should be used. The region filters are kept in the source file product P*SRCLI_*.FIT. (For our example data, this would be P0134520301R1S001SRCLI_0000.FIT).

More experienced users should be aware that with SAS 13, the *SRCLI* file's column information changed. rgsproc now outputs an M_LAMBDA column instead of BETA_CORR, and M_LAMBDA should be used to generate the light curve. (The *SRCLI* file that came with the PPS products still contains a BETA_CORR column if you prefer to use that instead.)

To create light curves, call xmmselect from the GUI. Then,

1)

Enter the filtering criteria in the ``Selection expression'' box at the top of the xmmselect GUI:
(CCDNR==9)&&(REGION(P0153950701R1S001SRCLI_0000.FIT:RGS1_BACKGROUND,M_LAMBDA,XDSP_CORR))

2)

Check the round box to the left of the time entry.

3)

Click on the ``OGIP Rate Curve'' button near the bottom of the page. This brings up the evselect GUI.

4)

Click on the ``Lightcurve'' tab and confirm that the withrateset box is checked. Change the timebinsize to a reasonable amount, e.g. 10 or 100 s, and change the default output file name in the rateset box to something appropriate, in this case, r1_ltcrv.fits.

5)

Click on the ``Run'' button at the lower left corner of the evselect GUI.

The resultant light curve is displayed automatically using Grace and is shown in Figure 11.3.

Figure 11.3: The event rate from the RGS1 CCD9 chip. The time units are elapsed mission time.

11.5 Generating the Good Time Interval (GTI) File

Examination of the lightcurve shows that there is a loud section at the end of the observation, after 1.36975e8 seconds, where the count rate is well above the quiet count rate of $\sim$ 0.05-0.2 count/second. To remove it, we need to make an additional Good Time Interval (GTI) file and apply it by rerunning rgsproc.

There are two tasks that make a GTI file: gtibuild and tabgtigen. Either will produce the needed file, so which one to use is a matter of the user's preference. Both are demonstrated below.

11.5.1 Filter on Time with gtibuild

The first method, using gtibuild, requires a text file as input. In the first two columns, refer to the start and end times (in seconds) that you are interested in, and in the third column, indicate with either a + or - sign whether that region should be kept or removed. Each good (or bad) time interval should get its own line. Comments can also be entered, if they are preceeded by a "#". In the example case, we would write in our ASCII file (named gti.txt):

1.36958e8 1.36975e8 + # The quiet part only, please.

and proceed to invoke the task gtibuild from the SAS GUI. Then,

1)

For the file parameter, enter the name of the text file, gti.txt. For the table parameter, enter the output file name. We will use gti.fits.

2)

Click "Run".

11.5.2 Filter on Time with tabgtigen

Alternatively, we could filter on time using tabgtigen using the filtering expression from the times noted previously. To do this, invoke tabgtigen from the SAS GUI, and then

1)

In the table box, enter the name of the lightcurve file, r1_ltcrv.fits. In the gtiset box, enter the name of the output file; we will use gti.fits. In the expression box, enter the filtering expression: (TIME in [1.36958e8:1.36975e8])

2)

Click "Run".

11.5.3 Filter on Rate with tabgtigen

Finally, we could filter on rate using tabgtigen. The quiet count rate for this observation is about 0.1-0.2 ct/s, so we will use that in the filtering expression. Invoke tabgtigen from the SAS GUI, and then

1)

In the table box, enter the name of the lightcurve file, r1_ltcrv.fits. In the gtiset box, enter the name of the output file; we will use gti.fits. In the expression box, enter the filtering expression. We will set the upper limit to 0.2 ct/s: RATE < 0.2

2)

Click "Run".

11.5.4 Applying the GTI

Now that we have GTI file, we can apply it to the event file by running rgsproc again. rgsproc is a complex task, running several steps, with five different entry and exit points. It is not necessary to rerun all the steps in the procudure, only the ones involving filtering. Call rgsproc from the SAS GUI, then

1)

In the "global" tab, set entrystage to 3:filter and finalstage to 5:fluxing.

2)

In the "filter" tab, set auxgtitables to the new gti file, gti.fits.

3)

Click "Run".

Since we made soft links to the event files, we don't need to rename them again.

11.6 Creating the Response Matrices (RMFs)

Response matrices (RMFs) are now provided as part of the pipeline product package, but you might want create your own. The task rgsproc generates a response matrix automatically, but as noted in §11.1.4, the source coordinates are under the observer's control. The source coordinates have a profound influence on the accuracy of the wavelength scale as recorded in the RMF that is produced automatically by rgsproc, and each RGS instrument and each order will have its own RMF.

Making the RMF is easily done with the package rgsrmfgen. Please note that, unlike with EPIC data, it is not necessary to make ancillary response files (ARFS).

To make the RMFs, call rgsrmfgen from the GUI, then

1)

In the spectrumset box, enter the name of the spectrum file; for RGS1, order 1, this would be P0153950701R1S001SRSPEC1001.FIT. In the evlist box, enter the name of the event list, r1_evt2.fits. Set emin to 0.4 and emax to 2.5. Set rmfset to the output file name. We will use r1_o1_rmf.fits.

2)

Click "Run".

RMFs for the RGS1 2nd order, and for the RGS2 1st and 2nd orders, are made in a similar way.

At this point, the spectra can be analyzed or combined with other spectra.

11.7 Combining Spectra

Spectra from the same order in RGS1 and RGS2 can be safely combined to create a spectrum with higher signal-to-noise if they were reprocessed using rgsproc with spectrumbinning=lambda, as we did in §11.1 (this also happens to be the default). (Spectra of different orders, from one particular instrument, can also be merged if the were reprocessed using rgsproc with spectrumbinning=beta.) The task rgscombine also merges response files and background spectra. When merging response files, be sure that they have the same number of bins. For this example, we assume that RMFs were made for order 1 in both RGS1 and RGS2 with rgsproc.

To merge RGS1 and RGS2 spectra, call rgscombine from the SAS GUI, and then

1)

Call the rgscombine task.

2)

In the pha box, enter the spectrum files to combine, seperated by a space:
P0153950701R1S001SRSPEC1001.FIT P0153950701R2S002SRSPEC1001.FIT. In the bkg box, enter the corresponding background files:
P0153950701R1S001BGSPEC1001.FIT P0153950701R2S002BGSPEC1001.FIT In the rmf box, enter the response files:
P0153950701R1S001RSPMAT1001.FIT P0153950701R2S002RSPMAT1001.FIT

3)

In the filepha box, enter the output merged spectrum, r12_o1_srspec.fits. In the filermf box, enter the output merged response files, r12_o1_rmf.fits. In the filebkg box, enter the output merged background spectrum, r12_o1_bgspec.fits.

4)

Confirm that the rmfgrid parameter is set to the same value as that used to make the RMFs, 4000.

5)

Click ``Run''.

The spectra are ready for analysis, so we can now prepare the spectrum for fitting (§14).