Next: 9. EPIC-MOS Data Processing Up: XMM ABC Guide Previous: 7. EPIC Data Processing Contents
Upon receipt of an XMM-Newton data set, after checking what the
observation consists of (see § 3.2), you should note
when the observation was taken. If it is a recent observation, it was likely
processed with the most recent calibrations and SAS, and you can immediately
start to analyze the Pipeline Processed data. However, if it is more
than a year old, it was probably processed with older versions of CCF and SAS
prior to archiving, and the pipeline should be rerun to generate event files
with the latest calibrations.
As noted in
Chapter 5, a variety of analysis packages can be used for the
following steps. However, as the SAS was designed for the basic reduction
and analysis of XMM-Newton data (extraction of spatial, spectral, and
temporal data), it will be used here for demonstration purposes.
SAS will be required at any rate for the production of detector response
files (RMFs and ARFs) and other observatory-specific requirements,
though the canned response files provided by the SOC can be used
if the source is an on-axis point source.
It is strongly recommended that you keep all reprocessed data in its own
directory! SAS places output files in whichever directory it is in when a task
is called. Throughout this primer, it is assumed that the Pipleline Processed data are
in the PPS directory, the ODF data (with upper case file names, and uncompressed)
are in the directory ODF, the analysis is taking place in the PROC directory, and
the CCF data are in the directory CCF.
If your data are recent, you need only to gunzip the files and prepare the data for processing (see §6. Feel free to skip the section on repipelining and proceed to the later discussions. In any case, for simplicity, it is recommended that you change the name of the unzipped event file to something easy to type. For example, a PN event list:
| cp PPS/PiiiiiijjkkPNSlllTIEVLI0000.FTZ PROC/pn.fits |
where
|
Various analysis procedures are demonstrated using the Holmberg II dataset, ObsID 0843840201. The following procedures are applicable to all XMM-Newton datasets, so it is not required that you use this particular dataset; any Timing Mode observation should be sufficient.
For detailed descriptions of PP data nomenclature, file contents, and which tasks can be used to view them, see Tables 3.2 and 3.3. For detailed descriptions of ODF data nomenclature and file contents, see Table 3.1.
We assume that the data was prepared and environment variables were set
according to §6. In the window where SAS was
initialized, in your “processing directory” PROC, run epproc to produce calibrated photon event files for the
PN camera.
Note that epproc will automatically detect what mode the data were taken
in.
On the command line, just type:
| epproc |
or, alternatively,
| epchain datamode=TIMING |
To do this with the GUI, select epproc. Epproc will automatically detect if the data are in Timing mode, so just click “Run”. (It is safe to ignore the warnings.) Alternatively, select epchain. In the “General” tab, next to datamode, select the TIMING option and click “Run”.
If your data are in Burst Mode:
| epproc burst=yes |
or, alternatively,
| epchain datamode=BURST |
To do this with the GUI, select epproc. In the “Data Processing”
tab, check the burst box and click “Run”. Alternatively, select
epchain. In the “General” tab, next to datamode,
select the BURST option and click “Run”.
By default, the task does not keep any intermediate files that were generated. Epchain maintains the naming convention described in §3.2.4. Epproc designates its output event files with “Evts.ds”, so “*TimingEvts.ds” and “*BurstEvts.ds” denote the timing and burst mode event lists, respectively. In any case, you may want to name the new files something easy to type. For example, to make a soft link to the new PN event files generated by epproc, type
| ln -s 3716_0843840201_EPN_S003_TimingEvts.ds pn.fits |
To do so with the output of epchain, type
| ln -s P0843840201PNS003TIEVLI0000.FIT pn.fits |
The filtering expression for the PN in Timing or Burst mode is:
(PATTERN  4) && (PI in [700:12000]) && #XMMEA_EP
|
The first expression will select good events with PATTERN in the 0 to 4 range.
Event patterns are discussed in §4, but to summarize here, it is related
to the number and pattern of the CCD pixels triggered for a given event.
Single pixel events have PATTERN = 0 and double pixel events have
PATTERN ranging from 1 to 4.
The second keyword in the expressions, PI, selects the preferred pulse
height of the event and is related to PHA; for the PN in Timing or Burst mode,
this should be between 700 and 12000 eV.
Finally, the #XMMEA_EP filter provides
a canned screening set of FLAG values for the event. (The FLAG value provides a
bit encoding of various event conditions, e.g., near hot pixels or outside of the
field of view.) Setting FLAG == 0 in the selection expression provides the
most conservative screening criteria and should always be used when serious spectral
analysis is to be done on PN data.
To filter the data, type
evselect table=pn.fits filteredset=pn_filt.fits \
 expression='(PATTERN 4) && (PI in [700:12000]) && #XMMEA_EP'
|
where
| table - input event table
expression - filtering expression filteredset - output file name |
To do this with the GUI,
| 1) | Call evselect. |
| 2) | In the “General” tab, in table, select the event file, pn.fits, and
then the EVENTS extension.
Check the “keepfilteroutput” and “withfilteredset” boxes, then set
filteredset to the output file name, pn_filt.fits. In the
expression box, enter the filtering expression,
(PATTERN 4) && (PI in [700:12000]) && #XMMEA_EP.
|
| 3) | Click “Run”. |
Sometimes, it is necessary to use filters on time in addition to those
mentioned above, due to soft proton flaring. This is discussed
more in §4.2.
To determine if our observation is affected by background flaring, we can examine the high energy, single-pattern event light curve. For the time binning, we will set it to something reasonable (usually between 10 and 100 s). We'll start off by making a directory to put our light curve in.
| mkdir ltcrv
cd ltcrv ln -s ../pn_filt.fits evselect table=pn_filt.fits withrateset=yes rateset=pn_ltcrv.fits \ maketimecolumn=yes timebinsize=50 makeratecolumn=yes \
expression='(PATTERN==0) && (PI in [10000:12000])'
|
where the parameters are as defined above and
| table - input event table
withrateset - make a light curve rateset - name of output light curve file maketimecolumn - control to create a time column timebinsize - time binning (seconds) makeratecolumn - control to create a count rate column, otherwise a count column will be created |
To do this with the GUI,
| 1) | Call evselect. |
| 2) | In the “General” tab, in table, select the event file (e.g. m1_filt.fits and then the EVENTS extension. In the expression box, enter the filtering expression, (PATTERN==0) && (PI in [10000:12000]). In the “Lightcurve” tab, check the withrateset box. In the rateset box, enter the name of the output lightcurve, pn_ltcrv.fits. In the timebinsize box, enter the bin size in seconds. Check the maketimecolumn and makeratecolumn boxes. |
| 3) | Click “Run”. |
The output file pn_ltcrv.fits can be viewed by using the ftool fv. If you have not called ftools yet, remember to do so before calling fv. (There is no harm in calling it more than once.) To see the light curve, type
| ftools
fv pn_ltcrv.fits & |
In the fv pop-up window, the RATE extension will be available in the second row
(index 1, as numbering begins with 0). Select “PLOT” from this row, and select the column
name and axis on which to plot it. The light curve is shown in Fig. 8.1.
Some flares are evident in the middle of the observation, which can be removed by using the same methods as for Imaging mode data (§7.4). Any of the methods described in that section will work on PN Timing or Burst mode data, so which one to use is a matter of the user's preference and what makes the most sense for the science being done. Here, we will make a secondary GTI file with tabgtigen, choosing to keep times where the rate is 0.1 c/s or lower:
| tabgtigen table=pn_ltcrv.fits gtiset=gti.fits expression='RATE<=0.1' |
where parameter names are as defined previously and
| gtiset - output secondary GTI file
|
To do this with the GUI,
| 1) | Call tabgtigen. |
| 2) | In table, enter the event file, pn_filt.fits
In the expression box, enter the filtering expression,
RATE 0.1. In the gtiset box, enter the output file name,
gti.fits.
|
| 3) | Click “Run”. |
And we apply it with evselect:
| evselect table=pn_filt.fits filteredset=pn_filt_time.fits expression='GTI(gti.fits,TIME)' |
To do this with the GUI,
| 1) | Call evselect. |
| 2) | In the “General” tab, in table, enter the event file, pn_filt.fits. Check the “keepfilteroutput” and “withfilteredset” boxes, then set filteredset to the output file name, pn_filt_time.fits. In the expression box, enter the filtering expression, GTI(gti.fits,TIME). |
| 3) | Click “Run”. |
The first step in extracting a spectrum from PN Timing data is to make an image of the event file over the energy range we are interested in. Since this is the PN, we need to remember to set (FLAG==0) to get a high-quality spectrum. We'll make a directory for the spectra first and a link to the filtered event file first.
| cd ../
mkdir epicspec cd epicspec ln -s ../ltcrv/pn_filt_time.fits evselect table=pn_filt_time.fits imageset=pn_image.fits xcolumn=RAWX ycolumn=RAWY \ imagebinning=binSize ximagebinsize=1 yimagebinsize=1 expression='(FLAG==0)'
|
where parameters are as defined above and
| withimageset - make an image
imageset - name of output image xcolumn - event column for X axis ycolumn - event column for Y axis imagebinning - form of binning, force entire image into a given size or bin by a specified number of pixels ximagesize - output image pixels in X yimagesize - output image pixels in Y |
To do this with the GUI,
| 1) | Call evselect. |
| 2) | In the “General” tab, in table, enter the event file, pn_filt_time.fits.
In the expression box, enter the filtering expression, FLAG==0.
In the “Image” tab, check the withimageset box. In the imageset box, enter the output image name, pn_image.fits. Verify that xcolumn and ycolumn are set to RAWX and RAWY, respectively. Set “binning” to binSize, and verify that both ximagebinsize and yimagebinsize are set to 1. |
| 3) | Click “Run”. |
The image can be displayed with ds9 and is shown in Fig. 8.2 (left). The source is centered on RAWX=37; we will extract this and the 5 pixels on either side of it:
| evselect table=pn_filt_time.fits spectrumset=source_pi.fits \
energycolumn=PI spectralbinsize=5 specchannelmin=0 specchannelmax=20479 \
filteredset=pn_filt_source.fits \
expression='(FLAG==0) && (RAWX in [32:42])'
|
where the keywords are the same as in §8.2, and
| energycolumn - energy column
spectrumset - name of output spectrum spectralbinsize - size of bin, in eV specchannelmin - minimum of spectral range specchannelmax - maximum of spectral range |
To do this with the GUI,
| 1) | Call evselect. |
| 2) | In the “General” tab, in table, enter the event file, pn_filt_time.fits.
Check keepfilteroutput and withfilteredset boxes.
In the filteredset box, enter the name of the event file output, pn_filtered.fits.
In the expression box, enter the filtering expression,
(FLAG==0) && (RAWX in [32:42]). In the energycolumn box, enter PI. In the “Spectrum” tab, check the withspectrumset box. In the spectrumset box, enter the output spectrum name, source_pi.fits. Verify that spectralbinsize is set to 5. Check the withspecranges box, and set specchannelmin and specchannelmax to 0 and 20479, respectively. |
| 3) | Click “Run”. |
For the background, the extraction area should be as far from the source as possible.
However, some sources are so bright (
200 ct/s) that they dominate
the entire CCD area, and there is no source-free region from which to extract a background.
(It goes without saying that this is highly energy-dependent.) In such a case, it may be
best not to subtract a background. Users are referred to Ng et al. (2010, A&A, 522, 96)
for an in-depth discussion. Luckily, we don't need to worry about this for the Holmberg II
data, so we will proceed as normal.
| evselect table=pn_filt_time.fits spectrumset=bkg_pi.fits \
expression='(FLAG==0) && (RAWX in [3:5])' \
energycolumn=PI spectralbinsize=5 withspecranges=yes specchannelmin=0 \
specchannelmax=20479 filteredset=pn_filt_bkg.fits
|
To do this with the GUI,
| 1) | Call evselect. |
| 2) | In the “General” tab, in table, enter the event file, pn_filt_time.fits.
Check keepfilteroutput and withfilteredset boxes.
In the filteredset box, enter the name of the event file output, pn_filtered.fits.
In the expression box, enter the filtering expression,
(FLAG==0) && (RAWX in [3:5]). In the energycolumn box, enter PI. In the “Spectrum” tab, check the withspectrumset box. In the spectrumset box, enter the output spectrum name, bkg_pi.fits. Verify that spectralbinsize is set to 5. Check the withspecranges box, and set specchannelmin and specchannelmax to 0 and 20479, respectively. |
| 3) | Click “Run”. |
|
Depending on how bright the source is and what modes the EPIC detectors are in, event pile up may be a problem. This is discussed in some detail in §7.8, and users are urged to refer to that section. The effects of pile up, and the use of the SAS task epatplot to diagnose it, are the same regardless of which science mode PN is in. For convenience, we also show the command here. Note that this task requires as input the event files created when the spectrum was made, not the usual time-filtered event file. To check for pile up:
| epatplot set=pn_filt_source.fits plotfile=pn_epat.ps useplotfile=yes \
withbackgroundset=yes backgroundset=pn_filt_bkg.fits
|
where
| set - input events file
plotfile - output postscript file useplotfile - flag to use file name from "plotfile" withbackgroundset - use background event set for background subtraction? backgroundset - name of background event file |
To do this with the GUI,
| 1) | Call epatplot. |
| 2) | In the “0” tab, in the set box, enter the name of the event file that was made when the spectrum was extracted, pn_filtered.fits. We want to send the output to a pdf file, so set useplotfile to yes, and enter the file name in the plotfile box. We will use pn_epat.pdf. |
| 3) | In the “1” tab, set withbackgroundset to yes and enter the name of the event file that was made when the background spectrum was made, pn_filtered_bkg.fits. Toggle withsrcxy to no. |
| 4) | Click “Run”. |
The output plot, pn_epat.pdf, can be seen with standard pdf viewers and is
shown in Figure 8.3.
How to interpret it is discussed in §7.8.
Examining the plots for the Holmberg II data, we see that there is agreement between the modeled and observed single and double pattern events, and that the observed-to-model fractions for singles and doubles are consistent with 1.0, indicating that there is no pile up. If there had been, we would have followed the same procedure as for Imaging mode, with the only difference being the shape of the region that is excised: In Imaging mode, the piled-up source core is a circle; in Timing mode, it is a rectangle corresponding to the boresight columns. See §7.9 for more information.
Now that we are confident that our spectrum is not piled up, we can continue by
finding the source and background region areas. This process is identical to that
used for Imaging data in §7.10, but is shown here for
convenience. We will use the task backscale, which takes
into account any bad pixels or chip gaps, and writes the result into the
BACKSCAL keyword of the spectrum table.
To find the source and background extraction areas:
| backscale spectrumset=source_pi.fits badpixlocation=pn_filt_time.fits
backscale spectrumset=bkg_pi.fits badpixlocation=pn_filt_time.fits |
where
| spectrumset - spectrum file
badpixlocation - event file containing the bad pixels |
To do this with the GUI,
| 1) | Call backscale. |
| 2) | In the “Main” tab, in the spectrumset box, enter the spectrum, pn_pi.fits. |
| 3) | In the “Effects” tab, in the badpixlocation box, enter the event file, pn_filt_time.fits. |
| 4) | Click “Run”. |
Repeat these steps with the appropriate changes for the background spectrum.
Next, we need to reformat the detector response by making redistribution matrix files (RMFs) and ancillary response files (ARFs). Making the RMF and ARF for PN data in Timing mode is exactly the same as in Imaging mode. To make the RMF:
| rmfgen rmfset=source_rmf.fits spectrumset=source_pi.fits |
where
| rmfset - output file
spectrumset - spectrum file |
To do this with the GUI,
| 1) | Call rmfgen. |
| 2) | In the “Main” tab, in the spectrumset box, enter the spectrum file name, pn_pi.fits. In the rmfset box, enter the output RMF file name, pn_rmf.fits. |
| 3) | Click “Run”. |
To make the ARF:
| arfgen arfset=source_arf.fits spectrumset=source_pi.fits \
withrmfset=yes rmfset=source_rmf.fits badpixlocation=pn_filt_time.fits
|
where
| arfset - output file
spectrumset - spectrum file withrmfset - use the RMF dataset to define the ARF energy grid? rmfset - RMF file badpixlocation - the file containing the bad pixel locations |
To do this with the GUI,
| 1) | Call arfgen. |
| 2) | In the “Main” tab, set the arfset parameter to the ARF file name, pn_arf.fits. Set the spectrumset parameter to the spectrum file, pn_pi.fits. |
| 3) | In the “Effects” tab, set the badpixlocation keyword to the event file name from which the spectrum was extracted, pn_filt_time.fits. |
| 4) | In the “Calibration” tab, check the withrmfset box and set the rmfset keyword to the RMF file name, in this case, pn_rmf.fits. |
| 5) | Click “Run”. |
If we had not run the backscale task, we could have used an option in arfgen
to run it, by simply adding setbackscale=yes to the arfgen call on the
command line, or by going to the “Backscale” tab and toggling the keyword to yes in the GUI.
At this point, the spectrum is ready to be analyzed. This is discussed in §12.1, including how to rebin it using ftools. However, SAS also has a rebinning task, which is demonstrated in §7.11. The timing data can also be examined in Xronos; this is discussed in §13.
To aid in scripting, the data preparation commands from §6.1 and §6.2 and the commands used in this chapter are consolidated below.
| cd ODF
setenv SAS_ODF `pwd` ; setenv SAS_ODFPATH `pwd` cifbuild setenv SAS_CCF `pwd`/ccf.cif odfingest setenv SAS_ODF `pwd`/*SUM.SAS
cd ..
mkdir ltcrv
cd ..
|
![\includegraphics[scale=0.15]{Holmberg2_hiE_lightcurve.eps}](img30.png)
![\includegraphics[scale=0.15]{Holmberg2_projection1.eps}](img31.png)
![\includegraphics[scale=0.15]{Holmberg2_projection2.eps}](img32.png)
![\includegraphics[scale=0.5]{Holmberg2_pn_epatplot.eps}](img33.png)