So, you've received an XMM-Newton EPIC data set. What are you going
to do with it? After checking what the observation consists of
(see § 3.2), you should note
when the observation was taken. If it is a recent observation, it was likely
processed with the most recent calibrations and SAS, and you can immediately
start to analyze the Pipeline Processed data. However, if it is more
than a year old, it was probably processed with older versions of CCF and SAS
prior to archiving, and the pipeline should be rerun to generate event files
with the latest calibrations.
As noted in
Chapter 4, a variety of analysis packages can be used for the
following steps. However, as the SAS was designed for the basic reduction
and analysis of XMM-Newton data (extraction of spatial, spectral, and
temporal data), it will be used here for demonstration purposes.
SAS will be required at any rate for the production of detector response
files (RMFs and ARFs) and other observatory-specific requirements.
(Although for the simple case of on-axis point sources the canned
response files provided by the SOC can be used.)
It is strongly recommended that you keep all reprocessed data in its own
directory! SAS places output files in whichever directory it is in when a task
is called. Throughout this primer, it is assumed that the Pipleline Processed data are
in the PPS directory, the ODF data (with upper case file names, and uncompressed)
are in the directory ODF, the analysis is taking place in the PROC directory, and
the CCF data are in the directory CCF.
If your data are recent, you need only to gunzip the files and prepare the data for processing (see §5. Feel free to skip the section on repipelining and proceed to the later discussions. In any case, for simplicity, it is recommended that you change the name of the unzipped event file to something easy to type. For example, a PN event list:
Various analysis procedures are demonstrated using the Cen X-3 dataset, ObsID 0400550201. The following procedures are applicable to all XMM-Newton datasets, so it is not required that you use this particular dataset; any observation should be sufficient.
For detailed descriptions of PP data nomenclature, file contents, and which tasks can be used to view them, see Tables 3.2 and 3.3. For detailed descriptions of ODF data nomenclature and file contents, see Table 3.1.
We assume that the data was prepared and environment variables were set according
to §5, the GUI has been invoked (see §5.3),
and we are in our working directory, ``PROC''.
From the upper window of the GUI, select epproc. Epproc will automatically detect the science mode of the data, so just click "Run". (It is safe to ignore the warnings.)
By default, no intermediate files that are generated are kept. Epproc designates its output event file with ``*TimingEvts.ds''. You may want to name the new file something easy to type:
Many popular data products, such as images and light curves, are made with the task
xmmselect. The following sections detail how to make them. For an introduction
to xmmselect and a discussion on how to load an event file in it, please see
Once the event file has been loaded,
The output file image.fits can be viewed by using a standard FITS display such as ds9 (see Fig. 9.1).
The filtering expressions for PN in Timing Mode is:
The first expression will select good events with PATTERN between 0 and 4.
The PATTERN value is similar the GRADE selection for ASCA data,
and is related to the number and pattern of the CCD pixels triggered for a given
event. The PATTERN assignments are: single pixel events: PATTERN == 0,
double pixel events: PATTERN in [1:4], triple and quadruple events:
PATTERN in [5:12].
The second keyword in the expression, PI, selects the preferred pulse
height of the event. Since we are working with PN data, this should be between
200 and 15000 eV. This should clean up the image significantly with most of the
rest of the obvious contamination due to low pulse height events. Setting the
lower PI channel limit somewhat higher (e.g., to 300 eV) will eliminate much of
Finally, the #XMMEA_EP filter provides a canned screening set
of FLAG values for the event. (The FLAG value provides a bit encoding of various
event conditions, e.g., near hot pixels or outside of the field of view.)
Setting FLAG == 0 in the selection expression provides the
most conservative screening criteria and should always be used when serious
spectral analysis is to be done on PN data.
To filter the data using xmmselect,
Sometimes, it is necessary to use filters on time in addition to those mentioned
above. This is because of soft proton background flaring, which can have count
rates of 100 counts/sec or higher across the entire bandpass. To determine if our
observation is affected by background flaring, we can make a light curve with
xmmselect. For the time binning, we will set it to something reasonable
(usually between 10 and 100 s).
Load the filtered event file, and then
The output file pn_ltcrv.fits can be viewed by using fv:
In the fv pop-up window, the RATE extension will be available in the second row (index 1, as numbering begins with 0). Select ``PLOT'' from this row, and select the column name and axis on which to plot it. The light curve is shown in Fig. 9.2. No flares are evident, so we will continue to the next section. However, if a dataset does contain flares, they should be removed in the same way as shown for EPIC Imaging mode data in §7.6.
The first step in extracting a spectrum from PN Timing data is to make an image of the event file over the energy range we are interested in; for this example, we'll say 0.5-15 keV. And since this is the PN, we need to remember to set (FLAG==0) to get a high-quality spectrum. Thus, the filtering expression would be set to (FLAG==0) && (PI in [500:15000]). So, with pn_filt.fits loaded in xmmselect,
As can be seen in Figure 9.3 (top), the source is centered on RAWX=37. We will extract this and the 10 pixels on either side of it.
For the background, the extraction area should be as far from the source as possible. However, sources with 200 ct/s (like our example!) are so bright that they dominate the entire CCD area, and there is no source-free region from which to extract a background. (It goes without saying that this is highly energy-dependent.) In such a case, it may be best not to subtract a background. Users are referred to Ng et al. (2010, A&A, 522, 96) for an in-depth discussion. While this observation is too bright to have a good background extraction region, the process is shown below nonetheless for the sake of demonstration:
Depending on how bright the source is and what modes the EPIC detectors are in, event pile
up may be a problem. Pile up occurs when a source is so bright that incoming X-rays strike
two neighboring pixels or the same pixel in the CCD more than once in a read-out cycle. In
such cases the energies of the two events are in effect added together to form one event.
If this happens sufficiently often, 1) the spectrum will appear to be harder than it actually
is, and 2) the count rate will be underestimated, since multiple events will be undercounted.
Briefly, we deal with it in PN Timing data essentially the same way as
in Imaging data, that is, by using only single pixel events, and/or removing the regions
with very high count rates, checking the amount of pile up, and repeating until it is no
longer a problem. We recommend to always check for it.
Note that this procedure requires as input the event files created when the spectrum was made, not the usual time-filtered event file.
To check for pile up,
The output of epatplot is a postscript file, pn_epat.ps, which may be
viewed with viewers such as gv, containing two graphs describing the distribution
of counts as a function of PI channel; see Figure 9.4.
A few words about interpretting the plots are in order. The top is the distribution of
counts versus PI channel for each pattern class (single, double, triple, quadruple),
and the bottom is the expected pattern distribution (smooth lines) plotted over
the observed distribution (histogram). The lower plot shows the model
distributions for single and double events and the observed distributions. It also
gives the ratio of observed-to-modeled events with 1- uncertainties for single and
double pattern events over a given energy range. (The default is 0.5-2.0 keV; this can be
changed with the pileupnumberenergyrange parameter.) If the data is not piled up,
there will be good agreement between the modeled and observed single and double event
pattern distributions. Also, the observed-to-modeled fractions for both singles and doubles
in the 0.5-2.0 keV range will be unity, within errors. In contrast, if the data is piled up,
there will be clear divergence between the modeled and observed pattern distributions, and the
observed-to-modeled fraction for singles will be less than 1.0, and for doubles, it will
be greater than 1.0.
Finally, when examining the plots, it should noted that the observed-to-modeled fractions
can be inaccurate. Therefore, the agreement between the modeled and observed single and double
event pattern distributions should be the main factor in determining if an observation is
affected by pile up or not.
Examining the plots, we see that there is a large difference between the modeled and
observed single and double pattern events, and that the observed-to-model fraction for
doubles is over 1.0, indicating that the observation is piled up.
There are a couple ways to deal with pile up. First, you can use event file filtering
procedures to include only single pixel events (PATTERN==0), as these events are
less sensitive to pile up than other patterns.
You can also excise areas of high count rates, i.e., the boresight column and several
columns to either side of it. (This is analogous to removing the inner-most regions of
a source in Imaging data.) The spectrum can then be re-extracted and you can continue your
analysis on the excised event file.
As with Imaging data, it is recommended that you take an iterative approach: remove an
inner region, extract a spectrum, check with epatplot, and repeat, each time
removing a slightly larger region, until the model and observed pattern distributions
To extract only the columns to either side of the boresight,
Be aware that if you do this and are using SAS v. 13.x or older, you will need to use a non-standard way to make the ancillary files (ARFs) for your spectrum! This is discussed further in a later section.
Now that we are confident that our spectrum is not piled up, we can continue by finding the source and background region areas. (This process is identical to that used for Imaging data.) This is done with the task backscale, which takes into account any bad pixels or chip gaps, and writes the result into the BACKSCAL keyword of the spectrum table. To find the source and background extraction areas, call backscale and then
If you extracted a background spectrum, follow the same steps to find its extraction area, changing the input spectrum file to bkg_pi.fits.
If you are using SAS v. 14 or higher, making the RMF and ARF for PN data in TIMING mode is exactly the same as in IMAGING mode, even if you had to excise piled up areas. This is a change from earlier SAS versions; if you are working with an older SAS, you will need to use the special recipe below to generate the ARF (the method to make a RMF file is the same as shown here.)
To make the RMF, call rmfgen from the GUI, and then
To make the ARF, call arfgen from the GUI, and then
If you excised regions to make a spectrum and are using SAS v. 13.x or older, you will
need to make an ARF for the full extraction area, another one for the piled up area, and then
subtract the two to find the ARF for the non-piled regions. To get those, we will need the spectra
of the full extraction area and the excised area. We already have it for the full extraction area,
so for the excised area, use the same procedure as in §9.5, but change
the "Selection Expression" to (FLAG==0) && (PI in [500:15000]) && (RAWX in [29:45]), set the
filteredset parameter to pn_filt_source_Excised.fits, and spectrumset
Now we can use the spectra to make the ARFs. Invoke arfgen from the SAS GUI, and then
Use the same procedure to make the ARF for the excised region, changing the arfset parameter
to source_arf_Excised.fits and spectrumset parameter to source_pi_Excised.
We can now subtract them. This is easiest to do on the command line:
At this point, the spectrum is ready to be analyzed, so skip ahead to
prepare the spectrum for fitting (§13).
The timing data can also be examined in Xronos; this is discussed in §15.