XMM-NEWTON DATA ANALYSIS WITH PERL SCRIPTS IN HERA
The GOF has developed Perl scripts in an effort to simplify the EPIC data reduction process and make XMM data more accessible to astrophysicists and students who may not be very familiar with the details of working with SAS. These have been implemented in Hera. A short guide to their use for both the GUI and the Command Window is available below, using a Lockman Hole observation (ObsID 0147511601) for the examples. All the steps given here are the same as in the ABC Guide.
Scripts for RGS and OM are currently in development.
EPIC Scripts
A Walk-Through with the GUI
A Walk-Through with the Command Window
EPIC Scripts
Short descriptions for each script are given below, as are the input parameters, both mandatory and optional . Possible values for each input parameter are in parentheses.
- sas_reprocess_epic: repipeline the data and apply standard filters. process_mos (Y/N), process_pn (Y/N)
- sas_make_expmap_epic: make exposure maps based on default energy ranges or user-supplied ranges. event_file (text file), energy_var (Y/N), energy_file (text file)
- sas_make_bkg_spectrum_epic: make a background spectrum. event_file (text file), position_file (text file)
- sas_make_filteredevents_epic: remove flares from event files. event_files (text file), times_file (text file)
- sas_make_image_epic: make images for display in ds9 or fv. event_files (text file), position_var (Y/N), position_file (text file)
- sas_make_lc_epic: make light curves. event_file (text file), lcbin (float or integer), position_var (Y/N), position_file (text file)
- sas_make_spectrum_epic: extract spectra, make RMFs and ARFs, check for pile-up. event_file (text file), position_file (text file)
A Walk-Through with the GUI
The scripts are located in the "Experimental" folder, under "XMM-SAS" in the "Available Tools" window of the interface. As always when working with Hera, be sure to call tasks from your working directory.
- Download the data to your Hera account and unzip the files in the ODF directory.
- Prepare the data, then reprocess and filter them. Highlight the ODF
directory, then call "sas_reprocess_epic". In the pop-up window, verify
the working directory is correct, and select which cameras to process.
The output event files will appear in the directory "proc" and follow the naming
convention "instrument_evt2.fits". sas_reprocess_epic will automatically
update your environment variables to point to the *SUM.SAS files it makes.
(You may need to refresh the directory listing to see the new files and
directory.)
- Download the event files to your local machine for easy examination and prepare some text
files for future use. (Files are copied between the host and
client by right-clicking and dragging.) We're interested in data from all
three detectors, so make a file listing all their event files on your local
machine, then upload it to your Hera account:
MyComputer> ls *evt2.fits > events.txt
There are two somewhat bright sources in the field. We can list these in a text file, using the convention:
unit
where
x y inner_radius outer_radius
unit = unit the coordinates and radii are in; can be either d (decimal degrees) or p (pixels)
So the region file in degree space would look like:
inner_radius = inner radius of annulus region to extract. Set this to 0 to extract a circular region.
outer_radius = outer radius of annulus region to extract. This is equal to the radius of a circular region.
> more coords.txt
d
163.16561 57.409134 0 0.005
163.32071 57.597477 0 0.005
If you wanted to use pixel units instead, the file would look like this:
> more coords.txt
p
25748.549 18870.529 0 600
19765.33 32438.468 0 600
We can also make a file denoting background regions for these sources:
> more coords_bkg.txt
d
163.16561 57.409134 0.005 0.015
163.32071 57.597477 0.005 0.015
If you wanted to use pixel units instead, the file would look like this:
> more coords_bkg.txt
p
25748.549 18870.529 360 1080
19765.33 32438.468 360 1080
When the region files have been made, upload them to the "proc" directory in Hera.
- Make images with sas_make_image_epic. In the pop-up window,
verify the working directory is correct, and enter the file that contains
the event file names, "events.txt". The full path is not needed. By default,
this task will make an image of the full field of view, but it can also make
images of specific regions. This can be done by selecting this option in
the pop-up and entering the region file name. The output will be in the
"image" directory and will follow the naming convention "instrument_img.fits".
If you use a region file, the images of each source will be numbered in the
order in which they are given in the file.
- Make light curves to see if there are any flares that need to be removed
with sas_make_lc_epic. In the pop-up window, verify the working directory
is correct, and enter the file that contains the event file names, "events.txt".
The full path is not needed. Enter the time bin size; for this dataset,
10 or 100 is a good choice. By default, this task will make a light curve of
the full field of view, but it can also make light curves of specific regions
by selecting this option in the pop-up and supplying the region file name.
If there is more than one region, the light curves of each object will be
numbered in the order in which they are given in the file.
The output will be in the "lightcurve" directory.
- Examine the lightcurves. If necessary, filter the event files to remove
flares. This is done by making a text file on your local machine that lists
the starting and ending BAD times, one per line, and uploading this to your
Hera account. For our example case, there are several time ranges
contaminated with flares:
> more bad_times.txt
1.54854e8 1.54865e8
1.54870e8 1.54873e8
1.54912e8 1.54921e8
1.54926e8 1.54952e8
To remove these time ranges, make sure you are in the "proc" directory and call sas_make_filteredevents_epic. In the pop-up window, verify that the working directory is correct, then enter the event file list and the bad times list. There will be several "DssBlockClash" warnings; these can be ignored. The output will be placed in the working directory and follow the naming convention "instrument_flt_evt2.fits". Remember to update the events file list on your local machine with the filtered files and upload the new file list!
- Use sas_make_spectrum_epic to extract a spectrum, test it for pile-up,
and produce response and ancillary files. In the pop-up window, verify
that the working directory is correct, then enter the event file list and
the region file. The output will be placed in the directory "spectrum".
Each source in the region file will have a spectrum (*.pi), a postscript file
comparing the measured and modeled count rates produced with the SAS task
epatplot, and response files (*.rmf and *.arf). These products will be
numbered in the order that the sources are given in the file.
- Make a background spectrum with sas_make_bkg_spectrum_epic. In
the pop-up window, verify that the working directory is correct, then enter
the event file list and the background region file, like what was shown in
step 3. The output spectrum will be placed in the "spectrum" directory. Each
spectrum will be numbered in the order that the sources are given in the
region file. Unlike sas_make_spectrum_epic, this task only extracts
the spectrum, and does not check for pile up or generate response files.
- Make exposure maps with sas_make_expmap_epic. The default energy
range for exposure maps is 2-4.5 keV. However, if you are interested in a
different energy range (or several ranges), this is easily accomodated:
make a text file on your local machine that contains the desired minimum and
maximum energies (in eV), one per line, and upload it to your Hera account's
"proc" directory.
For instance, the file "energy.txt" may look like this:
MyComputer> more energy.txt
1000 2000
2000 3000
After uploading the text file, call sas_make_expmap_epic. In the pop-up window, verify that the working directory is correct, then enter the event file list and the energy range list. The output is placed in the "expmap" directory and follows the naming convention "instrument_expmap_LowEnergy-HighEnergykeV.fits". So for our example, six exposure maps will be made: m1_expmap_1-2keV.fits, m1_expmap_2-3keV.fits, m2_expmap_1-2keV.fits, m2_expmap_2-3keV.fits, pn_expmap_1-2keV.fits, and pn_expmap_2-3keV.fits.
A Walk-Through with the Command Window
As always when working with Hera, be sure to call up the Command Window from your working directory. If you want to switch directories, close the Window you are working in, highlight the desired directory in the GUI, and open a new Window.
- Download the data to your Hera account and unzip the files in the ODF directory.
- Prepare the data, then reprocess and filter them. Highlight the ODF
directory and bring up a Command Window. For this example, we want to
reprocess data from the PN and MOS detectors, so enter
> sas_reprocess_epic process_mos=y process_pn=y
The filtered event files will appear in the directory "proc" and follow the naming convention "instrument_evt2.fits". Hera will automatically update your environment variables to point to the new *SUM.SAS file.
- Download the event files to your local machine for easy examination and prepare
some text files for future use. (Files are copied between the host and client by
right-clicking and dragging.) We're interested in data from all three detectors,
so make a file listing all their event files on your local machine, then upload
it to your Hera account:
MyComputer> ls *evt2.fits > events.txt
There are two somewhat bright sources in the field. We can list these in a text file, using the convention:
unit
where
x y inner_radius outer_radius
unit = unit the coordinates and radii are in; can be either d (decimal degrees) or p (pixels)
So the region file in degree space would look like:
inner_radius = inner radius of annulus region to extract. Set this to 0 to extract a circular region.
outer_radius = outer radius of annulus region to extract. This is equal to the radius of a circular region.
> more coords.txt
d
163.16561 57.409134 0 0.005
163.32071 57.597477 0 0.005
If you wanted to use pixel units instead, the file would look like this:
> more coords.txt
p
25748.549 18870.529 0 600
19765.33 32438.468 0 600
We can also make a file denoting background regions for these sources:
> more coords_bkg.txt
d
163.16561 57.409134 0.005 0.015
163.32071 57.597477 0.005 0.015
If you wanted to use pixel units instead, the file would look like this:
> more coords_bkg.txt
p
25748.549 18870.529 360 1080
19765.33 32438.468 360 1080
When the region files have been made, upload them to the "proc" directory in Hera.
- Make images with sas_make_image_epic. Close the current Command Window,
highlight the "proc" directory in the GUI, then open a new Command Window
and enter
> sas_make_image_epic event_files=events.txt position_var=n
To make images of the regions in the coordinate files, enter
> sas_make_image_epic event_files=events.txt position_var=y position_file=coords.txt
The output will be in the "image" directory and will follow the naming convention "instrument_img.fits". If you use a region file, the images of each source will be numbered in the order in which they are given in the file.
-
Make light curves to see if there are any flares that need to be removed with
sas_make_lc_epic. For this dataset, a time bin size of 10 or 100 is a
good choice. For this example, we will let the time bin size be 10.
By default, this task will make a light curve of the full field of view, but it
can also make light curves of specific regions by setting "position_var" to "y"
and supplying the region file name.
If there is more than one region, the light curves of each object will be numbered
in the order in which they are given in the file. The output will be in the
"lightcurve" directory.
> sas_make_lc_epic event_file=events.txt lcbin=10 position_var=y position_file=coords.txt
- Examine the lightcurves. If necessary, filter the event files to remove flares.
This is done by making a text file on your local machine that lists the starting and
ending BAD times, one per line, and uploading this to your Hera account. For our
example case, there are several time ranges contaminated with flares:
> more bad_times.txt
1.54854e8 1.54865e8
1.54870e8 1.54873e8
1.54912e8 1.54921e8
1.54926e8 1.54952e8
To remove these time ranges, enter:
> sas_make_filteredevents_epic events_files=events.txt time_file=bad_times.txt
There will be several "DssBlockClash" warnings; these can be ignored. The output will be placed in the working directory and follow the naming convention "instrument_flt_evt2.fits". Remember to update the events file list on your local machine with the filtered files and upload the new file list!
- Use sas_make_spectrum_epic to extract a spectrum, test it for pile-up, and
produce response and ancillary files by entering:
> sas_make_spectrum_epic event_file=events.txt position_file=coords.txt
The output will be placed in the directory "spectrum". Each source in the region file will have a spectrum (*.pi), a postscript file comparing the measured and modeled count rates produced with the SAS task epatplot, and response and ancillary files (*.rmf and *.arf). These products will be numbered in the order that the sources are given in the file.
- Make a background spectrum with sas_make_bkg_spectrum_epic by entering:
> sas_make_bkg_spectrum_epic event_file=events.txt position_file=coords_bkg.txt
The output spectrum will be placed in the "spectrum" directory. Each spectrum will be numbered in the order that the sources are given in the region file. Unlike sas_make_spectrum_epic, this task only extracts the spectrum, and does not check for pile up or generate response files.
- Make exposure maps with sas_make_expmap_epic. If there are particular energy
ranges you are interested in, you can enter a text file with those ranges. The file must
list the minimum and maximum energies (in eV), one per line. Make it on your
local machine and upload it to the "proc" directory in your Hera account. For instance,
MyComputer> more energy.txt
1000 2000
2000 3000
will produce two exposure maps for each event file listed in events.txt, over the ranges 1-2 keV and 2-3 keV. If no energy file is entered, the default range 2-4.5 keV is used. The output is placed in the "expmap" directory and follows the naming convention "instrument_expmap_LowEnergy-HighEnergykeV.fits". So for our example, six exposure maps will be made: m1_expmap_1-2keV.fits, m1_expmap_2-3keV.fits, m2_expmap_1-2keV.fits, m2_expmap_2-3keV.fits, pn_expmap_1-2keV.fits, and pn_expmap_2-3keV.fits.
> sas_make_expmap_epic event_file=events.txt energy_var=y energy_file=energy.txt
If you have any questions concerning XMM-Newton send e-mail to xmmhelp@lists.nasa.gov