skip to content
 
XMM-Newton Guest Observer Facility

XMM-NEWTON DATA ANALYSIS WITH PERL SCRIPTS


The GOF has developed Perl scripts in an effort to simplify the EPIC and RGS data reduction process and make XMM data more accessible to astrophysicists and students who may not be very familiar with the details of working with SAS. These, plus a short guide to their use, and a summary of commands, are available below. Knowledge of SAS is not required to run the scripts. However, all scripts assume that SAS has been initialized and the environmental variables have been set. In the walk-throughs, we assume that the scripts have been placed in a convenient directory and aliased, as shown below. All the steps given here are the same as in the ABC Guide. In the EPIC example, we use the observation of the Lockman Hole (ObsID 0147511601) and for RGS, we use AB Dor (ObsID 0134520301). As always when working with SAS, be sure to call tasks from your working directory.

Scripts
A Walk-Through with EPIC
A Summary of Commands with EPIC
A Walk-Through with RGS
A Summary of Commands with RGS


Scripts


These are the available scripts with a short description of each. Clicking on each name will take you to a page with the script, where they can easily be saved by your browser.


After grabbing them with your browser, the easiest way to use these is to place them in a convenient directory and edit your .cshrc (or .bashrc or .zcshrc) file to look for them:
> cd ~
> mkdir sas_scripts/
> mv *_epic sas_scripts/
> mv *_rgs sas_scripts/
And in your .cshrc, add the appropriate aliases, e.g.:
alias make_image_epic 'perl ${HOME}/sas_scripts/make_image_epic'
alias reprocess_rgs 'perl ${HOME}/sas_scripts/reprocess_rgs'
Remember to source your .cshrc to pick up the new aliases:
> source .cshrc


A Walk-Through with EPIC

  1. Download the data to your local machine and unzip the files in the ODF directory. Call reprocess_epic to prepare the data, then reprocess and filter them. The output event files will appear in the directory "proc" and follow the naming convention "instrument_evt2.fits". reprocess_epic will automatically update your environment variables to point to the new ccf.cif and *SUM.SAS files it makes.

    > cd ODF
    > gunzip *gz
    > reprocess_epic

  2. Examine the event files and prepare some text files for future use. We're interested in data from all three detectors, so make a file listing all their event files:

    > cd ../proc
    > ls *evt2.fits > events.txt

    There are two somewhat bright sources in the field. We can list these in a text file, using the convention:

    unit
    x y inner_radius outer_radius
    where
    unit = unit the coordinates and radii are in; can be either d (decimal degrees) or p (pixels)
    inner_radius = inner radius of annulus region to extract. Set this to 0 to extract a circular region.
    outer_radius = outer radius of annulus region to extract. This is equal to the radius of a circular region.
    So the region file in degree space would look like:

    > more coords.txt
    d
    163.16561 57.409134 0 0.005
    163.32071 57.597477 0 0.005

    In pixel space, it would look like:

    > more coords.txt
    p
    25748.549 18870.529 0 600
    19765.33 32438.468 0 600

    We can also make a file denoting background regions for these sources:

    > more coords_bkg.txt
    d
    163.16561 57.409134 0.005 0.015
    163.32071 57.597477 0.005 0.015

    In pixel space, this would be

    > more coords_bkg.txt
    p
    25748.549 18870.529 360 1080
    19765.33 32438.468 360 1080

  3. Make images with make_image_epic. You will be prompted for the text file that contains the names of the event files. By default, make_image_epic will make an image of the full field of view, but it can also make images of specific regions. You can do this by entering a region file at the prompt. The output will be in the "image" directory and will follow the naming convention "instrument_img.fits". If you use a region file, the images of each source will be numbered in the order in which they are given in the file. In the "proc" directory,

    > make_image_epic
    Enter the file listing the event files: events.txt
    Should I use a region file? If so, enter the file name; otherwise, enter N: n

  4. Make light curves to see if there are any flares that need to be removed. You will be prompted for the file that contains the names of the event files and the size of the time bins. By default, make_lc_epic will make a light curve over the entire field of view, but you can also make one (or several) for specific objects by providing a region file at the prompt. The output will be in the "lightcurve" directory. If you use a region file, the lightcurves of each object will be numbered in the order in which they are given in the file. In the "proc" directory,

    > make_lc_epic
    Enter the file listing the event files: events.txt
    Should I use a region file? If so, enter the file name; otherwise, enter N: coords.txt
    Please enter the size of the time bins. 10

  5. Examine the lightcurves. If necessary, filter the event files to remove flares. This is done by making a text file that lists the starting and ending BAD times, one per line. For our example case, there are several time ranges contaminated with flares:

    > more bad_times.txt
    1.54854e8 1.54865e8
    1.54870e8 1.54873e8
    1.54912e8 1.54921e8
    1.54926e8 1.54952e8

    To remove these time ranges, in the "proc" directory,

    > make_filteredevents_epic
    Enter the file listing the event files: events.txt
    Enter the file with the times to be removed: bad_times.txt

    There will be several "DssBlockClash" warnings; these can be ignored. The output will be placed in the working directory and follow the naming convention "instrument_flt_evt2.fits". Remember to update the events file list with the new files!

  6. Use make_spectrum_epic to extract a spectrum, test it for pile-up, and produce response and ancillary files. You will be prompted for the event file list and a region file. The output will be placed in the directory "spectrum". Each source in the region file will have a spectrum (*.pi), a postscript file comparing the measured and modeled count rates produced with the SAS task epatplot, and response files (*.rmf and *.arf). These products will be numbered in the order that the sources are given in the file. In the "proc" directory,

    > make_spectrum_epic
    Enter the file listing the event files: events.txt
    Enter the region file name: coords.txt

  7. Make a background spectrum with make_bkg_spectrum_epic. You will be prompted for a background region file, like what was made in step 2. The output spectrum will be placed in the "spectrum" directory. Each spectrum will be numbered in the order that the sources are given in the region file. Unlike make_spectrum_epic, this only extracts the spectrum, and does not test for pile up or generate response files. In the "proc" directory,

    > make_bkg_spectrum
    Enter the file listing the event files: events.txt
    Enter the background region file name: coords_bkg.txt

  8. Make exposure maps with make_expmap_epic. If there are particular energy ranges you are interested in, you can make a text file and list the minimum and maximum energies (in eV), one per line, and enter it at the prompt. For instance, if you have a text file "energy.txt" that looks like this:

    > more energy.txt
    1000 2000
    2000 3000

    it will produce two exposure maps for each event file, over the ranges 1-2 keV and 2-3 keV. If no energy file is entered, the default range 2-4.5 keV is used. In any case, the output is placed in the "expmap" directory and follows the naming convention "instrument_expmap_LowEnergy-HighEnergykeV.fits". In the "proc" directory,

    > make_expmap_epic
    Enter the file listing the event files: events.txt
    Should I use an energy file? If so, enter the file name; otherwise, enter N: energy.txt

    For our example, six exposure maps will be made: m1_expmap_1-2keV.fits, m1_expmap_2-3keV.fits, m2_expmap_1-2keV.fits, m2_expmap_2-3keV.fits, pn_expmap_1-2keV.fits, and pn_expmap_2-3keV.fits.

With these data products, further analysis can be done with XSpec or CIAO, as shown in the
ABC Guide.

Summary of Commands with EPIC

You can reduce XMM data and extract useful products with minimal input by using these commands and entering the appropriate text files when prompted.

> cd ODF
> reprocess_epic
> cd ../proc
> make_image_epic
> make_lightcurve_epic
> make_filteredevents_epic
> make_expmap_epic
> make_spectrum_epic
> make_bkg_spectrum_epic


A Walk-Through with RGS

  1. Download the data to your local machine and unzip the files in the ODF directory. Call reprocess_rgs to prepare and reprocess the data. The output event files will appear in the directory "proc". reprocess_rgs will automatically update your environment variables to point to the new ccf.cif and *SUM.SAS files it makes.

    > cd ODF
    > gunzip *gz
    > reprocess_rgs

  2. Examine the event files and prepare some text files for future use. We're interested in data from both RGS detectors, so make a file listing all their event files:

    > cd ../proc
    > ls *EVENLI* > rgs12_event.list

  3. Make "banana plot" images with make_image_rgs. You will be prompted for the text file that contains the names of the event files. The output will be in the "image" directory and will follow the naming convention "instrument_pi_bc.fits" and "instrument_xd_bc.fits" for the PI vs. Beta Correction and Cross Dispersion vs. Beta Correction plots, respectively. In the "proc" directory,

    > make_image_rgs
    Enter the file listing the event files: rgs12_event.list

  4. Make light curves to see if there are any flares that need to be removed. You will be prompted for the file that contains the names of the event files and the size of the time bins. The output will be in the "lightcurve" directory. In the "proc" directory,

    > make_lc_rgs
    Enter the file listing the event files: rgs12_event.list
    Please enter the size of the time bins in seconds for the light curves.
    100

    The output will have the naming convention "instrument_ltcrv_binsize.fits".

  5. If necessary, filter the event files to remove flares. This is done by making a text file that lists the starting and ending good or bad times, one per line, with a + or - to indicate what to keep or remove. You will also be asked what kind of spectrum binning you want. This is important if you want to combine spectra in Step 6. Wavelength binning (lambda) is the default, and is necessary if you want to combine spectra of the same order from the two RGSes; however, "beta" is also an option, and will be needed if you are interested in the 1st and 2nd order spectra from one RGS.

    For our example case, there are two time ranges contaminated with flares:

    > more gti.txt
    9.64046e7 9.64133e7 -
    9.64222e7 9.64259e7 -

    We will also want to combine the spectra from the same orders in Step 6, so we will use wavelength binning.

    In the "proc" directory,

    > make_filteredevents_rgs
    Enter the name of the file with the times to be kept or removed: gti.txt
    What spectrum binning should I use?
    1 = lambda (default; use this if you want to combine the same orders from different RGSes.)
    2 = beta
    > 1

    The output will be placed in the working directory and will overwrite the unfiltered files.

  6. Use make_spectrum_rgs to make the response files (RMFs; the ancillary files (ARFs) are not needed) and combine spectra, if desired. You will be prompted for the event file list. If there is more than one event file, you will be asked if they should be combined. The output will be placed in the directory "spectrum". In the "proc" directory,

    > make_spectrum_rgs
    Enter the file listing the event files: rgs12_event.list
    Do you want to combine the same orders from the two RGSes? (y/n)
    y

    The output files will have for format "instrument_order_type.fits", where "type" is the background spectrum (bgspec), the total spectrum (srspec), or the response (rmf).

    At this point, further analysis can be done with XSpec or CIAO, as shown in the ABC Guide.


    Summary of Commands with RGS

    You can reduce RGS data and extract useful products with minimal input by using these commands and entering the appropriate text files when prompted.

    > cd ODF
    > reprocess_rgs
    > cd ../proc
    > make_image_rgs
    > make_lightcurve_rgs
    > make_filteredevents_rgs
    > make_spectrum_rgs


    If you have any questions concerning
    XMM-Newton send e-mail to xmmhelp@lists.nasa.gov

    This file was last modified on Tuesday, 19-Nov-2013 17:23:38 EST
    Curator:Michael Arida (ADNET); michael.arida@nasa.gov

NASA Astrophysics

  • FAQ/Comments/Feedback
  • Education Resources
  • Download Adobe Acrobat
  • A service of the Astrophysics Science Division (ASD) at NASA/ GSFC

    XMM-Newton Project Scientist: Dr. Steve Snowden

    Responsible NASA Official: Phil Newman

    Privacy Policy and Important Notices.