skip to content
 
NASA/GSFC & ESA GSFC ESA   XMM Helpdesk
  XMM What's New
  HEASARC Site Map
  NASA Homepage

HEASARC HOME XMM-NEWTON HOME ARCHIVE DATA ANALYSIS PROPOSALS & TOOLS - BUDGET AO PROPOSALS & TOOLS - SCIENCE AO
XMM-Newton Guest Observer Facility
ABOUT XMM-NEWTON GOF SERVICES WHAT'S NEW CALIBRATION RELATED SITES GALLERY

XMM-NEWTON DATA ANALYSIS WITH PERL SCRIPTS

The GOF has developed Perl scripts in an effort to simplify the EPIC and RGS data reduction process and make XMM data more accessible to astrophysicists and students who may not be very familiar with the details of working with (locally-installed) SAS. These, plus a short guide to their use, and a summary of commands, are available below. Knowledge of SAS is not required to run the scripts. However, all scripts assume that SAS has been initialized and the intial environmental variables have been set. They also assume you are working with a point source that will be extracted in a circular region. In the walk-throughs, we assume that the scripts have been placed in a convenient directory and aliased, as shown below. All the steps given here are the same as in the ABC Guide. In the EPIC example, we use the observation of the Lockman Hole (ObsID 0123700101) and for RGS, we use AB Dor (ObsID 0134520301). As always when working with SAS, be sure to call tasks from your working directory.

Scripts
A Walk-Through with EPIC
A Summary of Commands with EPIC
A Walk-Through with RGS
A Summary of Commands with RGS

Scripts


These are the available scripts with a short description of each. Clicking on each name will take you to a page with the script, where they can easily be saved by your browser.


After grabbing them with your browser, the easiest way to use these is to place them in a convenient directory and edit your .cshrc (or .bashrc or .zcshrc) file to look for them:
> cd ~
> mkdir sas_scripts/
> mv *_epic sas_scripts/
> mv *_rgs sas_scripts/
And in your .cshrc, add the appropriate aliases, e.g.:
alias make_image_epic 'perl ${HOME}/sas_scripts/make_image_epic'
alias reprocess_rgs 'perl ${HOME}/sas_scripts/reprocess_rgs'
Remember to source your .cshrc to pick up the new aliases:
> source .cshrc

A Walk-Through with EPIC

  1. Download the data to your local machine and unzip the files in the ODF directory. Call reprocess_epic to prepare the data, then reprocess and filter them. The output event files will appear in the directory "proc" and follow the naming convention "instrument_evt2.fits". reprocess_epic will automatically update your environment variables to point to the new ccf.cif and *SUM.SAS files it makes. It will also ask you if you want to process the MOS data and/or PN data. We are interested in both, so enter 'y' at the prompts.

    > cd ODF
    > gunzip *gz
    > reprocess_epic
    > y
    > y

  2. Examine the event files and prepare some text files for future use. We're interested in data from all three detectors, so make a file listing all their event files:

    > cd ../proc
    > ls *evt2.fits > events.txt

    There are two somewhat bright sources in the field. We can make a region file with DS9, being sure to use physical coordinates, of both the sources and the background. (If using these scripts on both extended sources and point sources, be sure to keep them seperated, that is, have one region file for point sources and another for extended sources.)

    > more LH_src.reg

    # Region file format: DS9 version 4.1
    global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
    physical
    circle(26202.5,22793.5,600)
    circle(20219.586,36401.5,600)

    > more LH_bkg.reg

    # Region file format: DS9 version 4.1
    global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
    physical
    annulus(26202.5,22793.5,600,1400)
    annulus(20219.586,36401.5,600,1400)

  3. Make images with make_image_epic. You will be prompted for the text file that contains the names of the event files. By default, make_image_epic will make an image of the full field of view, but it can also make images of specific regions. You can do this by entering a region file at the prompt. The output will be in the "image" directory and will follow the naming convention "instrument_img.fits". If you use a region file, the images of each source will be numbered in the order in which they are given in the file. In the "proc" directory,

    > make_image_epic
    Enter the file listing the event files: events.txt
    Should I use a region file? If so, enter the file name; otherwise, enter N: n

  4. Make light curves to see if there are any flares that need to be removed. You will be prompted for the file that contains the names of the event files and the size of the time bins. By default, make_lc_epic will make a light curve over the entire field of view, but you can also make one (or several) for specific objects by providing a region file at the prompt. The output will be in the "lightcurve" directory. If you use a region file, the lightcurves of each object will be numbered in the order in which they are given in the file. In the "proc" directory,

    > make_lc_epic
    Enter the file listing the event files: events.txt
    Should I use a region file? If so, enter the file name; otherwise, enter N: LH_src.reg
    Please enter the size of the time bins. 10

  5. Examine the lightcurves. If necessary, filter the event files to remove flares. This is done by making a text file that lists the starting and ending BAD times, one per line. For our example case, there are several time ranges contaminated with flares:

    > more bad_times.txt
    7.32276e7 7.32405e7
    7.32118e7 7.3212e7
    7.32204e7 7.32206e7

    To remove these time ranges, in the "proc" directory,

    > make_filteredevents_epic
    Enter the file listing the event files: events.txt
    Enter the file with the times to be removed: bad_times.txt

    There will be several "DssBlockClash" warnings; these can be ignored. The output will be placed in the working directory and follow the naming convention "instrument_flt_evt2.fits". Remember to update the events file list with the new files!

  6. Use make_spectrum_epic to extract a spectrum, test it for pile-up, and produce the redistribution matrix and ancillary files. You will be prompted for the event file list and a region file. The output will be placed in the directory "spectrum". Each source in the region file will have a spectrum (*.pi), a postscript file comparing the measured and modeled count rates produced with the SAS task epatplot, and response files (*.rmf and *.arf). These products will be numbered in the order that the sources are given in the file. In the "proc" directory,

    > make_spectrum_epic
    Enter the file listing the event files: events.txt
    Enter the region file name: LH_src.reg
    Is this an extended source? y/n: n

  7. Make a background spectrum with make_bkg_spectrum_epic. You will be prompted for a background region file, like what was made in step 2. The output spectrum will be placed in the "spectrum" directory. Each spectrum will be numbered in the order that the sources are given in the region file. In the "proc" directory,

    > make_bkg_spectrum_epic
    Enter the file listing the event files: events.txt
    Enter the background region file name: LH_bkg.reg
    Is this an extended source? y/n: n
    Make rmf and arf files for the background? y/n: y

  8. Make exposure maps with make_expmap_epic. If there are particular energy ranges you are interested in, you can make a text file and list the minimum and maximum energies (in eV), one per line, and enter it at the prompt. For instance, if you have a text file "energy.txt" that looks like this:

    > more energy.txt
    1000 2000
    2000 3000

    it will produce two exposure maps for each event file, over the ranges 1-2 keV and 2-3 keV. If no energy file is entered, the default range 2-4.5 keV is used. In any case, the output is placed in the "expmap" directory and follows the naming convention "instrument_expmap_LowEnergy-HighEnergykeV.fits". In the "proc" directory,

    > make_expmap_epic
    Enter the file listing the event files: events.txt
    Should I use an energy file? If so, enter the file name; otherwise, enter N: energy.txt

    For our example, six exposure maps will be made: m1_expmap_1-2keV.fits, m1_expmap_2-3keV.fits, m2_expmap_1-2keV.fits, m2_expmap_2-3keV.fits, pn_expmap_1-2keV.fits, and pn_expmap_2-3keV.fits.

With these data products, further analysis can be done with XSpec or CIAO, as shown in the ABC Guide.

Summary of Commands with EPIC

You can reduce XMM data and extract useful products with minimal input by using these commands and entering the appropriate text files when prompted.

> cd ODF
> reprocess_epic
> cd ../proc
> make_image_epic
> make_lightcurve_epic
> make_filteredevents_epic
> make_expmap_epic
> make_spectrum_epic
> make_bkg_spectrum_epic

A Walk-Through with RGS

  1. Download the data to your local machine and unzip the files in the ODF directory. Call reprocess_rgs to prepare and reprocess the data. The output event files will appear in the directory "proc". reprocess_rgs will automatically update your environment variables to point to the new ccf.cif and *SUM.SAS files it makes. It will also query you to see if you will want the option to combine RGS1 and RGS2 first order spectra later on, or the first and second order spectra from one RGS instrument instead. This task also gives the user the option of producing images of PI vs. BETA_CORR and XDSP_CORR vs. BETA_CORR, as well as a light curve. If you opt to make lightcurves, you will be asked to provide a time bin size for the light curve, in seconds. (This task can take a long time and will produce a lot of warnings, which are safely ignored.) In the example below, we will opt for the ability to combine 1st order spectra from RGS1 and RGS2, plus make images and lightcurves with time bins of 50s.

    > cd ODF
    > gunzip *gz
    > reprocess_rgs
    In the ODF directory, continuing. You can process the data in one of these ways: 1) with the option to combine 1st order spectra from RGS1 and RGS2 later on, or 2) with the option to combine 1st and 2nd order spectra from one RGS instrument later on. Which would you like? (1/2) 1 Ok, setting the appropriate parameter value. Would you like to make light curves? Y/N: y Please enter the size of the time bins, in seconds, for the light curves. 50 Would you like to make images of PI, XDSP_CORR, and BETA_CORR? Y/N: y

  2. Examine the event files and prepare some text files for future use. We're interested in data from both RGS detectors, so make a file listing all their event files:

    > cd ../proc
    > ls *EVENLI* > rgs12_event.list

  3. As we saw,reprocess_rgs gives the user the option of making images and light curves. You can also get these data products with the tasks make_image_rgs and make_lc_rgs. To make "banana plot" images with make_image_rgs, enter the name of the text file that contains the names of the event files at the prompt. The output will be in the "image" directory and will follow the naming convention "instrument_pi_bc.fits" and "instrument_xd_bc.fits" for the PI vs. Beta Correction and Cross Dispersion vs. Beta Correction plots, respectively. In the "proc" directory,

    > make_image_rgs
    Enter the file listing the event files: rgs12_event.list

  4. In a similar way, if you decide you need to make another light curve, call make_lc_rgs. You will be prompted for the file that contains the names of the event files and the size of the time bins. The output will be in the "lightcurve" directory. In the "proc" directory,

    > make_lc_rgs
    Enter the file listing the event files: rgs12_event.list
    Please enter the size of the time bins in seconds for the light curves.
    100

    The output will have the naming convention "instrument_ltcrv_binsize.fits".

  5. If necessary, filter the event files to remove flares. This is done by making a text file that lists the starting and ending good or bad times, one per line, with a + or - to indicate what to keep or remove. You will also be asked what kind of spectrum binning you want. This is important if you want to combine spectra in Step 6. Wavelength binning (lambda) is the default, and is necessary if you want to combine spectra of the same order from the two RGSes; however, "beta" is also an option, and will be needed if you are interested in the 1st and 2nd order spectra from one RGS.

    For our example case, there are two time ranges contaminated with flares:

    > more gti.txt
    9.64046e7 9.64133e7 -
    9.64222e7 9.64259e7 -

    We will also want to combine the spectra from the same orders in Step 6, so we will use wavelength binning.

    In the "proc" directory,

    > make_filteredevents_rgs
    Enter the name of the file with the times to be kept or removed: gti.txt
    What spectrum binning should I use?
    1 = lambda (default; use this if you want to combine the same orders from different RGSes.)
    2 = beta
    > 1

    The output will be placed in the working directory and will overwrite the unfiltered files.

  6. Use make_spectrum_rgs to make the response files (RMFs; the ancillary files (ARFs) are not needed) and combine spectra, if desired. You will be prompted for the event file list. If there is more than one event file, you will be asked if they should be combined. The output will be placed in the directory "spectrum". In the "proc" directory,

    > make_spectrum_rgs
    Enter the file listing the event files: rgs12_event.list
    Do you want to combine the same orders from the two RGSes? (y/n)
    y

    The output files will have for format "instrument_order_type.fits", where "type" is the background spectrum (bgspec), the total spectrum (srspec), or the response (rmf).

    At this point, further analysis can be done with XSpec or CIAO, as shown in the ABC Guide.

    Summary of Commands with RGS

    You can reduce RGS data and extract useful products with minimal input by using these commands and entering the appropriate text files when prompted.

    > cd ODF
    > reprocess_rgs
    > cd ../proc
    > make_image_rgs
    > make_lightcurve_rgs
    > make_filteredevents_rgs
    > make_spectrum_rgs

    If you have any questions concerning     XMM-Newton send e-mail to xmmhelp@lists.nasa.gov

    This file was last modified on Monday, 04-Oct-2021 18:22:35 EDT
    Curator:Michael Arida (ADNET); michael.arida@nasa.gov

NASA Astrophysics

  • FAQ/Comments/Feedback
  • Education Resources
  • Download Adobe Acrobat
    		• A service of the Astrophysics Science Division (ASD) at NASA/ GSFC

    XMM-Newton Project Scientist: Dr. Kim Weaver

    Responsible NASA Official: Phil Newman

    Privacy Policy and Important Notices.