XMM-Newton Analysis with Hera

XMM-NEWTON ANALYSIS WITH HERA

Using Hera runtask

The Hera facility at the HEASARC has been adapted to run tasks from the SAS environment. This allows a user to have access to XMM-Newton data without the need for either SAS or the XMM-Newton CCF to be installed on a Users home machine. Hera can access data stored directly on the Hera servers, interfaced with a graphical interface. Alternatively, you can have all your data products stored locally on your machine, and use a command line interface, identical to actually running the SAS tasks. This is called runtask Hera which is what we use for this example script.

The actual client/server connection to Hera is handled through the fv Fits Viewer facility. When you download Hera you should also have a copy of fv as well. It is helpful to define an alias

alias herart 'fv -r'

for the invocation of the 'run task' command line interface to Hera. This is particularly true for Mac OS X users who installed the double clickable applications for Hera and fv, (e.g. in your Applications folder), where the definition should be something like

alias herart  '/Applications/fv.app/Contents/Resources/fv5.1/fv -r'

For now we will use the fv -r command.

What follows is a short script demonstrating the use of the Hera runtask interface to run SAS tasks to filter an EPIC spectrum for a point source and create the RMF and ARF files required for spectral analysis. In this example, we use the Hera server for SAS and non-SAS data processing, including two ftools tasks, Xselect and grppha. Since we start with the event lists, we omit the processing of the original data files (ODF) with, e.g., emchain.

Step 1 - Obtain a calibrated photon event file

This can be done in several different ways.
A) You can download the existing event files from the pipeline products in the archive, either from the XSA or from the GSFC archive. However, these event files may not be the products of the most recent SAS version or CCF.
B) Using the XSA reprocessing feature, you can download the event files processed with the most recent version of the SAS and CCF. Full XSA documentation can be found here, but a quick set of instructions to reprocess an observation's event lists are here.

Step 2 - Clean the data set using espfilt

Move to the pipeline products directory, and look for the raw event lists. Then select one, in this case the M1 file, and invoke the espfilt task exactly as if you were running the it locally, but proceeded by the "fv -r" invocation. (There is specific information on SAS tasks generally, via the Users' Guide and individually . End to end descriptions of the overall analysis sequences can be found in the XMM GOF produced ABC Guide.

User input is indicated in blue.

[Minerva:~/Analysis/HeraTest] rashafer% cd PPS
[Minerva:~/Analysis/HeraTest/PPS] rashafer% ls *EV*
P0157560101M1S001MIEVLI0000.FTZ	P0157560101R2S005EVENLI0000.FTZ
P0157560101M2S002MIEVLI0000.FTZ	PP0157560101EEVLIS000_0.HTM
P0157560101PNS003PIEVLI0000.FTZ	PP0157560101REVLIS000_0.HTM
P0157560101R1S004EVENLI0000.FTZ
[Minerva:~/Analysis/HeraTest/PPS] rashafer% fv -r espfilt eventset="P0157560101M1S001MIEVLI0000.FTZ"
uploading data file(s): P0157560101M1S001MIEVLI0000.FTZ
execute command: espfilt

downloading result file(s): P0157560101M1S001-objimg.FIT
downloading result file(s): P0157560101M1S001-objlc.FIT
downloading result file(s): P0157560101M1S001-corlc.FIT
downloading result file(s): P0157560101M1S001-hist.qdp
downloading result file(s): P0157560101M1S001-gti.txt
downloading result file(s): P0157560101M1S001-gti.FIT
downloading result file(s): P0157560101M1S001-objevlifilt.FIT
downloading result file(s): P0157560101M1S001-objimgfilt.FIT
downloading result file(s): P0157560101M1S001-corevlifilt.FIT
downloading result file(s): P0157560101M1S001-corimgfilt.FIT

This shows the basic cycle of a Hera runtask invocation. Hera identifies the files needed and uploads them to a Hera server at GSFC, where the requested command is executed using the most current version of the SAS software and calibration files. Any informational messages are presented (in this case none), and then Hera downloads the results file(s) produced back to our local machine. It is the "-objevlifilt.FIT" file that we will further process.

Step 3 - Define source and background regions using Xselect

If you have the FTOOLS installed locally you can go ahead and use your local Xselect to work on the downloaded event list file produced by espfilt. We continue here with an example that uses the Hera machine for Xselect.

A) Invoke Xselect, read in the data file (with Hera uploading the reqeusted file), extract an image (the "xybin=20" keyword will create 1 arc second pixels) and then plot it. Some of the informational messages from Xselect have been omitted. We also are omitting other data quality checks and selections that Xselect allows.

[Minerva:~/Analysis/HeraTest/PPS] rashafer% fv -r xselect
execute command: xselect
**  XSELECT V2.4   **
 
> Enter session name >[xsel] 
xmm
xmm:SUZAKU > read events
> Enter the Event file dir >[./]
> Enter Event file list >[]
P0157560101M1S001-objevlifilt.FIT
uploading file: P0157560101M1S001-objevlifilt.FIT
Got new mission: XMM
> Reset the mission ? >[yes]
OBJECT      OBS_ID      DATE-OBS    DATAMODE    FILTER
      1 NGC 4051    0157560101  2002-11-22T IMAGING     Medium
  Notes: XSELECT set up for      XMM
 Time keyword is TIME       in units of s
 Default timing binsize =   16.000
...
 Number of files read in:  1
 
******************** Observation Catalogue ********************
 
Data Directory is: /heradata/users/runtask.1/data/
HK Directory is: /heradata/users/runtask.1/data/
 
xmm:XMM-EMOS1-IMAGING > extract image xybin=20
extractor v4.79     13 Jul 2007
 Getting FITS WCS Keywords
 Doing file: /heradata/users/runtask.1/data/P0157560101M1S001-objevlifilt.FIT
 10% completed
...
 Image            has   142636 counts for  3.376     counts/sec
xmm:XMM-EMOS1-IMAGING > plot image

B) At this point Xselect will self-generate a command like

xmm:XMM-EMOS1-IMAGING > POW_DISPLAY 80b7103b:56864

which hands off the image to POW to display the actual image. This might take up to a minute. (When using Xselect locally you might be using SAOImage DS9 instead of POW for image display.) Using this image we will define a source and background region.

1) Under the Tools Menu on the POW 2.0 window menu bar, select Region files..., which will open up an Edit Region window.
2) Go back to the POW image window by clicking on it, then click on the source and drag outward to make a large circle to define the outer boundary of the background region. (If you want to zoom in on the source region before hand, you can do this with a right-click and drag).
3) You can adjust the region radius by clicking on the perimeter, and then dragging the handle. You can move the region by click and dragging in the center of the region. Note that in the Edit Region window, the top text window, labled Regions for xmm_image.xsl_img, has the region definition, a line that starts +Circle(... and then gives the region center (RA and declination) and the region radius. Below this is a Current Shape: subwindow that allows you to manually modify the region definition (you need to push the Apply button after making any changes).
4) Create a second region by clicking outside the background radius and dragging a small circle. Move this inside the background region, center on the source, and adjust the radius.
5) Note that the second smaller region is highlighted in the Regions for xmm_image.xsl_img list on the Edit Region window. In the Current Shape: subwindow, click the + button, to change it to minus, and click the Apply button. This means that events in the smaller region will not be included.
6) Click Save... at the bottom of the Edit Region window. You will be prompted for a name for the region file, enter background.reg. Click the (misnamed) Upload button. Close the informational box that says that background.reg has been saved to Hera.
7) Change the - back to +, and hit Apply. Then select the first (larger) region definition in the list above, followed by the Delete button. Save... again, naming the file source.reg.
8) Exit Pow (under the File menu on the POW 2.0 window).

C) Return to Xselect and continue to filter and create the source and background files.

xmm:XMM-EMOS1-IMAGING > filter region source.reg
xmm:XMM-EMOS1-IMAGING > extract spectrum
extractor v4.83      7 Mar 2008
 Getting FITS WCS Keywords
 Doing file: /Volumes/forum/rashafer/Analysis/HeraTest/PPS/P0157560101M1S001-objevlifilt.FIT
100% completed
	  Total      Good    Bad: Region      Time     Phase     Grade     Cut
	 142636     20883         121753         0         0         0         0
===============================================================================
   Grand Total      Good   Bad:  Region      Time     Phase     Grade    Cut
	142636     20883         121753         0         0         0         0
   in  42244.     seconds
 Spectrum         has    20883 counts for 0.4943     counts/sec
 ... written the PHA data Extension
xmm:XMM-EMOS1-IMAGING > save spectrum
> Give output file name >[] source.pi
Truncated the spectrum to        0 -    11999
The data will be rebinned to   800 channels
using compression mode LINEAR
> Group ( or rebin ) the spectra before outputting? >[yes] 
Wrote spectrum to source.pi
xmm:XMM-EMOS1-IMAGING > clear region
xmm:XMM-EMOS1-IMAGING > filter region background.reg
xmm:XMM-EMOS1-IMAGING > extract spectrum
extractor v4.83      7 Mar 2008
 Getting FITS WCS Keywords
 Doing file: /Volumes/forum/rashafer/Analysis/HeraTest/PPS/P0157560101M1S001-objevlifilt.FIT
100% completed
	  Total      Good    Bad: Region      Time     Phase     Grade     Cut
	 142636     43524          99112         0         0         0         0
===============================================================================
   Grand Total      Good   Bad:  Region      Time     Phase     Grade    Cut
	142636     43524          99112         0         0         0         0
   in  42244.     seconds
 Spectrum         has    43524 counts for  1.030     counts/sec
 ... written the PHA data Extension
xmm:XMM-EMOS1-IMAGING > save spectrum
> Give output file name >[source] background.pi
Truncated the spectrum to        0 -    11999
The data will be rebinned to   800 channels
using compression mode LINEAR
> Group ( or rebin ) the spectra before outputting? >[yes] 
Wrote spectrum to background.pi

D) Exit from Xselect, and the two .reg and .pi files are downloaded, among others. Important Note: If you wish to apply exactly these region definitions to other event files, do so before you exit. At present, the runtask Hera form of Xselect does not properly uplink the .reg files. To continue, enter clear events followed by a read events for each file and the filter region, extract spectrum and save spectrum commands for both the source and background. Be sure to give unique names to the .pi files for each input eventlist.

xmm:XMM-EMOS1-IMAGING > exit
> Save this session? >[no]
downloading result file(s): xselect.log
downloading result file(s): source.reg
downloading result file(s): background.reg
downloading result file(s): source.pi
downloading result file(s): background.pi

Step 4 - Create the rmf file with rmfgen

The same cycle of task specification, needed files uploaded, result files downloaded:

[minerva:~/Analysis/HeraTest/PPS] rashafer% fv -r rmfgen rmfset=source.rmf \
                               spectrumset=source.pi
uploading data file(s): source.pi
execute command: rmfgen
rmfgen:- CCF constituents accessed by the calibration server:
CifEntry{EMOS1, FILTERTRANSX, 12, /herasoft/sas/CCF/EMOS1_FILTERTRANSX_0012.CCF, 1998-01-01T00:00:00.000}
...
downloading result file(s): source.rmf

You see a list of all the calibration data used, which for Hera is the most current set available.

Creating the RMF file can take a few minutes to complete. If you wish, you can have the runtask invocation not wait on completion, and send you a notifying email with the -nowait option.

[minerva:~/Analysis/HeraTest/PPS] rashafer% fv -r -nowait email@address \
                                        rmfgen rmfset=source.rmf  spectrumset=source.pi
uploading data file(s): source.pi
execute command: rmfgen

where email@address represents an address where Hera will send a notification and download instructions when the task completes.

Step 5 - Create the arf file with arfgen

And again...

[minerva:~/Analysis/HeraTest/PPS] rashafer% fv -r arfgen arfset=source.arf spectrumset=source.pi  \
                withrmfset=yes rmfset=source.rmf withbadpixcorr=yes \
                badpixlocation="P0157560101M1S001-objevlifilt.FIT" useodfatt=no
uploading data file(s): source.pi
uploading data file(s): source.rmf
uploading data file(s): P0157560101M1S001-objevlifilt.FIT
execute command: arfgen
arfgen:- CCF constituents accessed by the calibration server:
CifEntry{EMOS1, FILTERTRANSX, 12, /herasoft/sas/CCF/EMOS1_FILTERTRANSX_0012.CCF, 1998-01-01T00:00:00.000}
...
** arfgen::attcalc: warning (outOfImageRange),  148056  events ( of  4410000  ) out of range of image

downloading result file(s): source.arf

Notice that you use the filtered event list file produced by evselect to designate the badpixlocation values.

Step 6 - Group channels (and ancillary information) using grppha

[minerva:~/Analysis/HeraTest/PPS] rashafer% fv -r grppha
execute command: grppha
Please enter PHA filename[]
source.pi
uploading file: source.pi
Please enter output filename[]
source-grp.pi
 
  -------------------------
  MANDATORY KEYWORDS/VALUES
...
--------------------------------------------------------------------
GRPPHA[]
chkey BACKFILE background.pi
GRPPHA[]
chkey RESPFILE source.rmf
GRPPHA[chkey RESPFILE source.rmf]
chkey ANCRFILE source.arf
GRPPHA[chkey ANCRFILE source.arf]
group min 9
GRPPHA[group min 9]
exit
 ... written the PHA data Extension
...... exiting, changes written to file : source-grp.pi
 ** grppha 3.0.1 completed successfully

downloading result file(s): source-grp.pi

You are now ready to roll to use Xspec or alternative spectral fitting package to analyze your data.

If you have any questions concerning XMM-Newton send e-mail to xmmhelp@lists.nasa.gov

This file was last modified on Monday, 14-Jul-2008 11:27:57 EDT
Curator:Michael Arida (SP Sys); michael.arida@nasa.gov