5. Data

5.1 Useful Documentation

The documents most likely to be useful to the Hera user are those pertaining to SAS. There are a number of these which users of XMM-Newton data should be aware of. These documents include the SSC Products Specification, Data Files Handbook, Reading Data Products CD's (the most recent versions of these documents can be found in the SOC Document section under
http://xmm.esac.esa.int/external/xmm_user_support/documentation/index.shtml), and the SAS Users Guide (http://xmm.esac.esa.int/external/xmm_user_support/documentation/sas_usg/USG/).

Additional information concerning XMM-Newton data files can be found in the Interface Control Document: Observation and Slew Data Files (XSCS to SSC) (SciSIM to SOCSIM) (XMM-SOC-ICD-0004-SSD). This is an impressive tome which goes into great detail about the file nomenclature and structure. This document can be found in the documents area of the SOC web pages:
http://xmm.esac.esa.int/.

5.2 The Data

One of the first steps that should be taken when examining your data is to check to see what you actually have. XMM-Newton observations can be broken into several exposures which are each assigned separate observation numbers. These separate exposures can be radically different in length and can also have the different instruments in different modes. For example, in one case the full observation was 60 ks with EPIC and RGS active but there was one delivered exposure which was $\sim3$ ks and had only RGS active. (This can happen because the RGS can operate farther into regions of higher radiation than the EPIC detector. The additional observation time can be considered an additional exposure with only the RGS active.) Two files are useful and should be downloaded to the user's machine for this examination. First, the primary HTML page is INDEX.HTM which is included in the Pipeline Products. This page lists basic information for the observation plus the operational modes, filters, and exposure start and stop times for the individual instruments. It also has links to various summary pages, including those for the instruments. (In the case above, the EPIC summary page simply stated that ``EPIC exposures processed by PPS None.'') Specifically, LOOK at the P*SUMMAR0000.HTM files in the pipeline products (easily available through the links). Second, to quickly access images from the various instruments examine the PPSGRA Pipeline Products page (§ 5.3.3).

5.3 PI Data

Proprietary XMM-Newton data is available for download via your XSA account. Email instructions from the XMM-SOC at Vilspa are sent to the address on record with detailed directions on how to retrieve your data via the XSA.

XMM-Newton data is available for download at the HEASARC data archive. The data files can be considered to come in two groups in separate subdirectories when retrieved, the Observation Data Files (ODF) files and Pipeline Processing (PPS) files. The ODF data contain all of the observation-specific data necessary for reprocessing the observation. The PPS data contain, among other things, calibrated photon event files and source lists.

5.3.1 ODF Data

ODF data come with file names in the following format:

• mmmm_iiiiiijjkk_aabeeeccfff.zzz

  mmmm - revolution orbit number

  iiiiii - proposal number

  jj - target ID number in proposal

  kk - exposure number for target

  (NOTE: The ten-digit combination of iiiiiijjkk is the observation number and is used repetitively throughout the file nomenclature.)

  aa - detector (M1 - MOS1, M2 - MOS2, PN - PN, OM - OM, R1 - RGS1, R2 - RGS2, SC - spacecraft)

  b - flag for scheduled (S) or unscheduled (U) observations, or X for general purpose files)

  eee - exposure number

  cc - CCD number or OM window number

  fff - data identifier for the three detectors or spacecraft itself; see Table 5.1.

  zzz - format (FIT - FITS, ASC - ASCII)

Table 5.1: ODF data file identifiers.

Data ID	Contents
	EPIC files
IME	Event list for individual CCDs, imaging mode
RIE	Event list for individual CCDs, reduced imaging mode
CTE	Event list for individual CCDs, compressed timing mode
TIE	Event list for individual CCDs, timing mode
BUE	Event list for individual CCDs, burst mode
AUX	Auxiliary file
CCX	Counting cycle report (auxiliary file)
HBH	HBR buffer size, non-periodic housekeeping
HCH	HBR configuration, non-periodic housekeeping
HTH	HBR threshold values, non-periodic housekeeping
PEH	Periodic housekeeping
PTH	Bright pixel table, non-periodic housekeeping
DLI	Discarded lines data
PAH	Additional periodic housekeeping
PMH	Main periodic housekeeping
	RGS files
AUX	on-board processing statistics
SPE	raw event list for one CCD
DII	diagnostic images
D1H	CCD readout settings
D2H	CCD readout settings
PFH	housekeeping data
ODX	pixel offset data
	XMM files
ATS	spacecraft attitude history
	OM files
IMI	imaging file
THX	tracking history file
WDX	window data auxiliary file
NPH	non-periodic housekeeping file
PEH	periodic housekeeping file
PAX	field acquisition data
RFX	priority reference frame data
PFX	priority fast mode data
FAE	event list (if fast mode was used)

5.3.2 Pipeline Product Data - Summary Files and Groupings

The Pipeline Processing (PP) produces quite a number of useful products which allow a first look at the data, but can overwhelm the user by their sheer numbers. The first place to look is the INDEX.HTM page which organizes the presentation of the data and provides links to other PP pages. The INDEX.HTM page also lists general observation information (target, date, time, etc.) and instrument modes.

The INDEX.HTM page provides links to various observation summary pages, which have names with the following nomenclature:

• PPiiiiiijjkkAAX000SUMMAR0000.HTM

  iiiiiijjkk - observation number

  AA - detector ID (EP - EPIC, OM - Optical Monitor, RG - RGS, OB - Observation)

PP data contain some immediately useful data products such as calibrated photon event lists, source lists, and images. While there are a large number of products which come in a single directory, they can be associated in up to 15 groupings; see Table 5.2. (The number of groups can vary depending on the number of operational instruments, e.g., if the OM is turned off there are no OM products.) Further information on each of these groupings and associated files, such as file contents, file types, and how they may be viewed, can be found in Table 5.3. Each group has an associated HTML file which organizes access to the files and provides a limited description of them. The names of the HTML files are of the following form:

• PPiiiiiijjkkAAAAAA000_0.HTM

  iiiiiijjkk - observation number

  AAAAAA - group identifier (see Table 5.2)

Table 5.2: Pipeline Processing groupings.

Group ID	Contents
	PP files
PPSDAT	Contains the Calibration Index File (CIF) used in the pipeline processing (*CALIND*),
	PPS information, and the attitude history time series (*ATTTSR*) in gzipped FITS
	or ASCII format.
PPSGRA	Contains the OM tracking history plots, PPS, EPIC, OM, RGS observation, and PPS
	run summaries. NOTE: CHECK THESE OUT
PPSMSG	ASCII file containing pipeline processing report
	EPIC files
CRSCOR	Contains PDF files of POSS II finding charts, HTML files of cross correlations
	with the SIMBAD data base, FITS tables for the detected sources
EANCIL	Contains the exposure maps in a variety of energy bands and the source-detection
	sensitivity maps for the EPIC instruments. The sensitivities are in units of
	counts s$^{-1}$ corrected for vignetting and corresponding to a likelihood specified
	in the FITS header. The files are gzipped with a .FTZ extension.
EEVLIS	Contains calibrated photon event files for the EPIC detectors. If the files are
	sufficiently large they may be separated into two tar files. The files are gzipped
	fits files with a .FTZ extension.
ESKYIM	This group contains the event images in a variety of energy bands. The fits files
	are gzipped with a .FTZ extension, the full images also come as PNG images.
ESRLIS	Contains EPIC observation source lists. There is an HTML page of the merged
	source list and gzipped fits tables of source lists from the different instruments
	and source detection tasks.
	OM files
OIMAGE	Contains OM sky images in gzipped FITS format.
OMSLIS	Contains OM observation source lists in gzipped FITS format.
OMSRTS	Contains OM star tracking time series in gzipped FITS format.
	RGS files
REVLIS	Contains the RGS source and event lists in gzipped FITS format
REXPIM	Contains the RGS exposure maps in gzipped FITS format
RIMAGE	Contains the RGS images (both energy dispersion and cross dispersion) in gzipped
	FITS and PNG formats
RSPECT	Contains the RGS source and background spectra in gzipped FITS and PDF formats

5.3.3 Pipeline Product Data - Data Files

The data file names are of the form (see Table 41 in the XMM Data Files Handbook):

• PiiiiiijjkkaablllCCCCCCnmmm.zzz

  iiiiiijjkk - observation number

  aa - detector, M1 - MOS1, M2 - MOS2, PN - PN, CA - for files from the CRSCOR group, R1 - RGS1, R2 - RGS2, OM - OM.

  b - flag for scheduled (S) or unscheduled (U) observations, or X for files from the CRSCOR group (and any product that is not due to a single exposure)

  lll - exposure number

  CCCCCC - file identification for data from each detector; see Table 5.3

  n - For EPIC data, this is the exposure map band number; for RGS data, this is the spectral order number; for the OM, this is the OM window within the exposure.

  mmm - source number in hexidecimal

  zzz - file type (e.g., PDF, PNG, FTZ, HTM)

  ASC - ASCII file, use a web browser, or the ``more'' command

  ASZ - gzipped ASCII file

  FTZ - gzipped FITS format, use ds9, Ximage, Xselect, fv

  HTM - HTML file, use Firefox or other web browser

  PDF - Portable Data Format, use Acrobat Reader

  PNG - Portable Networks Graphics file, use a web browser

  TAR - TAR file

Table 5.3: Pipeline Processing data files

Group ID	File ID	Contents	File Type	View With
		EPIC files
CRSCOR	FCHART	Finding chart	PDF	Acrobat Reader
	ROSIMG	ROSAT image of region	PDF	Acrobat Reader
	SNNNNN$^1$	Source cross-correlation results	Zipped FITS	fv
	DNNNNN$^1$	Catalog descriptions	PDF	Acrobat Reader
	FNNNNN$^1$	FOV cross-correlation result	Zipped FITS	fv
ESKYIM	IMAGE_8	Sky image 0.2 - 12.0 keV	Zipped FITS	ds9, Ximage, fv
	IMAGE_1	Sky image 0.2 - 0.5 keV	Zipped FITS	ds9, Ximage, fv
	IMAGE_2	Sky image 0.5 - 2.0 keV	Zipped FITS	ds9, Ximage, fv
	IMAGE_3	Sky image 2.0 - 4.5 keV	Zipped FITS	ds9, Ximage, fv
	IMAGE_4	Sky image 4.5 - 7.5 keV	Zipped FITS	ds9, Ximage, fv
	IMAGE_5	Sky image 7.5 - 12.0 keV	Zipped FITS	ds9, Ximage, fv
EANCIL	EXPMAP_8	Exposure map 0.2 - 12.0 keV	Zipped FITS, PNG	ds9, Ximage, fv, web browser
	EXPMAP_1	Exposure map 0.2 - 0.5 keV	Zipped FITS	ds9, Ximage, fv
	EXPMAP_2	Exposure map 0.5 - 2.0 keV	Zipped FITS	ds9, Ximage, fv
	EXPMAP_3	Exposure map 2.0 - 4.5 keV	Zipped FITS	ds9, Ximage, fv
	EXPMAP_4	Exposure map 4.5 - 7.5 keV	Zipped FITS	ds9, Ximage, fv
	EXPMAP_5	Exposure map 7.5 - 12.0 keV	Zipped FITS	ds9, Ximage, fv
EEVLIS$^2$	MIEVLI	MOS imaging mode event list	Zipped FITS	xmmselect, fv, Xselect
	PIEVLI	PN imaging mode event list	Zipped FITS	xmmselect, fv, Xselect
	TIEVLI	PN, MOS timing mode event list	Zipped FITS	xmmselect, fv, Xselect
ESRLIS	EBLSLI	Box-local detect source list	Zipped FITS	fv
	EBMSLI	Box-map detect source list	Zipped FITS	fv
	EMSRLI	Max-like detect source list	Zipped FITS	fv
	OBSMLI	Summary source list	Zipped FITS, HTML	fv, web browser
		RGS files
REVLIS	SRCLI_	RGS Source Lists	Zipped FITS	fv
	EVENLI	RGS Event lists	Zipped FITS	xmmselect, fv
REXPIM	EXPMAP	RGS Exposure Maps	Zipped FITS	ds9, Ximage, fv
RSPECT	SRSPEC1	1st Order Source Spectra	Zipped FITS	Xspec, fv
	SRSPEC2	2nd Order Source Spectra	Zipped FITS	Xspec, fv
	BGSPEC1	1st Order Source Spectra	Zipped FITS	Xspec, fv
	BGSPEC2	2nd Order Source Spectra	Zipped FITS	Xspec, fv
	SRSPEC	Spectra Plots	PDF format	Acrobat Reader
RIMAGE	ORDIMG	Images, disp. vs. X-disp	Zipped FITS, PNG	ds9, Ximage, fv, web browser
	IMAGE_	Images, disp. vs. PI	Zipped FITS, PNG	ds9, Ximage, fv, web browser
		OM files
OANCIL	SWSREG	OM Region File	ASCII text	text editor
OIMAGE	SIMAGE	OM Sky Image	Gzipped FITS	ds9, Ximage, fv
OMSLIS	SWSRLI	OM Source Lists	Zipped FITS	fv
OMSRTS	TSTRTS	Tracking Star Time Series	Zipped FITS	fv

$^1$ NNNNN - Alphanumeric ID
$^2$ Files for only those modes which were active will be included