XMM-Newton Users Handbook

A..1 Pipeline processing of XMM-Newton science data

Up to March 1st 2012, the SSC was responsible for systematically processing all XMM-Newton observations to produce data products for the three scientific instruments. Since then, processing responsibilities have been transferred to the XMM-Newton SOC. This processing is done with a pipeline that uses the same SAS software packages that are available for users to interactively analyse XMM-Newton data. However, the pipeline infrastructure used to carry out the processing has been developed by, and has been specific to, the SSC. The SSC was responsible for the pipeline software used until July 2013 when the SOC took over full control, following the transfer of the pipeline used to generate the 3XMM-DR4 catalogue from the SSC to the SOC.

Each dataset processed is subject to a screening procedure by the SSC and SOC teams which aims to verify the quality of the processing and identify any processing problems.

The aim of the pipeline processing is to provide a set of data products which are of immediate value for the XMM-Newton observer as well as for the XMM-Newton Science Archive where they are also stored for eventual public release. The pipeline products, together with the ODF (see § 6.1.1), are distributed to observers by the XMM-Newton SOC via the XMM-Newton Science Archive.

The XMM-Newton data products include calibrated, “cleaned” event lists for all X-ray cameras, which are intended to provide the starting point for most interactive analysis of the data. A number of higher-level products are also provided, such as sky images, source lists, cross-correlations with archival catalogues (performed by the Astronomical Catalogues Database Subsystem (ACDS) at the Observatoire Astronomique de Strasbourg), and spectra and time series of sufficiently bright individual sources. These provide a useful overview of the observation for the XMM-Newton observer as well as constituting key archival resources.

Since late 2010 XMM-Newton slew data have also been pipeline processed to create images, event lists and exposure maps along the slew path.

An outline of the pipeline processing system, information about the pipeline configurations and access to the pipeline software modules, are available at
http://www.cosmos.esa.int/web/xmm-newton/pipeline.
The data products produced by the pipeline are detailed in documents available on the SOC Manuals web page
(http://www.cosmos.esa.int/web/xmm-newton/documentation).

Up until early 2013 the SSC had the responsibility to perform bulk reprocessings of all XMM-Newton data from time to time. Bulk reprocessing enables the most up-to-date processing and calibrations to be applied to data and re-imposes uniformity on the archive. The latest bulk reprocessing of all XMM-Newton data took place along 2019, using a version of the SAS very similar to the public SAS v18.0 and the latest instrument calibrations at that time. This configuration was used for the release of the 4XMM-DR9 catalogue on December 2019 together with the stacked catalogue 4MM-DR9s. From that date, both pipeline processing and SAS codes have been updated regularly with no implications on the analyis results. Hence, the XMM-Newton EPIC Serendipity Sources catalogue has been updated from 4XMM-DR9/4XMM-DR9s to 4XMM-DR12/4XMM-DR12s without the need of a bulk reprocessing.