Up to March 1st 2012, the SSC was responsible for systematically processing all XMM-Newton observations to produce data products for the three scientific instruments. Since then, processing responsibilities have been transferred to the XMM-Newton SOC. This processing is done with a pipeline that uses the same SAS software packages that are available for users to interactively analyse XMM-Newton data. However, the pipeline infrastructure used to carry out the processing has been developed by, and has been specific to, the SSC. The SSC was responsible for the pipeline software used until July 2013 when the SOC took over full control, following the transfer of the pipeline used to generate the 3XMM-DR4 catalogue from the SSC to the SOC.
Each dataset processed is subject to a screening procedure by the SSC and SOC teams which aims to verify the quality of the processing and identify any processing problems.
The aim of the pipeline processing is to provide a set of data products which are of immediate value for the XMM-Newton observer as well as for the XMM-Newton Science Archive where they are also stored for eventual public release. The pipeline products, together with the ODF (see § 6.1.1), are distributed to observers by the XMM-Newton SOC via the XMM-Newton Science Archive.
The XMM-Newton data products include calibrated, ``cleaned'' event lists for all X-ray cameras, which are intended to provide the starting point for most interactive analysis of the data. A number of higher-level products are also provided, such as sky images, source lists, cross-correlations with archival catalogues (performed by the Astronomical Catalogues Database Subsystem (ACDS) at the Observatoire Astronomique de Strasbourg), and spectra and time series of sufficiently bright individual sources. These provide a useful overview of the observation for the XMM-Newton observer as well as constituting key archival resources.
Since late 2010 XMM-Newton slew data have also been pipeline processed to create images, event lists and exposure maps along the slew path.
An outline of the pipeline processing system, information about the pipeline
configurations and access to the pipeline software modules, are available at
The data products produced by the pipeline are detailed in documents available on the SOC Manuals web page
The SSC also has had a responsibility to perform bulk reprocessings of all XMM-Newton data from time to time. Bulk reprocessing enables the most up-to-date processing and calibrations to be applied to data and re-imposes uniformity on the archive. In this context, almost all existing XMM-Newton observations prior to May 2007 were reprocessed by the SSC in 2006/2007 using a version of the SAS very similar to the public SAS v7.0 and the latest instrument calibrations at that time. The 2XMM source catalogue (see § A.3) was constructed from selected observations from that reprocessing. The SSC performed another bulk reprocessing (based on SAS 12) during December 2012, from which the latest major catalogue release, 3XMM-DR4, has been derived. The products from this new bulk reprocessing and the associated 3XMM-DR4 catalogue were released in July 2013 (see § A.3.1).