HEASARC Staff Scientist Position - Applications are now being accepted for a Staff Scientist with significant experience and interest in the technical aspects of astrophysics research, to work in the High Energy Astrophysics Science Archive Research Center (HEASARC) at NASA Goddard Space Flight Center (GSFC) in Greenbelt, MD. Refer to the AAS Job register for full details.

Search:

[Advanced Search]



MPU Autoscreening Bug (HEASoft 6.32)

Overview

This page discusses a bug in HEASoft 6.32 (NICERDAS version 11), which relates to autoscreening of data. It impacts all analysis of NICER data with HEASoft 6.32 where an entire MPU is autoscreened. The impact is incorrect exposure accounting and event screening, which leads to normalization errors. Downloading of the HEASoft 6.32.1 patch, and reprocessing with nicerl2, is required to obtain correct results.

Read this thread if you want to: Understand the MPU autoscreening bug

Last update: 2023-08-23

Introduction

This report describes a bug in NICER standard analysis software NICERDAS version 11, which is distributed with HEASoft 6.32.

Overall, the impact of the bug is incorrect exposure accounting which leads to incorrect spectral normalization.

This report contains:

  • Overview of the bug
  • Impacts of the bug
  • Resolutions

Overview of the Bug

The bug is in the task nifpmsel, which is part of NICER's standard software, called NICERDAS. NICERDAS is distributed with the multi-mission HEASoft platform.

The bug only affects nifpmsel, as distributed in NICERDAS 11 / HEASoft 6.32. The bug is not present in earlier or later releases.

The nature of the bug is that the per-detector screening is performed incorrectly when "autoscreening" deselects an entire MPU's worth of detectors. Recall that NICER is composed of 56 detectors, and banks of 8 detectors are attached to a Measurement Power Unit (MPU), which receives and packages the data for downlink to the ground. Autoscreening examines data from each MPU and looks for certain conditions (such as "shredded GTI") which degrades the quality of data from that MPU. In that case, the autoscreening system is meant to "deselect" a bank of 8 detectors during the affected times, which means BOTH the events must be screened out of the cleaned event file, AND the per-detector exposure information must be recalculated. For whole-MPU deselections, nifpmsel in HEASoft 6.32 does these actions incorrectly.

NICER supports complex combinations of detector enablements. There are 56 detectors, 7 MPUs, and long exposures of targets consisting of many pointings. Detectors, or subsets of detectors, can be enabled / disabled during different time ranges, and autoscreening can further deselect subsets of detectors during certain time ranges. Handling all possible cases is a complex process, and per-MPU autoscreening was a testing case that was missed.

Impacts of the Bug

As described above, autoscreening for whole-MPU conditions is being handled incorrectly. The result is that events from deselected MPUs may survived in the cleaned event file, and also that the FPM Selection Information is calculated incorrectly.

The impact of this is that overall normalization will be incorrect. Events files may have "extra" events, compared to the expected number of enabled detectors, and these extra events will bias the counts accumulated in a spectrum. In addition, the per-detector exposure is incorrect, which will lead to incorrect response calculations.

Thus, the primary impact of this bug is that the normalization of spectra and light curves are incorrect, for data sets affected by the bug.

In addition, the fact that per-MPU screening was intended, but did not occur, means that the data produced by the buggy tool will very likely be more noisy or have other artifacts.

Per-MPU screening is more likely after the light leak that developed in May, 2023. The light leak environment increases the chances of high undershoots, high noise rates, and high count rates overall, which increases the chances of triggering the per-MPU screening criteria performed by niautoscreen. Thus, the impacts of this bug are more likely to occur after May, 2023. However, high count rates have occurs during other parts of the NICER mission as well, and per-MPU autoscreening is not rare. Therefore, this bug can impact data observed at any time.

For this reason, users should expect that data processed with HEASoft 6.32 (NICERDAS 11), observed at any date, is highly likely to incorrectly screened. Users will have to apply the remedies as described below.

Checking Your Data for Buggy Output

You can check your cleaned event file for the presence of buggy products.

The nifpmsel task, version 1.8, is affected by the bug. Earlier and later versions are not affected.

The version number is embedded in the HISTORY keyword of the cleaned event file. You can use the 'ftlist' command to print HISTORY keywords and search for the version number. For example, if your cleaned event file is named ni1234567890_0mpu7_cl.evt you can check with this command. ftlist ni1234567890_0mpu7_cl.evt+1 K | grep nifpmsel HISTORY keywords will appear like this, HISTORY START PARAMETER list for nifpmsel_V.V at YYYY-MM-DDThh:mm:ss HISTORY END PARAMETER list for nifpmsel_V.V where V.V is the version number, and YYYY-MM-DDThh:mm:ss is the date that the task was run.

If the V.V version number is 1.8, then your data are affected by the bug and you shoudl reprocess. If the version number is 1.7 or lower, or 1.9 and higher, or if no history appears, your data are not affected by this bug.

Starting with the bug-fixed version of nifpmsel, the task writes a keyword named NIMPUFIX which is set to boolean 'T'rue. You can print this keyword with the following command, ftlist ni1234567890_0mpu7_cl.evt K | grep NIPMUFIX If the following output appears, NIMPUFIX= T / Fix for NICER MPU autoscreening bug in place then you know that your data has been processed with an updated version of nifpmsel. However, the lack of this keyword is not an indicator of the presence of the bug, since older versions, unaffected by the bug, do not write this keyword. The HISTORY approach documented above is a better indicator.

Remedying the Bug

The bug is inherent to nifpmsel and cannot be avoided or worked around.

The only solution to this bug is to upgrade to HEASoft 6.32.1 (NICERDAS 11a).

NICERDAS 11a / HEASoft 6.32.1 is available from the HEASoft download page. For users with an existing HEASoft 6.32 installation, there is a patch available that can be installed with minimal work. You are also able to download and install the complete distribution from scratch, if you wish.

To fix your data, you will need to run nicerl2 again. You may simply re-run nicerl2 with the same command line options as you previously used.

You can save some processing time if you wish, by avoiding some redundant tasks that nicerl2 has already performed. The bug fix does not affect the calibration or generation of the filter file, and once you have performed these tasks once using HEASoft 6.32 (and higher), you don't need to redo them again. To gain the benefit of the bug fix, you only need to performing the screening step again.

To save execution time, you can tell nicerl2 to just perform the screening step, using the tasks=SCREEN option. For example, nicerl2 ... tasks=SCREEN clobber=YES where "..." would correspond to the command line parameters you previously used.

Then you can proceed by making products, such as light curves and spectra, as usual, with no other change to the process.

Acknowledgements

We acknowledge the assistance of Yuhan Yao in reporting the bug, and for performing rapid-turnaround testing of the proposed bug fixes.

Modifications

  • 2023-08-23 - initial draft