MPU Autoscreening Bug (HEASoft 6.32)
This page discusses a bug in HEASoft 6.32 (NICERDAS version 11), which relates to autoscreening of data. It impacts all analysis of NICER data with HEASoft 6.32 where an entire MPU is autoscreened. The impact is incorrect exposure accounting and event screening, which leads to normalization errors. Downloading of the HEASoft 6.32.1 patch, and reprocessing with nicerl2, is required to obtain correct results.
Read this thread if you want to: Understand the MPU autoscreening bug
Last update: 2023-08-23
This report describes a bug in NICER standard analysis software NICERDAS version 11, which is distributed with HEASoft 6.32.
Overall, the impact of the bug is incorrect exposure accounting which leads to incorrect spectral normalization.
This report contains:
Overview of the Bug
The bug only affects nifpmsel, as distributed in NICERDAS 11 / HEASoft 6.32. The bug is not present in earlier or later releases.
The nature of the bug is that the per-detector screening is performed incorrectly when "autoscreening" deselects an entire MPU's worth of detectors. Recall that NICER is composed of 56 detectors, and banks of 8 detectors are attached to a Measurement Power Unit (MPU), which receives and packages the data for downlink to the ground. Autoscreening examines data from each MPU and looks for certain conditions (such as "shredded GTI") which degrades the quality of data from that MPU. In that case, the autoscreening system is meant to "deselect" a bank of 8 detectors during the affected times, which means BOTH the events must be screened out of the cleaned event file, AND the per-detector exposure information must be recalculated. For whole-MPU deselections, nifpmsel in HEASoft 6.32 does these actions incorrectly.
NICER supports complex combinations of detector enablements. There are 56 detectors, 7 MPUs, and long exposures of targets consisting of many pointings. Detectors, or subsets of detectors, can be enabled / disabled during different time ranges, and autoscreening can further deselect subsets of detectors during certain time ranges. Handling all possible cases is a complex process, and per-MPU autoscreening was a testing case that was missed.
Impacts of the Bug
As described above, autoscreening for whole-MPU conditions is being handled incorrectly. The result is that events from deselected MPUs may survived in the cleaned event file, and also that the FPM Selection Information is calculated incorrectly.
The impact of this is that overall normalization will be incorrect. Events files may have "extra" events, compared to the expected number of enabled detectors, and these extra events will bias the counts accumulated in a spectrum. In addition, the per-detector exposure is incorrect, which will lead to incorrect response calculations.
Thus, the primary impact of this bug is that the normalization of spectra and light curves are incorrect, for data sets affected by the bug.
In addition, the fact that per-MPU screening was intended, but did not occur, means that the data produced by the buggy tool will very likely be more noisy or have other artifacts.
Per-MPU screening is more likely after the light leak that developed in May, 2023. The light leak environment increases the chances of high undershoots, high noise rates, and high count rates overall, which increases the chances of triggering the per-MPU screening criteria performed by niautoscreen. Thus, the impacts of this bug are more likely to occur after May, 2023. However, high count rates have occurs during other parts of the NICER mission as well, and per-MPU autoscreening is not rare. Therefore, this bug can impact data observed at any time.
For this reason, users should expect that data processed with HEASoft 6.32 (NICERDAS 11), observed at any date, is highly likely to incorrectly screened. Users will have to apply the remedies as described below.
Checking Your Data for Buggy Output
You can check your cleaned event file for the presence of buggy products.
The nifpmsel task, version 1.8, is affected by the bug. Earlier and later versions are not affected.
The version number is embedded in the HISTORY keyword of the cleaned
event file. You can use the 'ftlist' command to print HISTORY
keywords and search for the version number. For example, if your
cleaned event file is named ni1234567890_0mpu7_cl.evt you
can check with this command.
If the V.V version number is 1.8, then your data are affected by the bug and you shoudl reprocess. If the version number is 1.7 or lower, or 1.9 and higher, or if no history appears, your data are not affected by this bug.
Starting with the bug-fixed version of nifpmsel, the task writes a
keyword named NIMPUFIX which is set to boolean 'T'rue. You
can print this keyword with the following command,
Remedying the Bug
The bug is inherent to nifpmsel and cannot be avoided or worked around.
The only solution to this bug is to upgrade to HEASoft 6.32.1 (NICERDAS 11a).
NICERDAS 11a / HEASoft 6.32.1 is available from the HEASoft download page. For users with an existing HEASoft 6.32 installation, there is a patch available that can be installed with minimal work. You are also able to download and install the complete distribution from scratch, if you wish.
To fix your data, you will need to run nicerl2 again. You may simply re-run nicerl2 with the same command line options as you previously used.
You can save some processing time if you wish, by avoiding some redundant tasks that nicerl2 has already performed. The bug fix does not affect the calibration or generation of the filter file, and once you have performed these tasks once using HEASoft 6.32 (and higher), you don't need to redo them again. To gain the benefit of the bug fix, you only need to performing the screening step again.
To save execution time, you can tell nicerl2 to just perform
the screening step, using the tasks=SCREEN option. For example,
Then you can proceed by making products, such as light curves and spectra, as usual, with no other change to the process.
We acknowledge the assistance of Yuhan Yao in reporting the bug, and for performing rapid-turnaround testing of the proposed bug fixes.