The niautoscreen task performs standard per-FPM and per-MPU screening for a NICER observation. This task catches issues which may occur for a single FPM, or a single MPU, that the overall screening done by nimaketime may miss.
This task checks for "out-of-family" behavior of individual FPMs and MPUs, which may degrade NICER instrument performance if they are included in the data. Here, "out-of-family" means that a particular FPM or MPU differs significantly from the group as a whole, using robust statistical measures.
This task considers individual NICER FPMs and MPUs. The goal is to only exclude "bad" data for the individual detectors and individual time ranges where the bad behavior is occuring. Since NICER has 52 independently operating detectors, excluding a few "bad" detectors still allows the "good" detectors to contribute valid scientific data.
As described below, the resulting screening criterium can be used with the nifpmsel task. As long as the user properly maintains the FPM selection information of their data, accurate responses (ARFs and RMFs) can be computed and thus accurate fluxes can be reported.
niautoscreen performs screening during finite time intervals. By default, this interval is a per-pointing snapshot (i.e. each contiguous pointing GTI), A typical NICER pointing snapshot is 100-2000 seconds long.
Some degrading qualities will typically last during an entire pointing, for example, the round-robin effect (see below). However, the NICER team now recognizes that some degrading qualities may occur over a briefer time interval. After the NICER light leak, starting in May 2023, NICER is more sensitive to lighting conditions, which can evolve over shorter timescales. For example, the behavior of undershoot resets, which are sensitive to optical loading, can change over the timescale of hundreds of seconds. This can also apply to other data-related screening such as LOWMEM quantities, since they are related to optical loading.
For this reason, niautoscreen can also subdivide pointing GTI entries into even shorter segments. This duration is governed by the parameter tseg, which is the segmentation time in seconds. The screening described below, applies to each time segment, first defined by the on-target pointing, and then subdivied into segments by tseg. (Note that segmentation can be disabled by setting tseg to 0).
Since pointing GTIs are rarely exact multiples of "tseg," the actual segmentation may not be exactly tseg. niautoscreen never allows a segment be smaller than tseg, so tseg is a minimum segment size.
niautoscreen checks each time segment for certain NICER count rates that may indicate problems that could cause degradation.
niautoscreen checks FPMs and MPUs for "out-of-family" behavior. It does this by computing the median and robust standard deviation of the population of modules for each time segment. Modules whose value is more than "N" sigma plus a buffer amount are considered bad.
In other words, when considering a housekeeping rate X, if the module's rate X satisfies this inequality,
X > MEDIAN + nsigma x SIGMA + rbuf
then the module is considered bad. Here MEDIAN and SIGMA are the
robust quantities estimated from the population of modules, and
"nsigma" and "rbuf" are per-screening parameters. These parameters
are specified as xxxscr="nsigma:rbuf:min:max". For example, if
underonlyscr="2.0:20.0", then FPMs whose undershoot rates exceed the
median rate by 2 sigma plus 20 ct/s/detector are considered bad.
These two quantities, nsigma and rbuf, are used by all of the
screening criteria. The quantity nsigma is unitless (it is a number
of "sigma") and the quantity rbuf is in units of the quantity being
checked, as described below. The "min" and "max" values are described below.
In addition, niautoscreen settings for each screened parameters also allow for a short-circuiting minimum and maximum value, specified as the "min" and "max" value in the xxxscr parameter values. If the lower or upper limit values estimated from the data are too large, then the short circuiting values are used instead.
niautoscreen screens for the following conditions.
Undershoots. Individual FPM undershoot count rates in units of ct/s/FPM (MPU_UNDERONLY_COUNT) are checked using the above methodology (parameter underonlyscr).
Overshoots. Individual FPM overshoot count rates in units of ct/s/FPM (MPU_OVERONLY_COUNT) are checked using the above methodology (parameter overonlyscr).
Noise peak rate. Individual FPM noise peak rates below 0.25 in units of ct/s/FPM keV (MPU_NOISE25_COUNT) are checked using the above methodology. (parameter noise25scr).
0.3-0.7 keV rate. Individual FPM noise peak rates in the range 0.3-0.7 keV in units of ct/s/FPM keV (MPU_XRAY_PI_0030_0070) are checked using the above methodology. (parameter noiseextscr). In some cases, the detector noise peak becomes shifted or extended from ~0.15 keV into the range 0.3-0.7 keV, which contaminates a normally good region of spectral fitting range. This screening will remove detectors which have count rates much larger than the population. (parameter noiseextscr)
"Shredded" GTIs. Individual MPU GTIs are checked for the condition of "shredded" GTIs exists. This condition is indicated by many small GTIs during a given time segment, and may indicate telemetry data losses due to high counting rates. The quantity is number of GTI transitions per second of clock time. (parameter mpugtiscr)
"Round robbin" fault condition. In some cases an MPU may enter "round robbin" state, wherein it cycles an FPM off every few seconds. This may be due to extremely high count rates which create a race condition within the MPUs systems. Even if count rates reduce to nominal values, the condition may persist until the next time segment begins. The values are in units of GTI transitions per second of clock time. (parameter roundrobbinscr)
LOWMEM FIFO data losses. Each MPU tracks data losses within
its data processing system. Some losses occur due to a busy MPU or
downlink channel limit, and can be accounted for in the MPU's GTI.
However, if the MPU is extremely busy (which typically occurs at
extreme count rates), a LOWMEM FIFO condition can occur, which indicates
the number of dropped events that have not been processed. Such conditions
cannot be accounted for by revising exposures or GTIs and represent an
uncalibratable dead time. This setting removes MPUs that have a
high number of LOWMEM FIFO lost events.
There are two screening parameters lowmemscr which concerns the absolute
count of lowmems per second per GTI, and lowmemfrscr, which deals with
the fractional amount of lowmem occurrences compared to the total
event rate. The NICER team recommends to use the fractional screening
(lowmemfrscr) and not the absolute value.
Known bad times. A file containing times of known detector-related problems is contained in CALDB. This file consists of four required columns: START and STOP (as in any GTI file); DET_ID which contains the affected detector ID (x9 for all detectors in MPUx and 99 for full array); and QUALITY level (0 meaning good and 3 meaning completely bad). This file is used to screen out particular detectors at particular times.
niautoscreen requires as input a NICER event file and filter file (MKF file).
The NICER filter file must cover the entire observation in question, and must be processed with a recent version of niprefilter2 / nicerl2. The task will check for the required minimum version and report an error if the filter file is "stale."
The NICER event file is used to check for the "shredded GTI" condition. It should be the MPU-merged "ufa" file, i.e. suffix of 0mpu7_ufa.evt. It is not recommended to use the "cl" file since the goal is to screen raw data for known conditions, and the "cl" file may have already had some bad data screened out.
The output of niautoscreen is a detlist screening expression that can be used by nifpmsel.
To use this expression with nifpmsel, use the following notation,
nifpmsel ... detlist=launch,@myscreening.txtwhere myscreening.txt is the output file produced by niautoscreen. The "@" symbol indicates that nifpmsel should read detlist screening items from the listed file.
Run niautoscreen for the indicated event file and place the output screening expression in the text file filter.txt.
niautoscreen ni0000000000_0mpu7_ufa.evt filter.txt ni0000000000.mkf