NAME

niautoscreen - perform standard NICER per-FPM and per-MPU screening

USAGE

niautoscreen infile outfile mkfile

DESCRIPTION

The niautoscreen task performs standard per-FPM and per-MPU screening for a NICER observation. This task catches issues which may occur for a single FPM, or a single MPU, that the overall screening done by nimaketime may miss.

This task checks for "out-of-family" behavior of individual FPMs and MPUs, which may degrade NICER instrument performance if they are included in the data. Here, "out-of-family" means that a particular FPM or MPU differs significantly from the group as a whole, using robust statistical measures.

This task considers individual NICER FPMs and MPUs. The goal is to only exclude "bad" data for the individual detectors and individual time ranges where the bad behavior is occuring. Since NICER has 52 independently operating detectors, excluding a few "bad" detectors still allows the "good" detectors to contribute valid scientific data.

As described below, the resulting screening criterium can be used with the nifpmsel task. As long as the user properly maintains the FPM selection information of their data, accurate responses (ARFs and RMFs) can be computed and thus accurate fluxes can be reported.

Screening Performed

niautoscreen performs screening on a per-pointing snapshot basis. It checks each pointing snapshot (i.e. each contiguous pointing GTI), for certain NICER count rates that may indicate problems that could cause degradation. A typical NICER pointing snapshot is 100-1000 seconds long.

Data are excluded on pointing snapshot basis as well. This means that if a detector is misbehaving for some reason for 100 seconds of a pointing snapshot, this task will still exclude the entire snapshot. On the beneficial side, niautoscreen will only exclude detectors when bad behavior is detected, and will keep those detectors when the behavior is considered good.

niautoscreen checks FPMs and MPUs for "out-of-family" behavior. It does this by computing the median and robust standard deviation of the population of modules for each pointing snapshot. Modules whose value is more than "N" sigma plus a buffer amount are considered bad.

In other words, when considering a housekeeping rate X, if the module's rate X satisfies this inequality,

    X > MEDIAN + nsigma x SIGMA + rbuf
then the module is considered bad. Here MEDIAN and SIGMA are the robust quantities estimated from the population of modules, and "nsigma" and "rbuf" are per-screening parameters. These parameters are specified as xxxscr="nsigma:rbuf:min:max". For example, if underonlyscr="2.0:20.0", then FPMs whose undershoot rates exceed the median rate by 2 sigma plus 20 ct/s/detector are considered bad. These two quantities, nsigma and rbuf, are used by all of the screening criteria. The quantity nsigma is unitless (it is a number of "sigma") and the quantity rbuf is in units of the quantity being checked, as described below. The "min" and "max" values are described below.

In addition, niautoscreen settings for each screened parameters also allow for a short-circuiting minimum and maximum value, specified as the "min" and "max" value in the xxxscr parameter values. If the lower or upper limit values estimated from the data are too large, then the short circuiting values are used instead.

niautoscreen screens for the following conditions.

Undershoots. Individual FPM undershoot count rates in units of ct/s/FPM (MPU_UNDERONLY_COUNT) are checked using the above methodology (parameter underonlyscr).

Overshoots. Individual FPM overshoot count rates in units of ct/s/FPM (MPU_OVERONLY_COUNT) are checked using the above methodology (parameter overonlyscr).

Noise peak rate. Individual FPM noise peak rates below 0.25 in units of ct/s/FPM keV (MPU_NOISE25_COUNT) are checked using the above methodology. (parameter noise25scr).

0.3-0.7 keV rate. Individual FPM noise peak rates in the range 0.3-0.7 keV in units of ct/s/FPM keV (MPU_XRAY_PI_0030_0070) are checked using the above methodology. (parameter noiseextscr). In some cases, the detector noise peak becomes shifted or extended from ~0.15 keV into the range 0.3-0.7 keV, which contaminates a normally good region of spectral fitting range. This screening will remove detectors which have count rates much larger than the population.

"Shredded" GTIs. Individual MPU GTIs are checked for the condition of "shredded" GTIs exists. This condition is indicated by many small GTIs during a given pointing snapshot, and may indicate telemetry data losses due to high counting rates. The quantity is number of GTI transitions per second of clock time.

"Round robbin" fault condition. In some cases an MPU may enter "round robbin" state, wherein it cycles an FPM off every few seconds. This may be due to extremely high count rates which create a race condition within the MPUs systems. Even if count rates reduce to nominal values, the condition may persist until the next pointing snapshot beings. The values are in units of GTI transitions per second of clock time.

Known bad times. A file containing times of known detector-related problems is contained in CALDB. This file consists of four required columns: START and STOP (as in any GTI file); DET_ID which contains the affected detector ID (x9 for all detectors in MPUx and 99 for full array); and QUALITY level (0 meaning good and 3 meaning completely bad). This file is used to screen out particular detectors at particular times.

Inputs

niautoscreen requires as input a NICER event file and filter file (MKF file).

The NICER filter file must cover the entire observation in question, and must be processed with a recent version of niprefilter2 / nicerl2. The task will check for the required minimum version and report an error if the filter file is "stale."

The NICER event file is used to check for the "shredded GTI" condition. It should be the MPU-merged "ufa" file, i.e. suffix of 0mpu7_ufa.evt. It is not recommended to use the "cl" file since the goal is to screen raw data for known conditions, and the "cl" file may have already had some bad data screened out.

Outputs

The output of niautoscreen is a detlist screening expression that can be used by nifpmsel.

To use this expression with nifpmsel, use the following notation,

   nifpmsel  ... detlist=launch,@myscreening.txt
where myscreening.txt is the output file produced by niautoscreen. The "@" symbol indicates that nifpmsel should read detlist screening items from the listed file.

PARAMETERS

infile [filename]
NICER unscreened ("ufa") MPU-merged event file, typically with the 0mpu7_ufa.evt suffix. This file must have FPM selection information attached.
outfile [filename]
Name of output screening file.
mkfile [filename]
Name of filter (MKF) file which corresponds to infile.
(underonlyscr="2.0:20:0:1000") [string]
Screening criteria for the high undershoot condition. The value is of the form "nsigma:rbuf:min:max" as described above, where min and max are optional. The default value indicates a +/- 2 sigma range, with an additional +/- 20 ct/s buffer, and the lower limit must include 0 while the upper limit must be less than 1000 ct/s.
(overonlyscr="2.0:10:0:1200") [string]
Screening criteria for the high overshoot condition. The value is of the form "nsigma:rbuf:min:max" as described above, where min and max are optional. The default value indicates a +/- 2 sigma range, with an additional +/- 10 ct/s buffer, and the lower limit must include 0 while the upper limit must be less than 1200 ct/s.
(noise25scr="3.0:50.0:0:1200") [string]
Screening criteria for the high noise peak rate condition. The value is of the form "nsigma:rbuf:min:max" as described above, where min and max are optional. The default value indicates a +/- 3 sigma range, with an additional +/- 50 ct/s buffer, and the lower limit must include 0 while the upper limit must be less than 1200 ct/s.
(noiseextscr="3.0:0.05") [string]
Screening criteria for the 0.3-0.7 keV "X-ray" band. This band may occasionally have high count rates associated with an extended noise peak, which indicates detector problems. The value is of the form "nsigma:rbuf:min:max" as described above, where min and max are optional. The default value indicates a +/- 3 sigma range, with an additional +/- 0.05 ct/s buffer.
(mpugtiscr="0.0:0.003:0:0.5") [string]
Screening criteria for the per-MPU shredded GTI condition, in the form of number of GTI intervals per second in each pointing. The value is of the form "nsigma:rbuf:min:max" as described above, where min and max are optional. The default value indicates a +/- 0.003 GTI count per second range around the median, and the lower limit must include 0 while the upper limit must be less than 0.5 GTIs per second.
(roundrobbinscr="0:0.0015") [string]
Screening criteria for the "round robbin" condition, in the form of number of round robbin transitions per second in each pointing. The value is of the form "nsigma:rbuf:min:max" as described above, where min and max are optional. The default value indicates a +/- 3 +/- 0.0015 transition per second range around the median, and the lower limit must include 0 while the upper limit must be less than 0.5 transitions per second.
(btifile="CALDB") [string]
Name of file contoining known bad times, or CALDB or NONE.
(btilevel=1) [integer]
Numerical quality level. Entries in the btifile with QUALITY greater than btilevel are considered bad. The values are: 0 (good), 1, (non-routine condition, but otherwise good data); 2 (bad data with some possible good data, used with caution); 3 (bad data with little or no good data).
(minmkfver=2.0) [real]
Minimum version of niprefilter2 used to compute filter file.
(cleanup="YES") [boolean]
If yes, then clean up temporary files. If no, temporary files remain. This is typically for debugging.
(clobber = NO) [boolean]
If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.
(chatter = 2) [integer, 0 - 5]
Controls the amount of informative text written to standard output. Setting chatter = 4 or higher will produce detailed diagnostic output; chatter = 1 prints out a basic diagnostic message. The default is to produce a brief summary on output.
(history = YES) [boolean]
If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the niautoscreen task parameters that were used to produce the output file.

EXAMPLES

Run niautoscreen for the indicated event file and place the output screening expression in the text file filter.txt.


   niautoscreen ni0000000000_0mpu7_ufa.evt filter.txt ni0000000000.mkf

SEE ALSO

nifpmsel

nicerl2

LAST MODIFIED

Sep 2022