NAME

USAGE

DESCRIPTION

The nimaketime task creates a GTI file for event screening based on standard NICER screening criteria.

nimaketime does not actually screen events; rather it creates a GTI file that can be used to screen events using a task like extractor or niextract-events

nimaketime is a helper script that can construct a filtering expression for you. You can also run maketime yourself with the same expressions.

nimaketime allows you to input one or more GTIs to be combined with "AND" (intersection) merging. This allows you to bring in previous screening to be combined with the nimaketime screening. You can also give a custom filtering expression to be applied to the .mkf file which is not provided by nimaketime.

SCREENING CRITERIA

nimaketime can apply the following screening criteria. The exact filtering string is also given in quotes next to each item.

SAA

The following options are used for screening out times when NICER passes through the South Atlantic Anomaly (SAA). The SAA is a source of very high background and may also distort the NICER energy scale calibration.

Use either nicersaafilt=YES or saafilt=YES, but not both.

• nicersaafilt=YES. Excludes times within SAA as defined by NICER_SAA. This is a NICER-specific definition of the SAA stored in CALDB.
  "(NICER_SAA==0)
  The NICER_SAA contour is smaller than the standard SAA contour, but has been customized for NICER's environment.
• saafilt=NO. When YES, excludes times within the generic SAA defined by prefilter.
  "(SAA==0)
  The SAA contour is larger (more conservative) than NICER_SAA, and may be useful for faint targets where conservative screening is desired. This is "no" by default because normally you do not need to apply both the generic and NICER-specific screening.

Pointing and On-Target Tracking

• trackfilt=YES. Excludes times when NICER is not in target tracking mode.
  (ATT_MODE==1 && ATT_SUBMODE_AZ==2 && ATT_SUBMODE_EL==2)
  This is on by default to select data when NICER is tracking a target.
• ang_dist=DIST. Excludes times when the angular distance between the target and NICER's boresight is greater than DIST.
  (ANG_DIST < DIST)
  The pointing constraint is currently DIST=0.015 (within 0.9 arcmin of target). This may need to be loosened on rare occassions when pointing is compromised. See "DEALING WITH OFFSET POINTING" below for more information about offset pointing.
• st_valid=YES. Excludes time when star tracker solution is not valid.
  (ST_VALID == 1)
  This option was added in nimaketime version 1.4, and will exclude times when the NICER tracking system estimated that it was on-target but the star tracker did not provide valid solution. The NICER team generally recommends the screening cut.
• elv=MINELV. Excludes times when the distance to the earth limb is less than MINELV.
  (ELV > MINELV)
  The default elevation constraint is MINELV=15 degrees. This was changed from 30 degrees in version 1.3 and earlier. The primary reason to use this constraint is to avoid absorption and fluorescence by Earth's atmosphere and optical loading by reflected light.
• br_earth=MIN_BR_EARTH. Exclude times when distance to the bright earth is less than MIN_BR_EARTH.
  (BR_EARTH > MIN_BR_EARTH)
  The default bright earth constraint is MIN_BR_EARTH=30 degrees. This was changed to 30 from 40 degrees in version 1.3 and earlier. The primary reason to use this constraint is to avoid optical loading by reflected light.

Background and Detector Performance

• cor_range=A-B. Specify a range of magnetic cut-off rigidity (COR) to accept (in GeV/c). This can be used for filtering of high background regions. You can use the notation cor_range="A-" to specify a minimum but no maximum, cor_range="-B" to specify a maximum but no minimum, and the default cor_range="-" accepts all CORs.
  (COR_SAX > A && COR_SAX < B)
  NICER background is generally correlated with the geomagnetic location of the observatory, and lower numbers indicate higher background. By default the NICER team does not have a recommended COR range, but instead uses overshoot cuts (see below).
• min_fpm=MIN_FPM. Specify a minimum number of enabled detectors.
  (NUM_FPM_ON > MIN_FPM)
  The default value for this is MIN_FPM=7. This was changed from 38 in versions 1.3 and earlier. The reason for this cut is to ensure that at least some detectors are enabled, and exclude times when the full array was off.
• underonly_range=A-B. Excludes times when undershoot count rate is out of range A-B counts per second per FPM.
  (FPM_UNDERONLY_COUNT > A && FPM_UNDERONLY_COUNT < B)
  NICER's performance, especially the energy gain scale, is affected by optical loading. Reduce optical loading effects by reducing the default range 0-500 to a smaller range (0-50 is optimal). This option was added in version 1.4
• overonly_range=A-B. Excludes times when overshoot count rate is out of range A-B counts per second per FPM.
  (FPM_OVERONLY_COUNT > A && FPM_OVERONLY_COUNT < B)
  NICER's background level is correlated with "overshoots", which are saturating particle detections. The default range of 0-30 will exclude most strong background flare-ups. Note that this cut is applied in addition to overonly_expr. This option was added in version 1.4.
• overonly_expr=EXPR. Excludes times when FPM_OVERONLY_COUNT exceeds EXPR. EXPR is any maketime expression using columns from the filter file.
  (FPM_OVERONLY_COUNT < EXPR)
  This applies additional screening based on environment-specific conditions, using columns from the filter file. The default value is currently "NONE" indicating to not use a special expression. In versions of nimaketime from versions 1.4 through 1.10, the expression was ,
  overonly_expr="1.52*OVERONLY_NORM*COR_SAX**(-0.633)" (OBSOLETE)
  which was heuristically determined by the NICER team to exclude most background flares in different geomagnetic conditions. Because background models are now included in NICERDAS that can handle modest background increases, this expression has been changed to "NONE". This option was added in version 1.4. Note that the name OVERONLY_NORM is taken from the maximum value of overonly_range (or 1.0 if no maximum was given).
• max_lowmem=m. Excludes times when TOT_LOWMEM_FIFO exceeds the value specified.
  (TOT_LOWMEM_FIFO < m)
  This column indicates when the MPU is discarding events due extremely high count rates, in which case the calibration problems will likely arise. The default value of 0 disables this criterium, in favor of a similar screening done in niautoscreen.
• threshfilter=(ALL|NIGHT|DAY). A shortcut to selected desired threshold range, as described below. Use either threshfilter or thresh_range, but not both.
  threshfilter=ALL corresponds to thresh_range=*-*
  threshfilter=NIGHT corresponds to thresh_range=-3-3
  threshfilter=DAY corresponds to thresh_range=32-38
  The cleanest data will be found with threshfilter=NIGHT. Note that before the optical light leak in May 2023, the threshold was always at its "night" setting, so threshfilter=NIGHT will select all data, regardless of whether it was observed in orbit day or orbit night.
• thresh_range=A-B. Only includes times when the relative NICER threshold is within the given range.
  (DELTA_SLOW_LLD > A && DELTA_SLOW_LLD < B)
  The value reported in the DELTA_SLOW_LLD column gives the relative threshold setting. Here relative means relative to the nominal post-launch threshold, and threshold setting means the read-back value of the threshold, in units of the commanded threshold setting. Typically this value is ~0 within tight tolerances, and the default range is -3 to +3 (which is approximately within 12 eV of nominal). Set the range to -3-38 to accomodate data taken when NICER experienced a large light leak in 2023. However, mixing thresholds must be done with extreme caution.

GTI Filtering

• min_gti=m. Specify a minimum duration for a GTI of m seconds.
• erodedilate=e. Specify an erodedilate parameter of e seconds.

Occasionally, NICER performs offset pointing to avoid a contaminating spoiler target. This may also occassionally occur if the incorrect target coordinates were used and/or a preliminary target position was used for initial observations.

In that case, the standard ang_dist=NNNN filtering criterium may not return any valid data. This is because the offset pointing location is more than the specified amount (default 0.015 deg = 54 arcsec).

If the goal is to get some data, regardless of whether the data is high quality or on-axis, then using the following filtering expression:

   nimaketime ... ang_dist=180 expr="ANGSEP(RA,DEC,#RA_NOM,#DEC_NOM)<0.015"

ADDITIONAL GEOGRAPHIC SCREENING

Users may wish to apply additional screening criteria based on geographic position. Since this can be difficult, nimaketime provides some shorthand capability for geographic screening.

The screening should be the form of a region file, and on the command line as latlonregfile=filename.reg. The region file should in the format of a standard spatial region file, and specify the contour of good or bad data. The region file should specify longitude in the range SAT_LON=-180 to SAT_LON=+180; this is done to allow easy filtering of the South Atlantic Anomaly (SAA), which crosses SAT_LON==0.

Note that this screening will be applied in addition to the other screening criteria listed above. If you want to avoid double-screening of SAA, you may want to set nicer_saa=NO and saa=NO.

Users can also use standard regions that are located in NICER CALDB. Currently there are two named regions in CALDB:

• SAACONT - standard NICER contour
• SAACONTLARGE1 - enlarged NICER contour

If the sense of the region file should be inverted to keep good data, then use latlonreginvert=YES. This is true for SAA filtering.

For example, for enlarged SAA region filtering, use the following options:

  nimaketime ... latlonregfile=CALDB:SAACONTLARGE1 latlonreginvert=YES nicer_saa=NO

DIAGNOSING LOST DATA

nimaketime can help diagnose the reason for lost data. It can print the amount of good time after certain stages of filtering.

It does this if calc_intermediate=YES and chatter is 2 or more. It does not interfere with the final calculation of good time. If this process ends up using too much computing time, for example if your filter file is very large, you can disable the intermediate calculation by setting setting chatter to 0 or 1, or by setting calc_intermediate=NO.

If enabled, the task prints a table which shows the remaining good time after a set of filtering stages. Each good time printed is cumulative, including all the previous stages. The values shown include the remaining good time, as well as the number of good time intervals.

The stages that can be printed include:

• On-Target + Observing - pointing control is trakcing on target (trackfilt), target is in the field of view (given by ang_dist), target is above the earth and br_earth elevations, star tracker solution is valid (st_valid) and detectors are enabled (min_fpm).
• LOWMEM + Threshold Quality - total number of MPU lost events is less than min_lowmem, and FPM relative threshold setting is within range given by thresh_range. Note that older filter files may not have the required columns, and in that case these filtering steps are skipped.
• SAA + COR + Orbit - object is not in SAA (nicersaa/saa), and not in keep-out region if provided by latlonregfile, and COR_SAX is within cor_range.
• Undershoots - MPU_UNDERONLY_COUNTS are within the range given by underonly_range.
• Overshoots - MPU_OVERONLY_COUNTS are within the range given by overonly_range.
• User Expression - any filtering by the user screening expression (expr).
• User GTI - any additional reduction after applying the user-specified good time intervals (ingtis)
• Final Merged Screening - the final resulting good time.

DEALING WITH MANY TINY GTIs

In some cases a set of filtering criteria results in many, tiny GTIs. This often occurs if the filtering criteria sample a noisy distribution such that the filtered value is sometimes inside the range and sometimes outside of the range. The result may be many GTIs that are 1-2 seconds.

As of nimaketime version 1.12, the task has a way to combat this problem by filtering the GTI rows. The task uses the capability of ftadjustgti to remove small GTIs. It uses the capability called "erodedilate" which is first reduces each GTI by a set amount and then increases each GTI by the same amount. The net result is that small GTIs are removed.

Note that this process will effectively remove any GTI smaller than the erodedilate setting (in seconds) x 2. nimaketime acts to conservatively remove (rather than heal) any "shredded" GTI situations.

The process described above apply to any telemetry-based selections. Any user-suppled expressions (expr) or input GTIs (ingtis) are used as-is with no GTI filtering.

PARAMETERS

infile [filename]

Input prefilter (.mkf) file name.

outfile [filename]

Output GTI file name.

(nicersaafilt = "YES") [boolean]

Apply NICER_SAA filtering. See above.

(saafilt = "NO") [boolean]

Apply SAA filtering. See above.

(trackfilt = "YES") [boolean]

Apply pointing tracking filtering. See above.

(onlytrackfilt = "NO") [boolean]

If no, then only apply trackfilt=YES and disregard all other types of filtering. This is intended only for internal pipeline bookkeeping and not recommended for general scientist usage.

(st_valid = "YES") [boolean]

Require star tracker valid solution?

(ang_dist=0.015) [real]

Apply ANG_DIST filtering (degrees). ANG_DIST must be less than this value. The default of 0.015 degrees is 54 arcseconds. See above for more information.

(elv=15) [real]

Apply ELV filtering (degrees). ELV must be greater than this value. See above.

(br_earth=30) [real]

Apply BR_EARTH filtering (degrees). BR_EARTH must be greater than this value. See above.

(cor_range="*-*") [string]

Apply COR_SAX filtering (GeV/c). Specify a range A-B. See above.

(underonly_range="0-500") [string]

Apply undershoot count rate range per module. Specify a range A-B. See above.

(overonly_range="0-30") [string]

Apply overshoot count rate range per module. Specify a range A-B. See above.

(overonly_expr="NONE") [string]

Apply curve-base overshoot filtering as an expression, or NONE for no additional overonly expression-based screening. See above.

(min_fpm=7) [integer]

Apply NUM_FPM_ON filtering. See above.

(calc_intermediate=YES) [boolean]

Calculate intermediate good time summary statistics? If yes, then nimaketime calculates the Good Time Intervals after several intermediate stages of screening. The final output is not changed. Intermediate values are only printed if chatter is greater than or equal to 2. Note that the total good time values are successively cumulative, and include all the filtering from that stage and previous stages. Set calc_intermediate=NO if nimaketime is using too many computing resources.

(ingtis="NONE") [string]

Apply additional GTI filters. If you have done external filtering and produced a GTI file, you can merge with the maketime GTI. Either a comma-separated list of files or an @filelist.lis file. All GTIs (including the maketime GTI) are combined using "AND" (intersection) merge. A value of "NONE" indicates no additional GTI filtering.

(max_lowmem=0) [real]

Maximum value of TOT_LOWMEM_FIFO lost events, in units of events per second for the entire NICER array. A value of 0 disables this screening criterium.

(thresh_range="-3.0-3.0") [string]

Range of NICER lower-level threshold settings to include. This is the array-average relative threshold setting, where 0.0 indicates the nominal threshold level, and the units correspond to the flight relative threshold offset value. For NICER optical light leak conditions, it would be typical to use a range of "-3.0-38" to include both nominal and "+35" settings, and "32-38" to include just the daytime threshold settings.

(threshfilter="NIGHT") [string]

A shortcut to select a certain desired threshold range, as described above. One of "ALL", "DAY" or "NIGHT".

(mingti=5.0) [real]

Minimum GTI size in seconds for GTI filtering. This applies to any telemetry-selected filtering criteria (i.e. not user expressions or user GTIs).

(erodedilate=5.0) [real]

Specify the "erodedilate" parameter for GTI filtering of GTI rows. This applies to any telemetry-selected filtering criteria (i.e. not user expressions or user GTIs).

(expr="NONE") [string]

Apply additional maketime expressions, beyond the standard criteria. This should be a standard maketime filtering expression that evaluates to true when conditions are good. A value of "NONE" indicates to use only the standard filtering.

(latlonregfile="NONE") [string]

Apply an additional geographic screening as described above. Use "NONE" to skip this step, or "CALDB:name" to query CALDB for region named "name".

(latlonreginvert=NO) [boolean]

If NO, then the region file encloses good data. If YES, then the region file should enclose bad data.

(outexprfile="NONE") [string]

nimaketime will write its filtering expressions to this file so that the user can inspect them, or to use them in downstream processing. A value of "NONE" indicates to not save the filtering expressions.

(cleanup="YES") [boolean]

If yes, then clean up temporary files. If no, temporary files remain. This is typically for debugging.

(clobber = NO) [boolean]

If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.

(chatter = 2) [integer, 0 - 5]

Amount of verbosity of the task. For chatter=0, no output is printed. For chatter=5, debugging output is printed.

(history = YES) [boolean]

If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the task parameters that were used to produce the output file.

EXAMPLES

1. Apply standard NICER screening criteria.

  nimaketime ni1706260815.mkf good.gti 

This example will supply standard screening as detailed above. 2. Apply more custom screening

  nimaketime ni1706260815.mkf good40.gti elv=40 ang_dist=0.005 ingtis=extra.gti

This example applies the standard screening criteria, with the exceptions that the minimum ELV elevation is set to 40 degrees, the maximum angular distance is set to 0.005 degrees, and a user-supplied GTI is supplied in extra.gti (which is merged with "AND" merging with the maketime GTI).

SEE ALSO

niprefilter, maketime

LAST MODIFIED