niextract-events - Fast event filtering, similar to extractor


niextract-events filename eventsout timefile= regionfile= ...


The niextract-events task does basic NICER event filtering. It could be considered a stripped down version of 'extractor' which does only event filtering (no images, spectra or light curves), but it does it many times faster.

niextract-events can perform temporal or spatial filtering using good time interval files (GTIs) and/or spatial region files, respectively. Also, basic column filtering by command-line interval ranges are supported as in 'extractor'.

niextract-events does not accept all the parameters of extractor (in particular, not the spectrum, light curve, or image parameters), but the event filtering parameters are accepted, and have the same names.

Time filtering is performed by using GTIs. The GTIs from all the input 'timefile's (if any) are ORed together; the GTIs from all the input event files are ORed together; and then finally, these two sets are ANDed together to form the master time filter.


NICER's event data comes from 52 operating flight detectors. In order to manage the exposure information of each detector, a system called FPM Selection information has been devised and can be attached to a NICER event file. This information is stored in the extension FPM_SEL, as well as various GTI_* extensions that describe the exposures of individual modules.

This task does not allow selecting or de-selecting individual detector modules. Use nifpmsel for such selections. Instead, niextract-events is best suited for temporal filtering, as well as interval filtering as described below.

niextract-events properly handles combining files with FPM Selection information. It will examine the exposure information of each indvidual module and create new FPM Selection information in the output file. Note that this really only makes sense when gtimerge=OR, i.e. the combination of event files from different time ranges.

The processing of FPM selection information is enabled by setting fpmsel=YES, which is the default. However, if one or more of the input files is missing FPM Selection information, output FPM Selection cannot be created and its output is disabled. This is not considered an error, since files that pre-date the FPM Selection system may not have this information. A warning message is printed in this case. To disable the warning message, set fpmsel=NO. Also, if the intent is to merge event files from the same time interval with gtimerge=AND, it is recommended to set fpmsel=NO since FPM Selection doesn't make sense in this case.


nextract-events understands standard extractor interval filters, as well as custom filtering selections.

Interval filters can be specified by appending them inside [] on the end of the infile parameter. This should be a comma separated list of the form COLUMN=n or COLUMN=n:m to specify that column must equal n, or be in the range n to m, respectively. For instance, to accept only events with PHA lying between 7 and 30 the filename will be specified by "infile.evt[pha=7:30]".

Note that since square brackets are typically interpreted by Unix shells as wild card pattern matches, users will be required to enclose interval screening filters typed on the command line with quotation marks. So, for example,

  niextract-events "infile.evt[pha=7:30]" output.evt ...
the quotation marks prevents the unix command line from interpreting the special characters.

If a list of filenames is given (using "@") then the interval filtering should be specified on the input paramater, not on the individual event filenames. Successive interval filters should be separated by spaces or commas, such as infile.evt[pha=7:30,EVENT_FLAGS=bx11000]. Interval filters must correspond to a column name in the input event file.

niextract-events also accepts filters not possible with extractor. You can use simple arithmetic expressions for comparison such as < and >. For example to screen on PI_RATIO, use "PI_RATIO≥1.4".

Common interval filters to apply are:


niextract-events does several filtering steps. Using the standard on-the-fly CFITSIO filtering capability, the process could in principle use a large amount of memory. niextract-events therefore has memory-related optmizations.

If the input file size exceeds "minselectmb" megabytes (where "minselectmb" is a task parameter), then niextract-events uses an alternate filtering approach that conserves RAM memory but does use more disk space. Users should be prepared for run to use at least double the storage size of the inputs. For most use cases, the analyst should not need to change minselectmb from its default.

However, it should be pointed out that some processing may still require a large amount of memory. This may be unavoidable if the "colexpr" parameter is used.


filename [string]
The input event filename. If this starts with "@" then it is assumed to be an ascii file containing a list of event files to be read. Extractor interval filters can be specified in the file name. If an @filelist.lis file is specified, then the filters should be specified as @filelist.lis[filters] instead of applying them to the individual files in the file list. Note that any filters used on the command line will need to be enclosed in quotation marks to prevent the Unix command line shell from interpreting the square brackets as special characters.
The name for the output events file.
(regionfile = "NONE") [string]
The name of a spatial region file, or NONE for no spatial filtering. The region coordinates are assumed to be given by the xcolf and ycolf parameters.
(timefile = "NONE") [string]
The name of a temporal good time interval file, or NONE for no temporal filtering. Note that there is no 'gtinam' parameter; you should specify the extension name directly in the timefile parameter, for example as "timefile='mytime.gti[STDGTI]'".
(xcolf = "DETX") [string]
The name of the column used for the X coordinate when region filtering.
(ycolf = "DETY") [string]
The name of the column used for the Y coordinate when region filtering.
(tcol = "TIME") [string]
The name of the column used for the time of the event.
(events = "EVENTS") [string]
The name of the EVENTS extension in the input files.
(obsid = "NONE") [string]
Value to change OBS_ID keyword of output, or NONE for no change.
(gtimerge = "OR") [string]
The method to use to merge the GTI extensions of each event file, either AND or OR. Use "OR" if the input files represent data from different time ranges. Use "AND" if the input files represent data from different detectors in the same time range.
(fpmsel = "YES") [boolean]
If set to YES, the task will process FPM Selection information from the input files and update FPM Selection information of the output. Note that if one or more of the inputs does not have FPM Selection information, a warning will be printed and the task will carry on, but not write FPM Selection information to the output.
(gti = "GTI") [string]
The name of the GTI extension in the input event files, which is "GTI" for NICER.
(minselectmb = 2000) [integer]
If the input file exceeds this size in megabytes, then an alternate filtering strategy is used which conserves memory. The default of 2000 indicates a maximum task memory usage of about 2 GB. Decreasing this number will reduce RAM performance but increase the amount of disk I/O.
(copyall = NO) [boolean]
Not currently used.
(clobber = NO) [boolean]
If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.
(history = YES) [boolean]
If set, then write niextract-events-specific HISTORY information. Specifically, the FILIN* keywords documenting which input files were used are appended. If set, then the HISTORY keywords of the first input file only are copied to the output. If history=NO, then no FILIN* keywords are written and no HISTORY keywords are transferred.
(chatter = 2) [integer, 0 - 5]
Amount of verbosity of the task. For chatter=0, no output is printed. For chatter=5, debugging output is printed.

(history = YES) [boolean]
If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the task parameters that were used to produce the output file.


1. Perform filtering on theevent files listed in the ASCI file infile.lis. The temporal filters listed in the GTI file select.gti, and the spatial filters listed in the region file select.reg are applied. The output is sent to output.evt.

  niextract-events @infile.lis output.evt timefile=select.gti regionfile=select.reg


nifpmsel - Add or update FPM Selection information