NAME

nicerl2 - Apply NICER Level2 standard calibration and filtering

USAGE

nicerl2 indir ...

DESCRIPTION

The nicerl2 task applies standard NICER-recommended calibration processes to an entire NICER observation, as well as standard screening.

This is a high level task that calls multiple sub-tasks as a convenience to the user. It performs the following standard "Level2" processing steps.

  1. nicercal - apply standard NICER calibration
  2. niprefilter2 - derive calibrated filter (MKF) file
  3. nimaketime - create standard screening good time intervals
  4. nicermergeclean - combine per-MPU data and filter/screen
Please see the help files for these individual tasks for more information about what they do and how they can be used.

A user wishing to have more control over the analysis process may call these tasks individually. nicerl2 simply rolls these subtasks up into a single, easy-to-run task. Many, but not all, parameters of the individual subtasks are available for nicerl2, sometimes with different parameter names to avoid conflicts.

Unlike most HEASoft tasks, this task is designed to work on an entire observation directory. It is expected that the input adheres to the standard NICER directory layout: uncalibrated event files in $INDIR/xti/event_uf; MPU housekeeping data in $INDIR/xti/hk; and auxiliary files in $INDIR/auxil/.

However, an advanced programmer can adjust these locations with the hidden parameters ufdir, cldir, ufafile, clfile, and mkfile. When using these parameters, note that the certain patterns can be used as shortcuts: "$CLDIR" for the output directory name, "$OBSID" for the observation ID number, and "$INDIR" for the input directory. Note that these patterns should appear explicitly with the dollar sign included; typically the command shell will interpret '$name' as a shell variable so appropriate quoting will be required.

OUTPUTS

Output files are placed in the directory specified by cldir, typically $INDIR/xti/event_cl. The output products are:

Normally the "cl" file would be used in further analysis.

Note that the original unfiltered event files ("uf" files) are not modified.

DESTRUCTIVE OVERWRITING and WORKING WITH READ-ONLY INPUT DATA

Note that this task in its default operation will attempt to write to the directory specified by cldir, typically $INDIR/xti/event_cl (the "$INDIR" is a pattern replaced by the actual input directory as described above). In addition, the filter file (.mkf file, typically in $INDIR/auxil) is modified.

Any pre-existing files in these areas will not be destructively overwritten unless clobber=YES (the default is clobber=NO). This is a reminder to the user that destructive removal of data is a possibility and that the user must explicitly allow it by setting clobber=YES.

For most typical applications, where the user is working in an existing NICER observation directory with write access, clobber=YES should be used on the command line to allow nicerl2 to overwrite existing data, with the understanding that all pre-existing contents of 'cldir' will likely be destroyed.

In some cases users may be working with a read-only archive copy of NICER observation data, with no possibility of modifying the data in place. In that case, even if clobber=YES is used, nicerl2 will be unable to run using the defaults. Another slightly similar use case would be if the user is trying different (or new) calibrations and wishes to preserve the pre-existing calibrated outputs.

To accomodate these cases, the user must still have a writable output directory, which can be kept separate from the input directory. Let's say that indir=/archive/1000010101 and the output is placed in /writable/1000010101-output.

The steps to achieve this are:

Here are the commands to be used in this case.
     mkdir /writable/1000010101-output
     cp -p /archive/1000010101/auxil/ni*.mkf* /writable/1000010101-output/
     nicerl2 indir=/archive/1000010101 \
         cldir=/writable/1000010101-output \
         mkfile='$CLDIR/ni$OBSID.mkf'

PARAMETERS

indir [filename]
Input directory name. The directory should be a single NICER observation directory, which in turn contains xti/{events_uf,events_cl,hk,auxil} subdirectories.
ufdir = "$INDIR/xti/event_uf" [filename]
Subdirectory containing per-MPU "uf" event files, which are the input to the task. This parameter is subject to pattern substitutions $INDIR and $CLDIR as described above.
cldir = "$INDIR/xti/event_cl" [filename]
Subdirectory where the output files as listed above are placed. This parameter is subject to pattern substitution $INDIR as described above.
ufafile = "$CLDIR/ni$OBSID_0mpu7_ufa.evt" [filename]
Name of the master "ufa" event file to be created. This parameter is subject to pattern substitutions $INDIR, $CLDIR and $OBSID as described above.
clfile = "$CLDIR/ni$OBSID_0mpu7_cl.evt" [filename]
Name of the master "cl" event file to be created. This parameter is subject to pattern substitutions $INDIR, $CLDIR and $OBSID as described above.
mkfile = "$INDIR/auxil/ni$OBSID.mkf" [filename]
Name of the MKF filter file, to be used as input. This parameter is subject to pattern substitutions $INDIR, $CLDIR and $OBSID as described above.
(mpulist = "0-6") [string]
List of MPUs to process, either as a comma-seprated list, or as a hypenated range. For example, "0,1,2,3,4,5,6" is the same as "0-6".
(picalfile="CALDB") [filename]
Name of PI gain calibration data file for slow chain pulse heights, or "CALDB". For standard operation, the NICER team recommends to use "CALDB".
(pifastcalfile="CALDB") [filename]
Name of PI_FAST gain calibration data file for fast chain pulse heights, or "CALDB". For standard operation, the NICER team recommends to use "CALDB".
(timebiascalfile="CALDB") [filename]
Name of clock calibration file, or "CALDB". For standard operation, the NICER team recommends to use "CALDB". A value of "NONE" disables clock calibration.
(leapinit="AUTO") [string]
Initialize LEAPINIT keyword of event extensions? An integer value indicates to use that many leap seconds. A value of "AUTO" indicates use the appropriate number of leap seconds for the given observation TSTART. A value of "NONE" indicates to not change LEAPINIT.
(nicercal_filtexpr="EVENT_FLAGS=bxxxx00") [string]
Extractor filtering expression to be used for initial screening of events. Typically this screening expression will remove over-shoot and under-shoot events, which cannot be calibrated, but keep X-ray events and forced triggers (EVENT_FLAGS=bxxxx00).
(calstation="FLIGHT") [string]
Station where data was collected. For in-flight operations use calstation=FLIGHT. For ground-based testing, other values can be used. This parameter selects the NICSTATN keyword value in CALDB.
(niprefilter=YES) [boolean]
Run niprefilter to update the filter file?
(orbfile=$INDIR/auxil/ni$OBSID.orb) [string]
Name of the orbit file used by niprefilter. Both the orbfile and orbfile.gz patterns are checked, in that order. This parameter is subject to pattern substitutions $INDIR, $CLDIR and $OBSID as described above.
(attfile=$INDIR/auxil/ni$OBSID.att) [string]
Name of the attitude file used by niprefilter. Both the attfile and attfile.gz patterns are checked, in that order. This parameter is subject to pattern substitutions $INDIR, $CLDIR and $OBSID as described above.
(prefilter_columns=DEFAULT) [string]
A list of prefilter columns to compute, as a space-separated list. Please see the documentation of prefilter, parameter columns, for what values are allowed. A value of "DEFAULT" indicates to use the niprefilter task default.
(niprefilter2=YES) [boolean]
Run niprefilter2 to update the filter file?
(saaregfile="CALDB") [string]
For niprefilter2, the name of the NICER South Atlantic Anomaly (SAA) region file, which will be used to replace the existint NICER_SAA column. Use "CALDB" to query CALDB. Use "NONE" to keep the existing value unchanged.
(vehiclefile="CALDB") [string]
For niprefilter2, retrieve International Space Station vehicle docking information as a function of time. If set to "NONE" then this is disabled. If set to "CALDB" or "CALDB:vehiclename", then use CALDB information for the specified vehicle (default is "SOYUZ").
(issmanfile="CALDB") [string]
For niprefilter2, retrieve International Space Station maneuver information as a function of time. If set to "NONE" then this is disabled. If set to "CALDB" then use CALDB information.
(robofile="CALDB") [string]
For niprefilter2, retrieve robotic "robo" activity information as a function of time. If set to "NONE" then this is disabled. If set to "CALDB" then use CALDB information.
(geomag_columns="DEFAULT") [string]
For niprefilter2, the name of files containing geomagnetic quantities. More than one file can be specified using a comma-separated list. For each file on the list, the geomag_path is prepended. Use the notation "filename(COLNAME)" to specify that the output column should be named COLNAME. A value of NONE means no extra geomagnetic quantities, and a value of "DEFAULT" means to use the task default of niprefilter2.
(geomag_path="DEFAULT") [string]
For niprefilter2, name of the base path for all geomagnetic files. This path is prepended to each item in the geomag_columns list. If DEFAULT, then the task default of geomagterp is used.
(nicersaafilt = "YES") [boolean]
For nimaketime, apply NICER_SAA filtering? .
(saafilt = "NO") [boolean]
For nimaketime, apply SAA filtering?
(trackfilt = "YES") [boolean]
For nimaketime, apply pointing tracking filtering?
(st_valid = "YES") [boolean]
For nimaketime, require star tracker valid solution?
(ang_dist=0.015) [real]
For nimaketime, apply ANG_DIST filtering (degrees). ANG_DIST must be less than this value.
(elv=15) [real]
For nimaketime, apply ELV filtering (degrees). ELV must be greater than this value.
(br_earth=30) [real]
For nimaketime, apply BR_EARTH filtering (degrees). BR_EARTH must be greater than this value.
(cor_range="*-*") [string]
For nimaketime, apply COR_SAX filtering (GeV/c). Specify a range A-B.
(underonly_range="0-200") [string]
For nimaketime, apply undershoot count rate range per module. Specify a range A-B.
(overonly_range="0-1.0") [string]
For nimaketime, apply overshoot count rate range per module. Specify a range A-B.
(overonly_expr="1.52*COR_SAX**(-0.633)") [string]
For nimaketime, apply curve-base overshoot filtering as an expression.
(min_fpm=7) [integer]
For nimaketime, apply NUM_FPM_ON filtering.
(gtifiles="NONE") [string]
Apply additional GTI filters. If you have done external filtering and produced a GTI file, you can merge with the nimaketime GTI. Either a comma-separated list of files or an @filelist.lis file. All GTIs (including the maketime GTI) are combined using "AND" (intersection) merge. A value of "NONE" indicates no additional GTI filtering.
(pirange="20:1500") [string]
For nicermergeclean, pulse height screening criteria to apply. To select PI values A to B use "A:B" notation. The asterisk can be used to select the endpoints. For example 20:* indicates a PI range of 20 to the maximum.
(trumpetfilt=YES) [boolean]
Apply standard background PI_RATIO filtering.
(nicerclean_args="NONE") [string]
Any additional arguments to nicerclean, such as "EVENT_FLAGS=bx1x000 fastsig=250.0". See the help file for nicer clean for more information. The gtifile, trumpetfilt, and pirange parameters are automatically passed and there is no need to specify them again with nicerclean_args.
(cleanup_ufa_files="NO") [boolean]
If yes, then clean up calibrated per-MPU "ufa" files.
(cleanup="YES") [boolean]
If yes, then clean up temporary files. If no, temporary files remain. This is typically for debugging.
(clobber = NO) [boolean]
If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.

(chatter = 2) [integer, 0 - 5]
Amount of verbosity of the task. For chatter=0, no output is printed. For chatter=5, debugging output is printed.

(history = YES) [boolean]
If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the task parameters that were used to produce the output file.

EXAMPLES

1. Apply NICER calibration and screening to data from observation 1706221428.

  nicerl2 indir=1706221428 clobber=YES 

The input directory is a NICER observation number 1706221428. The output files are placed in the observation directory under 1706221428/xti/event_cl/.

SEE ALSO

nicercal,

nimaketime,

niprefilter2,

nicermergeclean

LAST MODIFIED

Apr 2019