niprefilter2 - create augmented NICER-specific filter file


niprefilter2 indir infile outfile ...


The niprefilter2 task produces an augmented NICER-specific filter file. It transforms an existing "Level1" filter file produced by niprefilter and adds additional columns and information to make it a "Level2" file. This additional information is more useful for filtering and generating background estimates.

The input to this task is an existing "Level1" filter file produced by niprefilter, as well as an existing NICER observation directory. The output is a modified "Level2" filter file.

The task will make numerous temporary files in the output directory. Users should have adequate free disk space to accomodate a significant number of temporary files. The temporary files are deleted during the normal course of the task, unless cleanup=NO.

It is possible to modify the filter file "in place" by specifying the same names for infile and outfile, or outfile="INFILE". In reality, a temporary copy of a new filter file is created, and then moved into place upon successful completion of the task. Thus, if the niprefilter2 operation fails at an intermediate step, the original file will not be modified.

The NICER team does not recommend to delete the original Level1 filter file, because it is not possible to run niprefilter2 upon the augmented Level2 file. Therefore is possible to retain the original filter file, with append=YES. This is the default operation. In this case niprefilter2 will append the original filter file Level1 extension to the output, with extension name ORIG_PREFILTER. At a later point in time, it is possible to run niprefilter2 on this output, and the task will automatically seek to the original Level1 extension and ignore the Level2 extension.


In addition to the "base" or default set of computed columns, users can select from a few recipes of additional columns. These recipes are useful for task-specific work.

By default, the task-specific columns are not computed, but users can select them using the coltypes parameter. The coltypes parameter is a comma-separated list of classes of columns. The default value, "base", indicates to compute the standard default columns (and in fact cannot be de-selected). To add more task-specific columns, turn this into a comma-separated list. For example,

would select both the base columns and the "3c50" specific columns. Currently the supported column types are: See below for more information about the task-specific column sets. In addition to the column sets listed above, coltypes may also contain items of the form "COLNAME" to ensure that the column "COLNAME" is included in the output, or "-COLNAME" to ensure that the column "COLNAME" is excluded from the output. Thus, the expression
would begin with the base set of default columns, but exclude ISS_ATT_STATE and be sure to include FPM_DEADTIME.


This is a summary of outputs produced by niprefilter2. Please note that the first group of columns is produced by niprefilter, and are documented here again for completeness (with heading ORIGINAL). The second group of columns is produced by niprefilter2 and have heading NEW.

IMPORTANT NOTE: column order may not be preserved in the output. Please do not depend on specific column numbers or orders of columns in the output file. The columns listed below are not given in exact column order.

Definition of "COUNT" Values

Below there are various columns with "COUNT" in their names. These are the number of events of the designated type, in the standard filter file sample interval (typically 1 second). Since the typical sample interval is 1 second, these counts are also effectively rates.


The first columns are produced by the mission-generic task prefilter which creates pointing and orbit-related quantities.
  Col  Name             Format[Units](Range)      Comment
    1 TIME               1D [s]               seconds since mission epoch
    2 POSITION           3E [km]              ECI position of satellite [X,Y,Z]
    3 VELOCITY           3E [km/s]            ECI velocity of satellite [X,Y,Z]
    4 QUATERNION         4E                   attitude quaternion
    5 PNTUNIT            3E                   pointing unit vector
    6 POLAR              3E [km, rad, rad]    geodetic radius lat lon
    7 RA                 1E [deg]             pointing axis right ascension
    8 DEC                1E [deg]             pointing axis declination
    9 ROLL               1E [deg]             pointing axis roll
   10 SAT_LAT            1E [deg]             sub-satellite point latitude
   11 SAT_LON            1E [deg]             sub-satellite point longitude
   12 SAT_ALT            1E [km]              distance from earth center
   13 ELV                1E [deg]             angle between pointing and earth limb
   14 BR_EARTH           1E [deg]             angle between pointing and bright earth
   15 SUNSHINE           1I                   1=in sunshine; 0=not
   16 FOV_FLAG           1I                   0=sky; 1=dark earth; 2=bright earth
   17 SUN_ANGLE          1E [deg]             angle between pointing and sun vector
   18 MOON_ANGLE         1E [deg]             angle between pointing and moon vector
   19 RAM_ANGLE          1E [deg]             angle between pointing and velocity vector
   20 ANG_DIST           1E [deg]             angular distance of pointing from nominal
   21 SAA                1I                   1=in SAA; 0=not
   22 SAA_TIME           1E [s]               time since entering/exiting SAA
   23 COR_ASCA           1E [GeV/c]           magnetic cut off rigidity (ASCA map)
   24 COR_SAX            1E [GeV/c]           magnetic cut off rigidity (IGRF map)
   25 MCILWAIN_L         1E                   McIlwain L parameter (SAX)
   26 SUN_RA             1E [deg]             RA of sun (equatorial)
   27 SUN_DEC            1E [deg]             Dec of sun (equatorial)
   28 MOON_RA            1E [deg]             RA of moon (equatorial)
   29 MOON_DEC           1E [deg]             Dec of moon (equatorial)
   30 EARTH_RA           1E [deg]             RA of earth (equatorial)
   31 EARTH_DEC          1E [deg]             Dec of earth (equatorial)
   32 TIME_ADJ           1D [s]               adjusted seconds since mission epoch

The RA,DEC columns indicate the pointing of NICER's estimated average boresight (the mean boresight found by averaging over all modules). The ELV column is the elevation of the pointing direction above the earth's limb, while the BR_EARTH column indicates the angle from the bright earth limb. In orbit night, BR_EARTH is 180 degrees.

The SUNSHINE flag indicates the presence of orbit data at the location of NICER (not daylight at the ground track on the earth's surface).

The ANG_DIST value indicates the angular separation between the target, specified by the 'ranom' and 'decnom' command line parameters, and the actual pointing direction. If ranom=0 and decnom=0, this quantity will be unreliable.

The SAA column indicates if NICER is in the South Atlantic Anomaly (SAA) as defined by prefilter. prefilter has a predefined model of the SAA which is mission-generic and not optimized for NICER. See NICER_SAA below. The SAA_TIME column indicates time since passage through the prefilter SAA boundary.

The COR_ASCA and COR_SAX columns indicate the magnetic cut-off rigidity (magnetic field) level. The COR_SAX value is to be trusted to a greater degree, because it is based on the International Geomagnetic Reference Field (magnetic field model) of the earth and NICER's orbit. The COR_ASCA model is based on an early map produced by the ASCA spacecraft and may not be applicable to NICER. The MCILWAIN_L column is the McIlwain L parameter derived for NICER's orbit using the IGRF.


    Name             Format[Units](Range)      Comment
   NICER_SAA          1B                   NICER-specific SAA definition

The NICER_SAA is a custom definition of SAA for the NICER payload on the ISS. It is produced from NICER data. The region file for this contour is stored in CALDB, but can be specified from the command line (via the saaregfile parameter).


The measurement power unit (MPU) produces individualized housekeeping data within housekeeping files, originally stored in the obs/xti/hk/ subdirectory. These files are specified using the mpuhkfiles command line parameter. niprefilter2 takes these files and creates vector columns so that all MPU statistics for certain telemetry points are available for screening.
    Name             Format[Units](Range)      Comment
   MPU_ALL_COUNT      56I                  Per-FPM ALL counts
   MPU_OVER_COUNT     56I                  Per-FPM OVER counts
   MPU_UNDER_COUNT    56I                  Per-FPM UNDER counts
   MPU_XRAY_COUNT     56I                  Per-FPM XRAY counts
   TOT_ALL_COUNT      1J                   Array total ALL counts
   TOT_UNDER_COUNT    1J                   Array total UNDER counts
   TOT_OVER_COUNT     1J                   Array total OVER counts
   TOT_XRAY_COUNT     1J                   Array total XRAY counts
   FPM_ON             56L                  Is FPM enabled for science output?
   NUM_FPM_ON         1J                   Number of FPMs enabled for science

The MPU_ALL_COUNT, MPU_OVER_COUNT, MPU_UNDER_COUNT and MPU_XRAY_COUNT represent count rates that the MPU measures for each module on a 1 second basis. The XRAY count represents the counts tagged by the on-board electronics as an X-ray, although it may not be an X-ray if it is below-threshold noise or background. The OVER and UNDER counts are over-shoot rates (pulse height exceeds maximum value) and under-shoot rates (detector resets). The ALL counts are the total of all counts registered by the MPU.

These columns are stored as a 2D array within each cell, so that the counts from each module can be accessed. These arrays are 1-based, meaning they start with 1 instead of 0. Therefore you add 1 to the 0-based MPU and FPM values to access the array. For example, to access module DET_ID=41, which is MPU slice 4 and FPM channel 1, add one to each value. In this case, use MPU_*_COUNT[5][2], where 5=4+1 and 2=1+1.

The TOT_ALL_COUNT, TOT_UNDER_COUNT, TOT_OVER_COUNT and TOT_XRAY_COUNT columns are array-wide totals of the corresponding individual counts.

The FPM_ON column indicates if an individual FPM is enabled for science. Again, this is a two dimensional array, so MPU4, FPM1 would be accessed as FPM_ON[5][2].

The NUM_FPM_ON column has the total number of enabled modules. The maximum value is 56 (all modules enabled); and the minimum value is 0 (all modules disabled).


    Name             Format[Units](Range)      Comment
   ST_BBO             1B                   Star tracker big bright obj flag
   ST_VALID           1B                   Star tracker valid flag
   ST_OBJECTS         1B                   Star tracker num. objects
   ST_VIDEO_VDC       1B [V]               Star tracker video voltage
   ST_STARS           1B                   Star tracker num. stars
   ST_FAILCODE        1B                   Star tracker failure code

The ST_VALID flag indicates if the star tracker camera solution is valid (1) or not (0). Note that the overall attitude solution in the attitude file incorporates other non-camera information and may be valid even if the star tracker solution is not valid.

The ST_BBO column indicates the presence of a Big Bright Object (BBO) in the star tracker field. The ST_OBJECTS and ST_STARS columns indicate the number of objects/stars in the star tracker field. If no stars are detected, a failure code ST_FAILCODE indicates the reason. The ST_VIDEO_VDC column can be useful to diagnose star tracker imaging issues.


    Name             Format[Units](Range)      Comment
   ATT_ANG_AZ         1D [deg]             Pointing azimuth angle
   ATT_ANG_EL         1D [deg]             Pointing elevation angle
   ATT_ERR_AZ         1D [deg]             Pointing error in azimuth angle
   ATT_ERR_EL         1D [deg]             Pointing error in elevation angle
   RA_CMD             1D [deg]             Commanded right ascension = J2000
   DEC_CMD            1D [deg]             Commanded declination = J2000
   TARG_CMD           1I                   Pointing commanded target ID
   ATT_STATE          1B                   Pointing control state 1=OPS
   ATT_MODE           1B                   Pointing control mode 1=SCIENCE
   ATT_SUBMODE_AZ     1B                   Pointing control azimuth track mode 2=TRACK
   ATT_SUBMODE_EL     1B                   Pointing control elevation track mode 2=TRACK
   XTI_PNT_JITTER     1E [arcsec]          Instrument 1-second pointing jitter (NICERV3)
   ANG_DIST_X         1E [arcsec]          Target position in instrument X coordinates
   ANG_DIST_Y         1E [arcsec]          Target position in instrument X coordinates

The ATT_ANG_AZ and ATT_ANG_EL columns represent the gimbal pointing angles in azimuth and elevation, respectively. The ATT_ERR_AZ and ATT_ERR_EL columns represent the control error in those values relative to the commanded value. The commanded right and ascension are reported in the RA_CMD and DEC_CMD columns, as well as the commanded target in TARG_CMD.

The XTI_PNT_JITTER column, added with NICERV3, contains the instrument pointing jitter with 1-second smoothing constant. The value is expressed in arcsec. Calculation of this value requires the presence of the original NICER attitude file. When the target is far out of the field of view (more than 90 degrees), or if the attitude file is not present, a NULL value is stored. Note that access to the NICER mission alignment "teldef" is required (which defaults to CALDB).

The ANG_DIST_{X,Y} columns, added with NICERV3, contains the target coordinates in instrument boresight coordinates. Note that access to the NICER mission alignment "teldef" is required (which defaults to CALDB).


    Name             Format[Units](Range)      Comment
    PPS_SOURCE         1J                   Source of time sync 0=SPS,1=GEONS
    PPS_ERR_LOWPASS    1D [s]               Filtered PPS error
    GPS_INIT           1B                   GEONS filter state initialized?
    GPS_CONVERGED      1B                   GEONS filter state converged?

The NICER GPS solution can be used in two fashions. The "raw" GPS solution, also known as Single Point Solution, and a filtered solution, also known as GEONS. The PPS_SOURCE column indicates which solution is being used for timestamps within the NICER system. The PPS_ERR_LOWPASS column indicates a measure of how much error is in the GPS solution, measured in seconds. This value is low-pass filtered to average several samples.

The GPS_INIT and GPS_CONVERGED columns indicate the state of the GEONS GPS filter solution, whether it is intialized and converged, respectively.


    Name             Format[Units](Range)      Comment
    FPM_XRAY_PI_0000_0025 1E                   X-ray rate 0.000 - 0.250 keV
    FPM_XRAY_PI_0025_0200 1E                   X-ray rate 0.250 - 2.000 keV
    FPM_XRAY_PI_0200_0800 1E                   X-ray rate 2.000 - 8.000 keV
    FPM_XRAY_PI_0800_1200 1E                   X-ray rate 8.000 - 12.000 keV
    FPM_XRAY_PI_1200_1500 1E                   X-ray rate 12.000 - 15.000 keV
    FPM_XRAY_PI_1500_1700 1E                   X-ray rate 15.000 - 17.000 keV
    MPU_XRAY_PI_0030_0070 56I                  X-ray rate 0.3 - 0.7 keV (NICERV4)

These are X-ray rates derived from the event files after screening out most non X-rays. They are different from the housekeeping rates because they are formed directly from the events with additional screening and energy cuts. The six columns represent rates in six energy bands as described above. They are the average rate per enabled FPM, with no attempt at background subtraction.

The MPU_XRAY_PI_0030_0070 column (created with NICERV4 and higher), is the 0.3-0.7 keV count rate for each FPM individually. This can be used for screening out detectors with an extended noise peak.


    Name             Format[Units](Range)      Comment
   MPU_DOUBLE_COUNT   56J                  MPU over+under reset count from events
   MPU_OVERONLY_COUNT 56J                  MPU over-only reset count from events
   MPU_UNDERONLY_COUNT 56J                 MPU under-only reset count from events
   MEDIAN_UNDERONLY_COUNT 1I               Median value of MPU_UNDERONLY_COUNT (added NICERV3)
   FPM_DOUBLE_COUNT   1E                   Per-FPM over+under reset count from events
   FPM_OVERONLY_COUNT 1E                   Per-FPM over-only reset count from events
   FPM_UNDERONLY_COUNT 1E                  Per-FPM under-only reset count from events
   MPU_NOISERING_COUNT 1E                  MPU noise ringer count (added NICERV5)
   FPM_NOISERING_COUNT 1E                  Per-FPM noise ringer count (added NICERV5)

These are are average non X-ray rates derived from the event files. They are different from the housekeeping rates because the event file allows additional control in separating different kinds of events.

The MPU_* columns are 2-dimensional arrays, representing the rate in each module for each second.

MPU_OVERONLY_COUNT is the rate of overshoot-only resets for each module, derived from the event files. Unlike the housekeeping rate, this rate contains only events with the overshoot only flag set, and thus is a better proxy of background.

MPU_UNDERONLY_COUNT is the rate of undershoot-only resets for each module, derived from the event files. This rate should be similar to the housekeeping rate.

The MEDIAN_UNDERONLY_COUNT column (added in NICERV3) is the median number of undershoots, where the median is taken over all 56 detectors, excluding detectors with zero undershoots. The median process may help during data screening since it is less susceptible to outliers compared to FPM_UNDERONLY_COUNT.

MPU_DOUBLE_COUNT are the rate of events with both the overshoot and undershoot flag set (so-called double-shoot events) for each module, derived from the event files. In principle MPU_OVERONLY_COUNT+MPU_DOUBLE_COUNT should be comparable to the MPU_OVER_COUNT rate from pure housekeeping.

The FPM_{OVERONLY,UNDERONLY_DOUBLE}_COUNT rates represent the average count rate of {over,under,double} counts per enabled FPM.

MPU_NOISERING_COUNT and FPM_NOISERING_COUNT provide the number of "noise ringer" counts. These are counts of events that occur within 110 usec of an undershoot, that help diagnose spectral noise. The the MPU_* variant is a 56-vector, one for each FPM, and the FPM_* variant is the average count rate per enabled FPM.

    Name             Format[Units](Range)      Comment
    FPM_RATIO_REJ_COUNT 1E                   Total PI_RATIO-rejected count
    MPU_FT_COUNT       56I                  MPU forced trigger count from events
    MPU_NOISE25_COUNT  56I                  MPU PI noise rate <0.25 keV
    FPM_FT_COUNT       1E                   Per-FPM forced trigger count from events
    FPM_NOISE25_COUNT  1E                   Per-FPM noise count <0.25 keV from events

As noted above, the MPU_* columns are 2-dimensional arrays, representing the rate in each module for each second.

MPU_FT_COUNT is the rate of forced trigger events for each module, derived from the event files. Under nominal configurations, this should be a rate of 5 forced trigger events per second per module, although the rate may be less if detectors are disabled.

MPU_NOISE25_COUNT is the rate of "noise" events for each module, derived from the event files. In reality it is the rate of events with have a PI value between 5 and 25 (50-250 eV), which is primarily noise events. However, it may be contaminated by source events as well.

The FPM_{FT,NOISE25}_COUNT rates represent the average count rate of {forced trigger,noise} events per enabled FPM.

The FPM_RATIO_REJ_COUNT is the rate of events that would be selected by the simple cut (PI > 300 && PI_RATIO > 1.53). These are nominally background which would be rejected by the so-called trumpet cut by nicerclean. This is the average count rate of ratio-rejected events per enabled FPM.


    Name             Format[Units](Range)      Comment
    MPU_FT_PI_AVG      56E [chan]           MPU forced trigger PI centroid
    MPU_FT_PI_ERR      56E [chan]           MPU forced trigger PI rms
    MPU_FT_PI_FAST_AVG 56E [chan]           MPU forced trigger PI_FAST centroid
    MPU_FT_PI_FAST_ERR 56E [chan]           MPU forced trigger PI_FAST rms
    MPU_NOISE25_PI_AVG 56E [chan]           MPU noise PI centroid < 0.25 keV
    MPU_NOISE25_PI_ERR 56E [chan]           MPU noise PI rms < 0.25 keV
    MPU_DEADTIME       7D  [s]              Per-MPU deadtime
    FPM_DEADTIME       56E [s]              Per-FPM deadtime (added by NICERV3)

These columns contain spectral diagnostics for NICER. Each of these is an 56-element array (8x7), sampled once per second. All columns (except for FPM_DEADTIME) are selected when NICERV2 or NICERV3 are set, but not NICERV4 or higher.

MPU_FT_PI_AVG is the centroid of the forced trigger peak of the slow (PI) channel for each module, derived from event files. Forced triggers represent a signal-free pulse-height and should be centered at an energy value of 0.0 eV. Because of the sampling of the PI variable, the forced trigger of the slow PI channel should be centered at -0.5. The MPU_FT_PI_ERR is the standard deviation of the measured forced triggers in that one second sample. If no forced triggers are received, both values will be NULL.

The MPU_FT_PI_FAST_{AVG,ERR} columns are the same as the MPU_FT_PI_{AVG,ERR} columns described above, for the PI_FAST (fast) channel.

MPU_NOISE25_PI_{AVG,ERR} are the centroid and standard deviation of the NOISE25 values (X-ray events between 50 and 250 eV).

The MPU_DEADTIME is the total recorded dead-time for all events, for each MPU. It is a seven-element vector, one for each MPU, and contains the amount of recorded deadtime in seconds for each, for all event types before any screening. The FPM_DEADTIME column (added with NICERV3) records the per-FPM deadtime, also in seconds.


   Name             Format[Units](Range)      Comment
   MPU_LOWMEM_FIFO_DELTA 7J             Per-MPU LOWMEM_FIFO Changes (added NICERV3)
   MPU_LOWMEM_SCI_DELTA  7J             Per-MPU LOWMEM_SCI Changes (added NICERV3)
   TOT_LOWMEM_FIFO       J              Array-total LOWMEM_FIFO counts (added NICERV5)
   TOT_LOWMEM_SCI        J              Array-total LOWMEM_SCI  counts (added NICERV5)
   HV_ON                 56L            Is FPM high voltage on? (added NICERV3)
   FPM_SLOW_LLD          56E            Slow channel LLD setting (added NICERV3)

These columns provide additional MPU housekeeping information beyond the original prefilter file. These columns were all added with NICERV3.

The MPU_LOWMEM_FIFO_DELTA column provides information about lost data processed by an MPU. It contains the number of increments of the LOWMEM_FIFO counter in the given one second interval. The MPU_LOWMEM_SCI_DELTA records the number of increments of the LOWMEM_SCI counter. An increment of either of these quantities may indicate degradation of data (loss of data due to transmission outages, etc). LOWMEM_FIFO indicates a loss of events before packetization. LOWMEM_SCI indicates a loss of packetized events.

Note that as of niprefilter2 v2.9, these columns are now 32-bit integers instead of 8-bit integers.

Added in with NICERV5, the TOT_LOWMEM_FIFO and TOT_LOWMEM_SCI columns provide the number of LOWMEM_FIFO and LOWMEM_SCI events, totaled over the entire array.

The HV_ON column indicates whether the FPM high voltage bias is enabled or not. There is one logical vector item for each detetector (56-vector). The voltage threshold for a value of 'T'rue is -115 Volts (typical is -125 volts).

The FPM_SLOW_LLD columns provides the slow channel low energy threshold (LLD) value. This value is the housekeeping readback, in Volts, of the on-board LLD setting.


   Name             Format[Units](Range)      Comment
   ISS_ATT_STATE      4A                   ISS attitude state
   ROBO_STATE         1B                   ISS robotics state

These columns capture information about the ISS attitude or pointing, as well as the ISS ROBO (robotics) activity.

The International Space Station (ISS), which hosts NICER, nominally "flies" in its orbit. This attitude is known as "+XVV" or "LVLH," referring to the ISS +X axis being in the same direction as the velocity vector. However, during some activities, the ISS attitude may change. Also, reboost maneuvers occur occasionally. These changes may disrupt NICER observations and/or disturb NICER pointing control systems beyond their typical limits. The ISS_ATT_STATE records some basic information to help reconstruct the ISS state during an observation.

The ISS_ATT_STATE column provides information about the International Space Station (ISS) attitude state at each sample interval. This basic information is recorded in the 'issmanfile' file, which is nominally accessed via CALDB. The state information is not 100% complete; only major changes are recorded. In addition, the information may not be available immediately in NICER CALDB. Scientific users need to monitor NICER CALDB for updates to get the most recent information.

The changes captured in ISS_ATT_STATE are:

An additional column, ROBO_STATE, captures information about ISS ROBO (robotics) activities that may be occuring around NICER. When ROBO activities occur, the robotic appendages may block the NICER field of view and lead to target occultations. When this condition is set, it indicates that obstructions may occur but not necessarily that obstructions will always occur. Only this general information is available to warn observers in case of unexpected obstructions.

The values captured in ROBO_STATE are:


   Name             Format[Units](Range)      Comment
   VEHICLE_SOYUZ_DC1  1B                   Vehicle SOYUZ present at port DC1?
   VEHICLE_SOYUZ_MRM1 1B                   Vehicle SOYUZ present at port MRM1?
   VEHICLE_SOYUZ_MRM2 1B                   Vehicle SOYUZ present at port MRM2?
   VEHICLE_SOYUZ_SM   1B                   Vehicle SOYUZ present at port SM?

The NICER background may depend on the configuration of the International Space Station (ISS). In particular, the Soyuz supply vehicles contain a radioactive source which may create additional background for NICER. Soyouz vehicles may dock at various docking ports.

In order to document the presence of vehicles, and where they are docked, the above columns are created. The columns are named,

where {vehicle} is the vehicle name (currently just SOYUZ) and {port} is the docking port at which the vehicle was berthed. A value of 1 indicates the vehicle was berthed, and a value of 0 indicates no vehicle was documented as berthed.

At present, the NICER team documents the presence of SOYUZ vehicles at docking ports DC-1 ("DC1"), MRM-1 ("MRM1"), MRM-2 ("MRM2") and SM-Aft ("SM"). However, the vehicle types and docking port locations are subject to change in the future.

To retrieve the default information, use vehiclefile="CALDB".


   Name             Format[Units](Range)      Comment
   NICER_SAA          1B                   NICER-specific SAA definition
   LATLONREG          1B                   inside lat/lon region? 1=yes

Users may wish to explore the data based on geographic position. bSince this can be difficult, niprefilter2 provides some shorthand capability for geographic calculations. The result is an additional user-designated column which has either 1 or 0 depending on the geographic filter. Users can specify an alternate NICER_SAA definition, which replaces the existing NICER_SAA column, and also their own custom region definition for any desired purpose.

The contour should be the form of a region file, and on the command line as saaregfile=filename.reg or latlonregfile=filename.reg. The region file should in the format of a standard spatial region file, and specify the contour of good or bad data. The region file should specify longitude in the range SAT_LON=-180 to SAT_LON=+180; this is done to allow easy filtering of the South Atlantic Anomaly (SAA), which crosses SAT_LON==0.

Users can also use standard regions that are located in NICER CALDB. Currently there are two named regions in CALDB:

Specify a CALDB contour using latlonregfile=CALDB:SAACONT or latlonregfile=CALDB:SAACONTLARGE1.

For saaregfile, the results of the evaluation replace the existing NICER_SAA column created by niprefilter. The results of the latlonregfile evaluation are placed in a user-designated column. By default, this column will be LATLONREG, but users can set a different column name with the latlonregcolname parameter.

If the sense of the latlonregfile region file should be inverted to keep good data, then use latlonreginvert=YES. Here are the possible results:

For example, to select points outside the enlarged SAA region filtering, use the following options:

  niprefilter2 ... latlonregfile=CALDB:SAACONTLARGE1 latlonreginvert=YES \ 
Upon completion, the column MYREG will be 1 outside the enlarged SAA and 0 inside the the enlarged SAA.

TASK-SPECIFIC: NICER Background Model 3C50

This is a set of task-specific columns for use with the NICER "3C50" model. See "SELECTING TASK-SPECIFIC COLUMNS" above to understand how this type of output is selected. The name to use with coltypes is "3c50"

When this column type is activated, the following columns are created.

   Name             Format[Units](Range)      Comment
   MPU_TRUMP_SEL_1500_1800 56E   NICER Trumpet-selected ct (15-18keV)
   MPU_RATIO_REJ_300_1800  56E   NICER PI_RATIO-rejected ct (3-18keV)
   MPU_NOISE20_COUNT       56I   Per-FPM noise rate in 0.0-0.2 keV band
The MPU_RATIO_SEL_1500_1800 column represents the number of counts in the 15-18 keV band which survive the "trumpet" cut (see nicerclean for more information). This is also known in the 3C50 model as the "IBG" term. The MPU_RATIO_REJ_300_1800 column represents the number of counts in the 3-18 keV band which are rejected by the PI_RATIO > 1.53 cut. This is also known in the 3C50 model as the "HREJ" term. The MPU_NOISE20_COUNT column was added in niprefilter2 version 1.17 (March 2021), and is the noise count rate in all 56 detectors in the 0.0-0.2 keV band, as defined by the 3C50 model. Only enabled if NICERV3 also set.

TASK-SPECIFIC: Geomagnetic Columns

niprefilter2 can attach geomagnetic-related columns to a filter file. These quantities can include the so called planetary Kp magnetic disturbance index among other quantities.

Adding these columns to a filter file requires access to external data with these values. niprefilter2 does not provide direct access to such data, but it does facilitate access to existing data stores. Going into too much detail about this topic is beyond the scope of the niprefilter2 help file. To find out more on this topic, please consult the NICER Geomagnetic Quantities web page.

A recommended way to run niprefilter2 to obtain useful geomagnetic quantities is,

  geomag_path=/your/geomag_path \
The final column, COR_NYM, is derived with the cornymmik task, is an adjusted cut-off rigidity value which responds to different geomagnetic conditions. As input, KP is required to be present in order to run cornymmik.

When using the above settings, the following columns result.

   Name             Format[Units](Range)      Comment
   KP               1E           Potsdam planetary Kp index
   SOLAR_PHI        1E [GV]      Solar modulation potential
   COR_NYM          1E [GeV/c]   Adjusted COR (Nymmik et al)


indir [string]
Name of NICER observation directory. The directory must be arranged as a typical NICER observation directory with auxil/, xti/event_uf and xti/event_cl directories.
infile [filename]
Name of input "Level1" filter file produced by niprefilter.
outfile [filename]
Name of output filter file, or "INFILE". A new file named outfile will be created with the contents of the filter file described above.
(append="YES") [boolean]
If YES, then the original Level1 filter file extension will be appended to the output.
(saaregfile="CALDB") [string]
The name of the NICER South Atlantic Anomaly (SAA) region file, which will be used to replace the existint NICER_SAA column. Use "CALDB" to query CALDB. Use "NONE" to keep the existing value unchanged.
(vehiclefile="CALDB") [string]
Retrieve International Space Station vehicle docking information as a function of time. If set to "NONE" then this is disabled. If set to "CALDB" or "CALDB:vehiclename", then use CALDB information for the specified vehicle (default is "SOYUZ").
(issmanfile="CALDB") [string]
Retrieve International Space Station maneuver information as a function of time. If set to "NONE" then this is disabled. If set to "CALDB" then use CALDB information.
(robofile="CALDB") [string]
Retrieve robotic "robo" activity information as a function of time. If set to "NONE" then this is disabled. If set to "CALDB" then use CALDB information.
(alignfile="CALDB") [string]
Retrieve NICER boresight alignment calibration file. If set to "NONE" then columns that require boresight information is disabled. If set to "CALDB" then use CALDB information.
(latlonregfile="NONE") [string]
Calculate an additional geographic region value as described above. Use "NONE" to skip this step, or "CALDB:name" to query CALDB for region named "name".
(latlonregcolname="LATLONREG") [string]
Column name to use for geographic region value.
(latlonreginvert=NO) [boolean]
If YES, then invert the sense of the region computation.
(geomag_columns="NONE") [string]
Name of files containing geomagnetic quantities. More than one file can be specified using a comma-separated list. For each file on the list, the geomag_path is prepended. Use the notation "filename(COLNAME)" to specify that the output column should be named COLNAME. A value of NONE means no extra geomagnetic quantities.
(geomag_path="DEFAULT") [string]
Name of the base path for all geomagnetic files. This path is prepended to each item in the geomag_columns list. If DEFAULT, then the task default of geomagterp is used.
(geomag_tcheck="YES") [boolean]
If set, then perform rigorous time checking when interpolating geomagnetic quantities. When "NO" then no checking is performed and stale geomagnetic tables are allowed.
(coltypes="base,NICERV5") [string]
A comma-separated list of column types or column names to be included using (COLNAME) or excluded (using -COLNAME) as described above. The default "base" columns cannot be deselected even if not listed. See above for the allowed values.
(cleanup="YES") [boolean]
If yes, then clean up temporary files. If no, temporary files remain. This is typically for debugging.
(clobber = NO) [boolean]
If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.
(chatter = 1) [integer, 0 - 5]
Controls the amount of informative text written to standard output. Setting chatter = 4 or higher will produce detailed diagnostic output; chatter = 1 prints out a basic diagnostic message. The default is to produce a brief summary on output.
(history = YES) [boolean]
If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the niprefilter2 task parameters that were used to produce the output file.


This is a typical invocation. It is assumed the user is processing observation 1234567890, which is stored in directory ./1234567890/

  niprefilter2 indir=./1234567890 infile=./1234567890/auxil/ni1234567890.mkf \
    outfile=./1234567890/auxil/ni1234567890.mkf clobber=YES

In this case, the output file is called 1234567890/auxil/ni1234567890.mkf2. The clobber=YES parameter means that the output file can be overwritten. The original PREFILTER extension will be preserved in the output file as the second extension, and the new "enhanced" extension will be placed as the first extension.




NICER Geomagnetic Quantities



Sep 2022