niprefilter - create NICER-specific filter file


niprefilter outfile mpuhkfiles orbfile attfile hkfile ...


The niprefilter task produces a NICER-specific filter file. A filter file is used for data screening based on relevant housekeeping or instrument related quantities collected in this file.

niprefilter is based upon the prefilter task which computes a generic filter file. The output of prefilter has many pointing and orbit related quantities which are useful for general filtering.

In addition, niprefilter collects quantities from various engineering housekeeping files into one place. These include the detector housekeeping as well as the payload housekeeping located within the master housekeeping file, which is typically in auxil/ni*.hk1.

niprefilter has some hidden parameters which it may be worth setting. The start and stop parameters specify the requested start and stop timestamps of the filter file. If you set these to the default (start=stop=0), niprefilter will use a heuristic to determine the start and stop. The ranom and decnom are the requested nominal pointing position. If they are set to zero, again niprefilter will use a heuristic to determine these quantities.


Users may desire to re-run the base tool prefilter again. This may be necessary or desireable, in order to capture improvements to prefilter. For example, prefilter may update its magnetic field model, or have additional useful columns added. Since prefilter creates some columns that are difficult to reproduce from public archive data (i.e. columns derived from internal housekeeping), re-running can capture improvements to prefilter while preserving these hard-to-compute NICER-specific columns.

To do this, use the task parameter runmode=UPDATE. The default runmode=CREATE will create a new file from scratch, while runmode=UPDATE will update an existing file by re-running prefilter.

The required parameters when using this mode are:

all other parameters can be set to their defaults.

To adjust the columns that prefilter produces, use the task parameter prefilter_columns. The default is "ALL", but it can be set to "ALLV4" or any combination as described in the prefilter documentation.

In additional, special shortcuts for NICER-specific columns can be made.

You may specify additional columns using a comma-separated list. So, for example, prefilter_columns="NICERV2,BEARING_ANGLE" would add BEARING_ANGLE to the above list, and prefilter_columns="NICERV2,-LOCAL_TIME" would remove LOCAL_TIME from the list. If you do use a shortcut in a comma-separated list, the shortcut must appear first.


This is a summary of outputs produced by niprefilter.

IMPORTANT NOTE: column order may not be preserved in the output. Please do not depend on specific column numbers or orders of columns in the output file. The columns listed below are not given in exact column order.


The first columns are produced by the mission-generic task prefilter which creates pointing and orbit-related quantities.
  Col  Name             Format[Units](Range)      Comment
    1 TIME               1D [s]               seconds since mission epoch
    2 POSITION           3E [km]              ECI position of satellite [X,Y,Z]
    3 VELOCITY           3E [km/s]            ECI velocity of satellite [X,Y,Z]
    4 QUATERNION         4E                   attitude quaternion
    5 PNTUNIT            3E                   pointing unit vector
    6 POLAR              3E [km, rad, rad]    geodetic radius lat lon
    7 RA                 1E [deg]             pointing axis right ascension
    8 DEC                1E [deg]             pointing axis declination
    9 ROLL               1E [deg]             pointing axis roll
   10 SAT_LAT            1E [deg]             sub-satellite point latitude
   11 SAT_LON            1E [deg]             sub-satellite point longitude
   12 SAT_ALT            1E [km]              distance from earth center
   13 ELV                1E [deg]             angle between pointing and earth limb
   14 BR_EARTH           1E [deg]             angle between pointing and bright earth
   15 SUNSHINE           1I                   1=in sunshine; 0=not
   16 FOV_FLAG           1I                   0=sky; 1=dark earth; 2=bright earth
   17 SUN_ANGLE          1E [deg]             angle between pointing and sun vector
   18 MOON_ANGLE         1E [deg]             angle between pointing and moon vector
   19 RAM_ANGLE          1E [deg]             angle between pointing and velocity vector
   20 ANG_DIST           1E [deg]             angular distance of pointing from nominal
   21 SAA                1I                   1=in SAA; 0=not
   22 SAA_TIME           1E [s]               time since entering/exiting SAA
   23 COR_ASCA           1E [GeV/c]           magnetic cut off rigidity (ASCA map)
   24 COR_SAX            1E [GeV/c]           magnetic cut off rigidity (IGRF map)
   25 MCILWAIN_L         1E                   McIlwain L parameter (SAX)
   26 SUN_RA             1E [deg]             RA of sun (equatorial)
   27 SUN_DEC            1E [deg]             Dec of sun (equatorial)
   28 MOON_RA            1E [deg]             RA of moon (equatorial)
   29 MOON_DEC           1E [deg]             Dec of moon (equatorial)
   30 EARTH_RA           1E [deg]             RA of earth (equatorial)
   31 EARTH_DEC          1E [deg]             Dec of earth (equatorial)
   32 TIME_ADJ           1D [s]               adjusted seconds since mission epoch

The RA,DEC columns indicate the pointing of NICER's estimated average boresight (the mean boresight found by averaging over all modules). The ELV column is the elevation of the pointing direction above the earth's limb, while the BR_EARTH column indicates the angle from the bright earth limb. In orbit night, BR_EARTH is 180 degrees.

The SUNSHINE flag indicates the presence of orbit data at the location of NICER (not daylight at the ground track on the earth's surface).

The ANG_DIST value indicates the angular separation between the target, specified by the 'ranom' and 'decnom' command line parameters, and the actual pointing direction. If ranom=0 and decnom=0, this quantity will be unreliable.

The SAA column indicates if NICER is in the South Atlantic Anomaly (SAA) as defined by prefilter. prefilter has a predefined model of the SAA which is mission-generic and not optimized for NICER. See NICER_SAA below. The SAA_TIME column indicates time since passage through the prefilter SAA boundary.

The COR_ASCA and COR_SAX columns indicate the magnetic cut-off rigidity (magnetic field) level. The COR_SAX value is to be trusted to a greater degree, because it is based on the International Geomagnetic Reference Field (magnetic field model) of the earth and NICER's orbit. The COR_ASCA model is based on an early map produced by the ASCA spacecraft and may not be applicable to NICER. The MCILWAIN_L column is the McIlwain L parameter derived for NICER's orbit using the IGRF.


       Name             Format[Units](Range)      Comment
      NICER_SAA          1B                   NICER-specific SAA definition

The NICER_SAA is a custom definition of SAA for the NICER payload on the ISS. It is produced from NICER data. The region file for this contour is stored in CALDB, but can be specified from the command line (via the saaregfile parameter).


The measurement power unit (MPU) produces individualized housekeeping data within housekeeping files, originally stored in the obs/xti/hk/ subdirectory. These files are specified using the mpuhkfiles command line parameter. niprefilter takes these files and creates vector columns so that all MPU statistics for certain telemetry points are available for screening.
    Name             Format[Units](Range)      Comment
   MPU_ALL_COUNT      56I                  Per-FPM ALL counts
   MPU_OVER_COUNT     56I                  Per-FPM OVER counts
   MPU_UNDER_COUNT    56I                  Per-FPM UNDER counts
   MPU_XRAY_COUNT     56I                  Per-FPM XRAY counts
   TOT_ALL_COUNT      1J                   Array total ALL counts
   TOT_UNDER_COUNT    1J                   Array total UNDER counts
   TOT_OVER_COUNT     1J                   Array total OVER counts
   TOT_XRAY_COUNT     1J                   Array total XRAY counts
   FPM_ON             56L                  Is FPM enabled for science output?
   NUM_FPM_ON         1J                   Number of FPMs enabled for science

The MPU_ALL_COUNT, MPU_OVER_COUNT, MPU_UNDER_COUNT and MPU_XRAY_COUNT represent count rates that the MPU measures for each module on a 1 second basis. The XRAY count represents the counts tagged by the on-board electronics as an X-ray, although it may not be an X-ray if it is below-threshold noise or background. The OVER and UNDER counts are over-shoot rates (pulse height exceeds maximum value) and under-shoot rates (detector resets). The ALL counts are the total of all counts registered by the MPU.

These columns are stored as a 2D array within each cell, so that the counts from each module can be accessed. These arrays are 1-based, meaning they start with 1 instead of 0. Therefore you add 1 to the 0-based MPU and FPM values to access the array. For example, to access module DET_ID=41, which is MPU slice 4 and FPM channel 1, add one to each value. In this case, use MPU_*_COUNT[5][2], where 5=4+1 and 2=1+1.

The TOT_ALL_COUNT, TOT_UNDER_COUNT, TOT_OVER_COUNT and TOT_XRAY_COUNT columns are array-wide totals of the corresponding individual counts.

The FPM_ON column indicates if an individual FPM is enabled for science. Again, this is a two dimensional array, so MPU4, FPM1 would be accessed as FPM_ON[5][2].

The NUM_FPM_ON column has the total number of enabled modules. The maximum value is 56 (all modules enabled); and the minimum value is 0 (all modules disabled).


    Name             Format[Units](Range)      Comment
   ST_BBO             1B                   Star tracker big bright obj flag
   ST_VALID           1B                   Star tracker valid flag
   ST_OBJECTS         1B                   Star tracker num. objects
   ST_VIDEO_VDC       1B [V]               Star tracker video voltage
   ST_STARS           1B                   Star tracker num. stars
   ST_FAILCODE        1B                   Star tracker failure code

The ST_VALID flag indicates if the star tracker camera solution is valid (1) or not (0). Note that the overall attitude solution in the attitude file incorporates other non-camera information and may be valid even if the star tracker solution is not valid.

The ST_BBO column indicates the presence of a Big Bright Object (BBO) in the star tracker field. The ST_OBJECTS and ST_STARS columns indicate the number of objects/stars in the star tracker field. If no stars are detected, a failure code ST_FAILCODE indicates the reason. The ST_VIDEO_VDC column can be useful to diagnose star tracker imaging issues.


    Name             Format[Units](Range)      Comment
   ATT_ANG_AZ         1D [deg]             Pointing azimuth angle
   ATT_ANG_EL         1D [deg]             Pointing elevation angle
   ATT_ERR_AZ         1D [deg]             Pointing error in azimuth angle
   ATT_ERR_EL         1D [deg]             Pointing error in elevation angle
   RA_CMD             1D [deg]             Commanded right ascension = J2000
   DEC_CMD            1D [deg]             Commanded declination = J2000
   TARG_CMD           1I                   Pointing commanded target ID
   ATT_STATE          1B                   Pointing control state 1=OPS
   ATT_MODE           1B                   Pointing control mode 1=SCIENCE
   ATT_SUBMODE_AZ     1B                   Pointing control azimuth track mode 2=TRACK
   ATT_SUBMODE_EL     1B                   Pointing control elevation track mode 2=TRACK

The ATT_ANG_AZ and ATT_ANG_EL columns represent the gimbal pointing angles in azimuth and elevation, respectively. The ATT_ERR_AZ and ATT_ERR_EL columns represent the control error in those values relative to the commanded value. The commanded right and ascension are reported in the RA_CMD and DEC_CMD columns, as well as the commanded target in TARG_CMD.


    Name             Format[Units](Range)      Comment
   PPS_SOURCE         1J                   Source of time sync 0=SPS,1=GEONS
   PPS_ERR_LOWPASS    1D [s]               Filtered PPS error
   GPS_INIT           1B                   GEONS filter state initialized?
   GPS_CONVERGED      1B                   GEONS filter state converged?

The NICER GPS solution can be used in two fashions. The "raw" GPS solution, also known as Single Point Solution, and a filtered solution, also known as GEONS. The PPS_SOURCE column indicates which solution is being used for timestamps within the NICER system. The PPS_ERR_LOWPASS column indicates a measure of how much error is in the GPS solution, measured in seconds. This value is low-pass filtered to average several samples.

The GPS_INIT and GPS_CONVERGED columns indicate the state of the GEONS GPS filter solution, whether it is intialized and converged, respectively.


outfile [filename]
Name of output filter file, or "INFILE". A new file named outfile will be created with the contents of the filter file described above.
mpuhkfiles [filename]
Name of NICER MPU housekeeping files, either as a comma-separated list, or an @filename.lis file list (which is recommended). All seven MPU housekeeping files should be specified.
orbfile [filename]
Name of NICER orbit file, as appears in obs/auxil/niNNNNNNNNNN.orb.
attfile [filename]
Name of NICER attitude file, as appears in obs/auxil/niNNNNNNNNNN.att.
hkfile [filename]
Name of NICER master engineering file, as appears in obs/auxil/niNNNNNNNNNN.hk1. This file contains multiple engineering housekeeping extensions used by the task to compute the above quantities.
(attmode="DEPLOYED") [string]
Processing mode for attitude data, either DEPLOYED or STOWED. It is not recommended to change this parameter.
(orbmode="EPHEM") [string]
Processing mode for orbit, either TLE_TEXT3 or EPHEM. It is not recommended to change this parameter.
(alignfile="CALDB") [string]
The name of the NICER boresight alignment teldef file, or "CALDB" to query CALDB.
(saaregfile="CALDB") [string]
The name of the NICER South Atlantic Anomaly (SAA) region file, or "CALDB" to query CALDB.
(start=0.0) [real]
The mission elapsed time of the start of the filter file. A value of zero (the default) indicates to use the start and stop times derived from the time coverage in the MPU housekeeping files.
(stop=0.0) [real]
The mission elapsed time of the stop of the filter file. A value of zero (the default) indicates to use the start and stop times derived from the time coverage in the MPU housekeeping files.
(interval=1) [real]
The time sample interval of the output filter file. Timestamps in the output file are spaced by interval.
(ranom=0) [real]
The nominal requested right ascension in J2000 degrees. The default value of zero will cause niprefilter to use a heuristic to compute this quantity.
(decnom=0) [real]
The nominal requested declination in J2000 degrees. The default value of zero will cause niprefilter to use a heuristic to compute this quantity.
(runmode="CREATE") [string]
Execution mode for the task. If runmode=CREATE then a filter file is created from scratch. If runmode=UPDATE then an existing filter file is updated. See above for more information.
(prefilter_columns="NICERV4") [string]
Comma- or space-separated list of columns for prefilter to produce, in addition to mission-specific aliases. See above for more information.
The MPU housekeeping columns to transfer to the filter file. It is not recommended to change this parameter.
(apid192file="HKFILE") [string]
The NICER housekeeping file corresponding to APID 192, or "HKFILE" if this APID is found in the master housekeeping file.
(apid201file="HKFILE") [string]
The NICER housekeeping file corresponding to APID 201, or "HKFILE" if this APID is found in the master housekeeping file.
(apid216file="HKFILE") [string]
The NICER housekeeping file corresponding to APID 216, or "HKFILE" if this APID is found in the master housekeeping file.
(apid262file="HKFILE") [string]
The NICER housekeeping file corresponding to APID 262, or "HKFILE" if this APID is found in the master housekeeping file.
(apid270file="HKFILE") [string]
The NICER housekeeping file corresponding to APID 270, or "HKFILE" if this APID is found in the master housekeeping file.
(missepoch=2014) [int]
The NICER mission epoch year, either 2014 or 2017. It is not recommended to change this value.
(cleanup="YES") [boolean]
If yes, then clean up temporary files. If no, temporary files remain. This is typically for debugging.
(clobber = NO) [boolean]
If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.
(chatter = 1) [integer, 0 - 5]
Controls the amount of informative text written to standard output. Setting chatter = 4 or higher will produce detailed diagnostic output; chatter = 1 prints out a basic diagnostic message. The default is to produce a brief summary on output.
(history = YES) [boolean]
If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the niprefilter task parameters that were used to produce the output file.


This is a typical invocation. It is assumed the caller has changed working diretory to the top of observation directory.

ls xti/hk/ni*mpu*.hk > @mpuhkfiles.lis
niprefilter niprefilter.mkf @mpuhkfiles.lis auxil/ni*.orb auxil/ni*.att auxil/ni*.hk1 
Note that this example uses Unix wildcard globbing to access the orbit file, attitude file and master housekeeping file. Normally you would specify these file names explicitly on the command line. The output is niprefilter.mkf.





Jan 2020