'sxiflagpix' flags SXI events that may be unsuitable to use in data analysis, such as those that fall on defective or non-sensitive pixels; those that might have missing or compromised telemetry; and those for which processing has encountered certain errors. The STATUS column is updated, encoding the flagging condition for each event. This tool does not actually filter or remove events, however the STATUS column can be used for later filtering. An image in DET coordinates showing the location of unsuitable pixels is also output for later use in constructing an ARF or flat field image.
The tool uses several input files to assign STATUS: a list of bad or dead pixels or columns that don't change ("badpixfile"); a list of hot pixels, which can change on short timescales and are observation-specific ("hotpixfile"); a list of flickering pixels, which can be produced by the task searchflickpix ("flickpixfile"); and a mask file that encodes the location of CCD boundaries, segments, calibration source regions, and other regions that don't change ("maskfile"). The locations of the charge injection rows are read from the input event list header, as are settings for window mode and area discrimination.
In addition to an output event list with updated STATUS column, sxiflagpix optionally outputs a bad pixel list (specified by the parameter 'outbadpix') containing the events that have any STATUS flag set by the tool. Also output is an image (specified by the parameter 'outbadimg') in SXI DET coordinates containing a value for each pixel corresponding to its STATUS. The values in these files are, in order of increasing precedence:
0: a good pixel
1: a pixel inside the calibration source region
2: a bad pixel set by the parameter "bad_status"
-1: a pixel out of the CCD, window, or area discrimination region
For example, a pixel inside the calibration region but flagged as bad has value 2 in the output image. Flickering pixels are not included in this map because they can depend on time. Note that the output bad pixel list contains only those pixels for which events are found in the input file, while the output image contains all pixels.
Below is a table mapping flag bit position in the STATUS column to particular flagging conditions. If the flag is set to 1, that condition is flagged. Unless otherwise noted, the STATUS values apply to the central pixel of an event. Tools other than sxiflagpix that set STATUS values are noted in parentheses. "CI" indicates "charge injection."
=========================================================================== flag description =========================================================================== 1 All bad events set by "bad_status" parameter 2 Inside the calibration source region --------------------------------------------------------------------------- 3 Out of CCD (Out of area) 4 Out of window 5 Out of area discrimination --------------------------------------------------------------------------- 6 CI row (Pixels) 7 Bad pixel from CALDB 8 Bad column from CALDB 9 Hot pixel from pre-pipeline 10 Flickering pixel --------------------------------------------------------------------------- 11 CCD boundary (Boundaries) 12 Window boundary 13 Segment boundary 14 Area discrimination boundary 15 At least one 3x3 surrounding pixel has a bad status --------------------------------------------------------------------------- 16 CI trailing row (Neighbors) 17 CI preceding row 18 Preceding/following of bad column 19 Neighbors of bad/hot pixel and bad column 20 Neighbors of flickering pixel 21 Neighbors of preceding/following of bad column 22 Neighbors of CCD/window boundary 23 Neighbors of segment boundary --------------------------------------------------------------------------- 24 (sxiphas) 3x3 info is present but 5x5 is absent (Others) 25 (sxiphas) 3x3 is absent 26 (sxipi - general) PHAS[0] < event threshold 27 (sxipi - vtevnodd) Video temperature is out of range 28 (sxipi - vtevnodd) Lack of video temp HK at time close to the event 29 (sxipi - chtrail/CTI) Correction value is negative 30 (sxipi - general) Null value by correction process --------------------------------------------------------------------------- 31 1st trailing row of the CI rows (Diagnostics) 32 1st preceding row of the CI rows 33 2nd trailing row of the CI rows 34 2nd preceding row of the CI rows 35 3rd trailing row of the CI rows 36 3rd preceding row of the CI rows --------------------------------------------------------------------------- 37 Cosmic ray echo pixel (Cosmic Rays) --------------------------------------------------------------------------- 38-48 Reserved ===========================================================================
The special STATUS flag 1 is set for any event that has a STATUS flag listed in the "bad_status" parameter. This is to enable easy filtering syntax later in the processing, since good events can be extracted with a simple filter expression.
Several STATUS conditions apply to events where neighboring pixels are flagged. Because an event is composed of a 3x3 pixel island, the quality of the region immediately surrounding the event center must be taken into account. For example, a bad pixel in the outer ring of a 3x3 island does not produce a valid pulse height, and this could result in a event with an otherwise unacceptable grade being accepted as a good event. The parameters 'npixnbr' and 'nboundnbr' can be used to set the maximum distance away from a bad pixel or boundary to flag further pixels and events. In addition, for more flexibility, sxiflagpix writes a new column PHAS_MASK to the output event list. This is a 9-element mask of the PHAS column, with each element indicating whether the corresponding pixel i in PHAS is good (PHAS_MASK[i]=0) or bad (PHAS_MASK[i]=1). PHAS_MASK can be used later in sxipi to determine how to grade the event and calculate the summed PHA, using the sxipi 'badpixopt' parameter. Since sxipi can overwrite the PHAS column, the ability to reset it to the original telemetry values is provided here with the 'copyphas' parameter. Similarly, 'resetflags' can be used to reset all the sxiflagpix STATUS flags and the elements of PHAS_MASK to zero. Using 'resetflags=yes' does not change the flags shown above that are set by sxipi or sxiphas.
Some pixels are affected by cosmic ray echo, in which a cosmic ray detected during an SXI dark frame can cause an echo in a different readout segment. This echo then causes those pixels to have a pulse height above the split threshold for the duration of the observation, leading to incorrect grades for events centered in neighboring pixels. Specifying 'echoflag=yes' causes 'sxiflagpix' to search for such pixels in all SXI segments except the aimpoint segment (CCD_ID==1, SEGMENT==1), which is free of this effect. If at least 'echofrac' of the events containing a given pixel have a pulse height greater than or equal to 'echospth' in that pixel, then the pixel is considered a cosmic ray echo pixel. Any events centered 'echonbr' pixels from such pixels have STATUS flag 37 set, and if flag 37 is specified in 'bad_status', any pixels within 'echonbr' of that pixel will be marked as bad in the 'outbadimg'. For SXI, 'echonbr=2' is the recommended value, which will flag a 5x5 area of pixels around any cosmic ray echo pixel. Only pixels that have data from at least 'echomin' events are considered. An image of the echo fraction can be written out as an extension to 'outbadimg' if 'echomap=yes', and used for diagnostic purposes.
Charge trailed or leaked from charge injection rows can produce spurious events in several rows immediately surrounding the injected rows, manifesting as noise at soft energies. The parameters 'citrailnbr' and 'ciprenbr' can be used to tune a compromise between lower noise and higher efficiency, since flagging additional rows reduces the effective area.
Some examples of how to use the STATUS flags in filtering and extracting events are shown below in the "Examples" section. Also see the help for 'calc_express' for proper filtering syntax.
sxiflagpix infile="input.evt" outfile="output.evt" outbadimg="outbadimg.fits" \
hotpixfile="hotpix.fits" flickpixfile="flickpix.fits"
sxiflagpix infile="input.evt" outfile="output.evt" outbadimg="outbadimg.fits" \
hotpixfile="hotpix.fits" flickpixfile=NONE badpixfile=NONE
sxiflagpix infile="input.evt" outfile="output.evt" outbadimg="outbadimg.fits" \
hotpixfile="hotpix.fits" flickpixfile="flickpix.fits" \
citrailnbr=4 ciprenbr=2
sxiflagpix infile="input.evt" outfile="output.evt" outbadimg="outbadimg.fits" \
hotpixfile="hotpix.fits" flickpixfile="flickpix.fits" \
bad_status="3:9,11:12,16"
ftcopy "output.evt[events][STATUS[1]==b0]" output_filtered.evt
ftcopy "output.evt[events][STATUS[1]==b0 && STATUS[2]==b1]" output_filtered.evtAnother way to do the same filtering, using the "bxxx" bit mask notation, which is explained fully under "calc_express". The wild card "x" indicates any value is allowed.
ftcopy \
"output.evt[events][STATUS==b01xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx]" \
output_filtered.evt
searchflickpix, sxiphas, sxipi, calc_express
November 2016