NAME

USAGE

DESCRIPTION

Xtend data, even after standard screening, may suffer from pixels that have anomalously high counts due to non-source events, most often due to flickering pixels or cosmic-ray echo events. These pixels, hereafter referred to as anomalous pixels, should not be included in data analysis. Their exclusion should be propagated to the exposure map task (xaexpmap), which in turn informs corrections to the effective area (using the xaarfgen task). The xtdpixclip task identifies Xtend anomalous pixels inside of user-specified regions referred to as inclusion regions. A pixel is flagged as anomalous when the number of time-integrated counts is above a user-specified threshold for a given region. Only pixels inside of these inclusion regions are considered. Multiple regions may be specified, each with a corresponding count threshold. This task allows a user to, for example, specify one threshold for a source region and different thresholds for other, source-free regions. The task is typically run twice: once with 'pmode=histo' (this aids the user in selecting the thresholds by constructing distributions of number of pixels vs. counts per pixel), and again with 'pmode=apply', which applies the thresholds. After all anomalous pixels are flagged, the task generates a .fpix file which contains the flagged events in the EVENTS extension, and the list of flagged pixels, along with their coordinates, in the PIXELS extension. The .fpix file can be used as input to the xaexpmap task, and the .expo file output from the latter task is input to the xaarfgen task. The xtdpixclip task also produces a "cleaned" event file that has all of the events from the anomalous pixels removed, and it is this event file that is used for subsequent science or other data analysis. The reason for anomalous numbers of counts in a pixel is not relevant for operation of this task, because selection of the thresholds is empirically based on the counts per pixel distribution. In many cases there will be a clear change in the shape of the distribution above a certain value of counts per pixel that deviates from that due to a celestial source. However, this may not always be the case, and the user must consider the trade-off between accepting some anomalous pixels vs. accepting some false positives. In such cases, an iterative approach can be pursued.

When xtdpixclip is operated with 'pmode=histo', the pixel groups inside the specified regions are sorted into uniform intervals of counts per pixel, producing a histogram of number of pixels vs. counts per pixel interval for each region. A FITS file is created with a name based on the 'outroot' parameter; e.g. if 'outroot=xa300036010xtd_p031100010_cl', the histogram file would be named "xa300036010xtd_p031100010_cl_xpc_hist.fits" (where xpc stands for xtdpixclip). This file contains an extension named HISTOGRAM, with columns for the lower boundary, upper boundary, and center of each counts-per-pixel bin (BINLO, BINHI, and BINCEN respectively). The columns REGION01, REGION02, etc. contain the arrays of number of pixels vs. counts per pixel interval. The number in these column names corresponds to the order of the inclusion region files as entered for the 'incregionfiles' parameter. For example, if 'incregionfiles=file1.reg,file2.reg' then column REGION01 will correspond to file1.reg and column REGION02 will correspond to file2.reg. The header of this extension contains keywords corresponding to each region column name whose value is the corresponding region file name. The data in this histogram file can be examined directly to determine a count rate threshold. Alternatively, plots of the region columns may be examined to determine a threshold visually. Simple versions of these plots are generated by this task when 'pmode=histo', using fplot. These plots are created as .gif files and are also named based on the 'outroot' parameter; e.g. if 'outroot=xa300036010xtd_p031100010_cl', the histogram plot for the first region would be named xa300036010xtd_p031100010_cl_xpc_reg01_hist.gif.

After determining an appropriate threshold for each region from running xtdpixclip with 'pmode=histo', the task should be run with 'pmode=apply'. This mode applies the thresholds to the pixels in each specified region, flagging anomalous pixels. Each event belonging to a flagged pixel ends up in the EVENTS extension of the output .fpix file. The .fpix file also has a PIXELS extension that lists all of the anomalous pixels by their DET coordinates. The .fpix file is named based on the 'outroot' parameter; e.g. 'outroot=xa300036010xtd_p031100010_cl' would produce a .fpix file named "xa300036010xtd_p031100010_cl_xpc_rmvpix.fpix" (where xpc stands for xtdpixclip). Additionally, this mode creates a cleaned event file if 'mkclean=yes'. This file is the complement of the .fpix file and is the list of events with all of the flagged events removed. This file is also named based on the 'outroot' parameter. For example, 'outroot=xa300036010xtd_p031100010_cl' would produce a cleaned events file called "xa300036010xtd_p031100010_cl_xpc_clnevt.fits".

The 'incregionfiles' and 'excregionfiles' parameters can also take a default value of "NONE". When using 'incregionfiles=NONE', a single, internally-generated inclusion region that covers the whole FoV of the image will be applied. If 'excregionfiles=NONE' then no exclusion regions will be applied at all. There is an exclusion region file provided in the $HEADAS/refdata directory called calsrc_XTD_det.reg. This file excludes the Xtend calibration sources from contributing to histograms and to threshold selections (but the calibration source events are passed onto the cleaned event file). Users may consider using the calibration sources exclusion region file if it is appropriate for their needs, but it will not be used unless specified.

Both modes ('pmode=histo' or 'pmode=apply') make use of inclusion, and optionally, exclusion region files. These files have a standard SAO region file format, and are made from images in DET coordinates, and saved as "physical" or "image" in ds9. Complex shapes can be constructed by using exclusion shapes/regions in an inclusion file. However an exclusion region is used, it must contain a "dash" as the first character in every line that contains an exclusion shape or region. In "apply" mode the number of inclusion region files must match the number of threshold values in the 'thresholds' parameter. Each input region file may contain multiple SAO regions. In the case of an inclusion region file having multiple regions, the threshold that corresponds to that file will be used for all regions in that file. Additionally, complicated regions that make use of multiple inclusion and exclusion regions in a single file may be used. Pixels that are inside of exclusion regions are completely ignored by the tool. This means that all events inside of an exclusion region will be listed in the cleaned events list if one is generated.

The user may specify a PI channel range by means of the energy bounds parameters 'emin' and 'emax' (in units of keV), for the lower and upper boundary respectively. Events outside of this range are not counted in either the histogram or the pixel flagging algorithm, but are not completely removed from the possible outputs. The cleaned event file will have no events from anomalous pixels, regardless of whether or not the event's energy was within the energy band. If an event does not occur in an anomalous pixel it will be included in the cleaned event file, even if its energy is outside of the energy band. Likewise, all events that do occur in anomalous pixels will be included in the flagged pixel file, regardless of whether or not their energy falls between 'emin' and 'emax'. If a value for 'emax' that is larger than the maximum value, 24.575 keV, is used, the value will be clamped to this maximum.

Similarly, the user may specify a time intervals file (with the parameter 'timeintfile') for narrowing down searches for anamalous pixels based on making histograms from events that only occur within the time intervals in this file. The FITS format of the time intervals file is the same as a regular GTI file. Like the energy band filtering, events that occur outside of the time intervals in the file specified by 'timeintfile' are not all removed from the possible outputs. An event will be in the cleaned event file if it is not in an anomalous pixel, even if the event is outside of all of the time intervals. Likewise, an event in an anomalous pixel will be in the .fpix file even if it was within one of the time intervals. If 'timeintfile' is not set to NONE, a version of the input event file filtered by the time intervals file will be output, with the name 'outroot'_timeintflt.fits.

The user may also specify a GTI file using the 'gtifile' parameter to filter the input event file. Unlike the 'timeintfile' parameter, events outside of the GTI are rejected completely. They are therefore not included in either the histogram or the pixel flagging algorithm, and the rejected events will not appear in either the .fpix file or the cleaned event file, as they are completely filtered out before those files are generated.

Note that the overlaps between the regions in different inclusion region files should be avoided if possible. If regions in two or more files overlap, the threshold for the first of the overlapping files, as ordered in 'incregionfiles', will be applied in the overlapping region or regions. Overlapping inclusion regions in the same region file get the same threshold. Exclusion regions in the 'excregionfiles' list take precedence over inclusion regions if an exclusion region in that list overlaps with an inclusion region. However, the treatment of exclusion regions overlapping with inclusion regions in the 'incregionfiles' list is more complex. In that case, an exclusion region over-rides all inclusion regions that precede it in the list.

Further details on xtdpixclip usage can be found in the xtdpixclip user guide.

PARAMETERS

evtinfile [filename]

Name of the input Xtend FITS event list. This would most commonly be the screened event file from the pipeline output.

outroot [string]

Root name for output files. Various suffixes are added to 'outroot' to create file names, based on the type of output file. Histogram data files are named 'outroot'_xpc_hist.fits. Histogram plots are named 'outroot'_xpc_reg01_hist.gif, 'outroot'_xpc_reg02_hist.gif, where 'regNN' (NN=01 to 99) refers to the region file identifier in the list in 'incregionfiles', in the order that the files are listed. The flagged pixel file name is 'outroot'_xpc_rmvpix.fpix, and the cleaned event file is 'outroot'_xpc_clnevt.fits.

pmode [string histo|apply]

Mode for the task to run. If 'pmode=histo', histograms are generated of the number of pixels vs. counts per pixel for each region file provided by the 'incregionfiles' parameter. These histograms can be used to determine appropriate threshold values for flagging pixels as anomalous. If 'pmode=apply', the thresholds specified by the user will be used to flag pixels as anomalous. An .fpix file will be generated containing all of the flagged events, as well as a list of the flagged pixels, and their coordinates.

(emin = 0.0) [double]

Minimum energy [keV] an event must have to be considered for contributing to a histogram or to the cumulative counts per pixel that will be tested against the clipping thresholds. Events that are not from flagged pixels that have an energy below this value will still be saved to the cleaned events file if one is generated.

(emax = 12.0) [double]

Maximum energy [keV] an event may have to be considered for contributing to a histogram or to the cumulative counts per pixel that will be tested against the clipping thresholds. Events that are not from flagged pixels that have an energy above this value will be still be saved to the cleaned events file if one is generated. There are 4096 PI channels with each channel having a width of 0.006 keV. Therefore, the maximum allowed value of 'emax' is 24.575 keV. If a value larger than this is entered, the maximum will automatically be used instead.

(incregionfiles [filename NONE|file name])

Name (or list of names) of input inclusion region files. Pixels falling inside these regions will be considered for both histogram generation and the pixel clipping algorithm, depending on which mode of operation the task is run in. Events outside of these regions are ignored and will not be flagged as anomalous, therefore they will be passed on to the cleaned events file if one is generated. The region files must have standard SAO region file format, and are made from images in DET coordinates, but saved in ds9 as "physical" or "image". There may be multiple regions per file, but the number of inclusion region files and number of thresholds must match. If 'incregionfiles=NONE', then a single inclusion region that covers the whole FoV of the detector will be internally generated and applied.

(excregionfiles [filename NONE|file name])

Name (or list of names) of input exclusion region files. Pixels falling inside these regions will not be considered for making histograms ('pmode=histo'), nor for applying threshold cuts ('pmode=apply'). Events falling inside of these regions will not be marked as anomalous and therefore will be passed on to the cleaned events file if one is generated. The region files must have standard SAO region file format, and are made from images in DET coordinates, but saved in ds9 as "physical" or " image". There may be multiple regions per file. If 'excregionfiles=NONE', then no exclusion regions will be applied.

(thresholds = 1000) [string]

List of threshold values of counts per pixel for each inclusion region. There must be one threshold value for each inclusion region file. If multiple regions are included in a single file, the threshold corresponding to that file will be applied to all regions in that file. The values may be separated by spaces or commas.

(nhistbins = -1) [integer]

Number of bins in the generated histograms. If set to -1 the number of bins will be equal to the maximum number of counts. This means that there will be one histogram bin per count increment.

(gtifile = NONE) [filename NONE|file name]

The name of the input GTI file for filtering the input event file, 'evtinfile'. If a file name is provided, the filtered event file will also be saved and its named is based on the 'outroot' parameter; e.g. if 'outroot=xa300036010xtd_p031100010_cl', the filtered event file will be named 'xa300036010xtd_p031100010_cl_xpc_filtered.fits'. Events that do not lie inside any of the GTI in 'gtifile' will not be considered for making histograms nor for applying thresholds, and will not be passed on to the output cleaned event file if one is made. If 'gtifile=NONE' then all events in the input file are used.

(timeintfile = NONE [filename NONE|file name])

The name of an input GTI file for time-selected searches of anomalous pixels. Events outside of any of these GTI will not be considered for making histograms, nor for applying thresholds. Unlike events filtered out by the 'gtifile' parameter however, events filtered out by 'timeintfile' will be passed on to the cleaned event file if they did not occur in anomalous pixels.

(mkclean = yes) [boolean yes|no]

This parameter determines whether or not a cleaned events list is produced. This file does not include any events that occur in anomalous pixels, and is the complement of extension 1 of the .fpix file.

(buffer = -1) [integer -1|0|N]

Rows to buffer (-1=auto, 0=none, N>0=number of rows).

(clobber = no) [boolean yes|no]

Overwrites the existing output file if set to yes.

(chatter = 1) [integer 0|1|2|3]

Chatter level for output. Set to 0 to suppress output, or to 1, 2, or 3 for increasing the chatter of the output.

(logfile = !DEFAULT) [string]

Log file name. If set to DEFAULT, uses the name of the task and, if preceded by "!", overwrites the file if it exists. If set to NONE, no log file is created.

(debug = no) [boolean yes|no]

Diagnostic output is printed to the screen if set to yes.

(history = yes) [boolean yes|no]

Records task parameters in HISTORY.

EXAMPLES

1. Run xtdpixclip in 'histo' mode with three inclusion region files with one or more inclusion regions in each, and a text file list of exclusion region files (exc_files.txt). This will produce the histogram file 'clipped_xpc_hist.fits' that contains the columns REGION01, REGION02, REGION03 corresponding to the inclusion region files box1.reg, circle1.reg, and box2.reg respectively. These columns will contain the number of pixels vs. the counts per pixel interval, which will also be plotted in the output files clipped_xpc_reg01_hist.gif, clipped_xpc_reg02_hist.gif, and clipped_xpc_reg03_hist.gif.

  xtdpixclip evtinfile=xa100050020xtd_p0100004b1_cl.evt outroot=clipped pmode=histo \
  incregionfiles="box1.reg,circle1.reg,box2.reg" excregionfiles="@exc_files.txt"

2. Run xtdpixclip in 'apply' mode with the same three inclusion region files and list of exclusion regions as in the previous example, with three thresholds that are determined by examining the results from the previous run 'pmode=histo'. A cleaned event list will also be generated because 'mkclean=yes' is set. This will produce two output files: clipped_xpc_rmvpix.fpix and clipped_xpc_clnevt.fits.

  xtdpixclip evtinfile=xa100050020xtd_p0100004b1_cl.evt outroot=clipped pmode=apply \
  incregionfiles="box1.reg,box2.reg,circle1.reg" excregionfiles="@exc_files.txt" \
  thresholds="100.0 1250.0 200.0" mkclean=yes

3. Generate a histogram for a single region around the source, only counting events within a specific energy range from 3 keV to 24.575 keV, and GTI from a separate GTI file named times.gti. Use no exclusion region files.

  xtdpixclip evtinfile=xa100050020xtd_p0100004b1_cl.evt output=clipped pmode=histo \
  incregionfiles="source.reg" emin=3 emax=24.575 timeintfile=times.gti

4. Apply the threshold selected from the histogram output by the previous example. Use the same energy and time filtering.

  xtdpixclip evtinfile=xa100050020xtd_p0100004b1_cl.evt output=clipped pmode=apply \
  incregionfiles="source.reg" thresholds="300" emin=3 emax=24.575 \
  timeintfile=times.gti

5. Generate a histogram for a single region covering the entire FoV, completely filtering out events outside of the time intervals specified by "xa100050020_gen.gti".

  xtdpixclip evtinfile=xa100050020xtd_p0100004b1_cl.evt output=clipped pmode=histo \
  gtifile=xa100050020_gen.gti

6. Apply the threshold selected from the histogram output by the previous example. Use the same GTI filtering. The events outside of the specified GTI will not appear in either the .fpix file, or the cleaned event file.

  xtdpixclip evtinfile=xa100050020xtd_p0100004b1_cl.evt output=clipped pmode=apply \
  gtifile=xa100050020_gen.gti thresholds="100"

SEE ALSO

xaexpmap, searchflickpix

LAST MODIFIED

July 20, 2026