NAME

FTEXTRACTOR -- Extracts products from input events files (C++ rewrite of extractor)

USAGE

       ftextractor <filename> [<eventsout> [<imgfile> [<binf> [<fullimage>
               [<phafile> [<specbin> [<wtmapb> [<wtmapfix> [<swmapx> [<swmapy>
               [<binh> [<wmapver> [<fitsbinlc> [<binlc> [<lcthresh>
               [<lcthwarn> [<lcstart> [<lctzero> [<unbinlc> [<regionfile>
               [<timefile> [<adjustgti> [<gtinam> [<xcolf> [<ycolf> [<zcolf> [<xint>
               [<yint> [<tcol> [<ecol> [<stokes> [<polwcol> [<ccol> [<gcol>
               [<gstring> [<xcolh> [<ycolh> [<gtitxt> [<xronoutfile> [<events>
               [<gti> [<timeorder> [<timeref> [<tpixrpar> [<usetpixrsel>
               [<eventkey> [<phamax> [<xfkey> [<yfkey> [<xhkey> [<yhkey>
               [<copyall> [<ignoredskeys> [<clobber> [<maxgti> [<maxccd>
               [<stride>
               ]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

       ftextractor filename=<filename> [eventsout=<eventsout>] [imgfile=<imgfile>]
               [binf=<binf>] [fullimage=<fullimage>] [phafile=<phafile>]
               [specbin=<specbin>] [wtmapb=<wtmapb>] [wtmapfix=<wtmapfix>]
               [swmapx=<swmapx>] [swmapy=<swmapy>] [binh=<binh>] [wmapver=<wmapver>]
               [fitsbinlc=<fitsbinlc>] [binlc=<binlc>] [lcthresh=<lcthresh>]
               [lcthwarn=<lcthwarn>] [lcstart=<lcstart>] [lctzero=<lctzero>]
               [unbinlc=<unbinlc>] [regionfile=<regionfile>] [timefile=<timefile>]
               [adjustgti=<adjustgti>] [gtinam=<gtinam>] [xcolf=<xcolf>]
               [ycolf=<ycolf>] [zcolf=<zcolf>] [xint=<xint>] [yint=<yint>]
               [tcol=<tcol>] [ecol=<ecol>] [stokes=<stokes>] [polwcol=<polwcol>]
               [ccol=<ccol>] [gcol=<gcol>] [gstring=<gstring>] [xcolh=<xcolh>]
               [ycolh=<ycolh>] [gtitxt=<gtitxt>] [xronoutfile=<xronoutfile>]
               [events=<events>] [gti=<gti>] [timeorder=<timeorder>]
               [timeref=<timeref>] [tpixrpar=<tpixrpar>] [usetpixrsel=<usetpixrsel>]
               [eventkey=<eventkey>] [phamax=<phamax>] [xfkey=<xfkey>]
               [yfkey=<yfkey>] [xhkey=<xhkey>] [yhkey=<yhkey>] [copyall=<copyall>]
               [ignoredskeys=<ignoredskeys>] [clobber=<clobber>]
               [maxgti=<maxgti>] [maxccd=<maxccd>] [stride=<stride>]

DESCRIPTION

FTEXTRACTOR produces event, image, spectrum, and/or lightcurve files from a set of input event files. The input events can be filtered on selected regions, time intervals, phase bins, or within some interval based on any integer event attribute (such as PHA or PI). FTEXTRACTOR is the modern C++ replacement for the Fortran EXTRACTOR task and is intended to be called by XSELECT in place of EXTRACTOR.

Spatial filtering is performed using region files. These are ascii files containing one region per line. If a region is to be excluded then it is prefixed with a minus sign. If the first region is an exclusion region then the entire image is first included. The valid regions are POINT(X,Y), CIRCLE(X,Y,Radius), ELLIPSE(X,Y,Radius_1,Radius_2,Rotation), ANNULUS(X,Y, Inner_radius,Outer_radius), (ROT)BOX(X,Y,Side_1,Side_2,Rotation), RECTANGLE(X_min,Y_min,X_max,Y_max), POLYGON(X_vertex_1,Y_vertex_1,...,X_vertex_N,Y_vertex_N), PIE(X,Y,Radius_1,Radius_2,Angle_1,Angle_2). A region extension describing the selection is appended to the output file.

Time filtering is performed using good time intervals. The good time intervals from all the input files are ORed together and then these are ANDed with the good time intervals from the input event files. For detectors such as CCDs which provide a time for each frame the time selection is performed based on the center time of the frame. It is best to use GTIs which match exactly to complete frames (ie the start of the GTI is the start time of some frame and the end of the GTI is the end time of a frame). If this is not possible then the adjustgti boolean parameter can be used to set the output GTIs so this criterion is met.

The grade (or pattern) can be filtered using a combination of single grades (eg 0), ranges (eg 2-6), upper limits (eg < 3), and lower limits (eg > 4). Multiple specifications should be separated by commas (eg 0,2).

If the zcolf parameter is set then each pixel in the output image will be the mean value of that column for all events in the pixel. The two axes of the image can be any two columns of the event list. If an axis is for a real event attribute then the xint or yint parameter should be set to define the image pixel size.

For spectropolarimetric data the stokes parameter selects whether the Stokes I/Q/U products are written. When stokes != NONE the output PHA and lightcurve filenames are split into per-component files (the root name with I, Q, and U appended) and each is written in count-rate form with a STAT_ERR column.

The stokescov parameter controls which inter-spectrum covariances are written so that XSPEC can fit the Stokes spectra jointly with the chistokes/chicov statistic instead of treating them as independent. With stokescov=QU (the default) the Q and U PHA files carry a per-bin Cov(Q,U) column (XCOV), a sibling-pointer keyword (XCOVSIB) and the marker COVFRMT='XCOV-PAIR-1'. With stokescov=IQU the full I/Q/U covariance group is written in the upper-triangle XCOV-SET-1 format: the I file carries XCOV2 (Cov(I,Q)) and XCOV3 (Cov(I,U)), the Q file carries XCOV3 (Cov(Q,U)), and all three carry the group keywords COVGRP (group id), COVIDX (1-based member index) and NCOVMEM (member count) with COVFRMT='XCOV-SET-1'. With stokescov=NONE no covariance columns are written. The covariances use the same NEFF/Kislat normalization as the Stokes products; under xselect the group id is rewritten to the saved output basename at save time. The grouping must be applied identically to all members (group the I spectrum and copy its GROUPING to Q and U; ftgrouppha sums the XCOV columns), and the chistokes/chicov statistic range must cover every member used.

When extracting event files only the events and GTI extensions are written out unless the copyall parameter is set to true. If copyall=yes then the extra extensions will be copied from the input file to the end of the output file. If multiple input files are specified then only the extra extensions from the first file will be copied.

The data subspace (DS) keywords carried on the input events extension describe the cumulative filtering history of the file. By default ftextractor applies these as a runtime filter on the input events; set ignoredskeys=yes to retain the historical EXTRACTOR behaviour, in which the DS keys are written through to the output as documentation only.

DIFFERENCES FROM EXTRACTOR

FTEXTRACTOR is a from-scratch C++ rewrite that aims for ftdiff parity with EXTRACTOR on the keywords and data that matter scientifically. The user-visible interface differs in three places:

• The deprecated qdpfile parameter has been removed. Use fitsbinlc for the binned light curve.
• The XRONOS window output parameter has been renamed from xronwn to xronoutfile.
• A new ignoredskeys parameter controls whether DS keywords on the input events HDU are applied as a runtime filter (default) or treated purely as documentation (set ignoredskeys=yes for the historical EXTRACTOR behaviour).
• regionfile accepts an "@list.txt" form to fan out per region; phasewndfile accepts the same form to fan out per phase window. See MULTI-AXIS MODE below.

MULTI-AXIS MODE

ftextractor supports producing one output spectrum (and matched WMAP) per cell of an N x M x K grid in a single pass, where the axes are:

• region: regionfile=@list.txt names N region files; one spectrum per region.
• phase: phasewndfile=@list.wnd names M XRONOS phase-window files; one spectrum per window-set. A single literal phasewndfile keeps the existing single-phase semantics (one window-set OR'd internally per the XRONOS spec). Passing a .wnd file via the legacy timefile parameter still works and triggers a one-shot deprecation warning that points the user at phasewndfile.
• time: timesplit=yes plus a non-empty timefile fans the timefile content. With timefile=foo.gti having K rows, each row becomes one cell. With timefile=@list.gti naming K files, each file becomes one cell (rows within a file are OR'd into that cell). With timesplit=no (default) the timefile content AND-merges into the per-chip event-file GTIs as before, and the time axis is inactive.

The three axes can be freely combined, producing N*M*K cells. All other selection criteria (grade, key-range, DS) are shared across cells; only the per-event region, phase, and time predicates vary by cell.

The phafile parameter accepts three shapes in multi-axis mode:

• An "@list.txt" naming exactly N*M*K output PHA paths in the fixed dimension order region (fastest), then phase, then time.
• A template using named placeholders %R / %P / %T such as "spec_%R_%P_%T.pha". Each placeholder must correspond to an axis that is active and multi (size > 1); a placeholder for an inactive axis is an error and a missing placeholder for an active multi axis is also an error.
• A literal path such as "multi.pha", which produces a single Type II PHA file holding all N*M*K spectra as rows of one SPECTRUM extension. WMAPs are not supported in this case (Type II PHA has no WMAP convention); set wtmapb=no, or use one of the first two shapes.

The wmap output (when wtmapb=yes) is paired one-to-one with the per- cell spectra, since the WMAP encodes the spatial distribution of events that fed each spectrum and is consumed by RMF construction.

Per-cell metadata. Each Type I PHA carries PHASE_LO / PHASE_HI / PHASEPOC / PHASEPER / PHWNDFIL keywords identifying the phase window-set, plus TIMEFILE / TIMEIDX identifying the time-axis cell. TSTART / TSTOP / TELAPSE keywords reflect the cell's narrowed merged GTI; per-chip GTI extensions in the cell's PHA are intersected with the cell time GTI. EXPOSURE / ONTIME / LIVETIME are phase-corrected and time-narrowed for that cell. Region metadata flows through the standard XFLT0001 / REGFILE / REGION extension keys as in single- region mode.

The image output (when imgfile is set) stays single in multi-axis mode (one global image of the unfiltered-by-region/phase/time event distribution). Likewise gtitxt and xronoutfile remain single.

Multi-axis mode does not yet support fan-out for fitsbinlc, unbinlc, eventsout, or Stokes processing; setting any of these together with an "@list" regionfile, phasewndfile, or timesplit=yes is an error at parameter-read time.

At end of run, the chain counter table is unchanged in shape (so single-axis parity is preserved). Below the Grand row, diagnostic lines report events that landed outside every region, every phase window, and / or every time interval, plus (only when nonzero) events that landed in more than one region, in multiple phase-window files, or in multiple time intervals. The product summary block prints a per-cell indented entry for the spectrum and WMAP totals, labelled by the cell's region file basename and / or phase / time axis index.

PARAMETERS

filename [string]

The input event filename. If this starts with "@" then it is assumed to be an ascii file containing a list of event files to be read. Interval filters can be specified by appending them inside [] on the end of this parameter. For instance, to accept only events with PHA lying between 7 and 30 the filename will be specified by "infile.evt[pha=7:30]". If a list of filenames is given (using "@") then the interval filtering should be specified on the input parameter, not on the individual event filenames. Successive interval filters should be separated by spaces or commas.

eventsout [string]

The name for any output events file.

imgfile [string]

The name for any output image file. The image is constructed from the event attributes given by the ftextractor parameters xcolf and ycolf.

binf [int]

The factor by which the event coordinates will be binned to make the image.

fullimage [bool]

If yes then output image is full-size. If no then the output image is the bounding box around the selected region.

phafile [string]

The name for any output spectrum file. The spectrum is constructed from the event attribute given by the ftextractor parameter ecol. If the wtmapb parameter is set to true then a wmap will be written as the primary array in the output spectrum file. This map is constructed from the event attributes given by the ftextractor parameters xcolh and ycolh binned by a factor binh. In multi-axis mode (regionfile=@list, phasewndfile=@list, or both), phafile may be an "@list" of N*M output paths, a template using %R / %P named placeholders, or a literal path that triggers Type II PHA output. See the MULTI-AXIS MODE section above.

specbin [int]

Binning factor for the spectrum.

wtmapb [bool]

If yes then a WMAP will be written as the primary array in phafile. This WMAP is constructed from the event attributes given by the ftextractor parameters xcolh and ycolh binned by a factor binh.

wtmapfix [bool]

If yes then all WMAP pixels outside the selected region are set to -1. If no then they are set to 0.

swmapx [bool]

If yes then invert the X-axis when converting WMAP to IMAGE coordinates.

swmapy [bool]

If yes then invert the Y-axis when converting WMAP to IMAGE coordinates.

binh [int]

The factor by which the event coordinates will be binned to make the WMAP.

wmapver [int]

If 1 then the old-style HDUVERS=1 WMAP will be written. If 2 then the newer HDUVERS=2 format will be used (this version is consistent with CIAO).

fitsbinlc [string]

The name for any FITS format binned light curve. The binsize is given by the parameter binlc.

binlc [double]

The binsize used to construct light curves.

lcthresh [double]

When constructing a lightcurve any bins whose exposure is less than lcthresh times the binsize are ignored. This removes bins at the edges of the good time intervals.

lcthwarn [double]

If a lightcurve bin is ignored due to lcthresh and the number of counts in the bin exceeds lcthwarn times the average then a warning is written.

lcstart [double]

The start time of the lightcurve in the same units as the GTI.

lctzero [bool]

If yes then the lightcurve times will be relative to that of the start bin. If no then the lightcurve times will be in spacecraft units.

unbinlc [string]

The name for any output unbinned light curve. This is simply an ascii list of events with time and the contents of ecol for each event.

regionfile [string]

Any region file used for spatial filtering. The region coordinates are assumed to be xcolf and ycolf binned by binf. If the value starts with "@" it is interpreted as an ascii file naming N region files (one per line), which activates multi-region fan and produces one spectrum per region in a single pass; see the MULTI-AXIS MODE section above.

phasewndfile [string]

XRONOS phase-window file (.wnd) for phase-folded selection. A literal path applies one window-set in the chain (single-phase mode, identical to passing the same .wnd via timefile in earlier releases). If the value starts with "@" it is interpreted as an ascii file naming M .wnd files (one per line), which activates the multi-phase fan and produces one spectrum per window-set in a single pass; see the MULTI-AXIS MODE section above. Passing a .wnd file via timefile still works in single-phase mode but emits a one-shot deprecation warning that points the user at this parameter.

timesplit [bool]

If yes, fan the timefile content into separate output spectra (multi-time mode); see the MULTI-AXIS MODE section above. With timefile=foo.gti each row of the GTI becomes one output cell; with timefile=@list.gti each listed file becomes one cell. timesplit=yes requires timefile to be set. Default no, in which case timefile content AND-merges into the per-chip event-file GTIs as before.

timefile [string]

Any file used for time filtering. This can be either a single time filter file or an ascii file listing other time filter files. In the latter case the filename should be prefixed by "@". Time filter files can be either FITS files containing GTI extensions, Xronos window files, or ascii files containing a column of start and stop times in units of the tcol column. Note that for CCD detectors time filters should ideally be lined up with readouts ie a time filter should start at the beginning of a frame and end at the end of a frame.

adjustgti [bool]

If yes then the output GTIs will be adjusted to match to frame boundaries (useful for detectors such as CCDs with large TIMEDEL values).

gtinam [string]

If a FITS file is used for time filtering then gtinam specifies the name of the extension containing the list of good time intervals.

xcolf [string]

The name of the column used for the X coordinate when region filtering and constructing the output image.

ycolf [string]

The name of the column used for the Y coordinate when region filtering and constructing the output image.

zcolf [string]

The name of the column used for constructing the output image. The value in each pixel of the output image will be the mean of the zcolf column for all events in the pixel.

xint [double]

Digitization used for the X-axis of an image. This will define the pixel size in the output image. Should be used when making an image using real event attributes for the axes.

yint [double]

Digitization used for the Y-axis of an image. This will define the pixel size in the output image. Should be used when making an image using real event attributes for the axes.

tcol [string]

The name of the column used for the time of the event.

ecol [string]

The name of the column used for constructing the output spectrum.

stokes [string]

For spectropolarimetric data choose whether to output the Stokes parameter products. If NONE then do not, if NEFF or SIMPLE then use the weighting column defined by polwcol, and if UNWEIGHTED then do not use the weighting column. In this last case the I Stokes parameter product is the same as that if NONE had been chosen. NEFF is the Baldini et al. 2203.06385 weighting scheme and SIMPLE is that from Kislat et al. 1409.6214.

(stokescov = QU) [string]

Which inter-spectrum covariances to write with the Stokes products, for joint fitting in XSPEC with the chistokes/chicov statistic. QU (the default) writes only the per-bin Cov(Q,U) as a pair (XCOV column plus XCOVSIB keyword on the Q and U files, COVFRMT='XCOV-PAIR-1'). IQU writes the full I/Q/U covariance group in the upper-triangle XCOV-SET-1 format (XCOV2/XCOV3 on I, XCOV3 on Q, with COVGRP/COVIDX/NCOVMEM on all three); the I-Q and I-U covariances are second order in the polarization fraction and usually negligible, so IQU is mainly for completeness. NONE writes no covariance columns. Ignored unless stokes != NONE.

polwcol [string]

The name of the column containing the polarization weights.

ccol [string]

The name of the column used to index the GTIs. In most cases this is the CCD ID.

gcol [string]

The name of the column used for the GRADE or PATTERN.

gstring [string]

The filter string for the GRADE or PATTERN. The grade (or pattern) can be filtered using a combination of single grades (eg 0), ranges (eg 2-6), upper limits (eg < 3), and lower limits (eg > 4). Multiple specifications should be separated by commas (eg 0,2).

xcolh [string]

The name of the column used for the X coordinate when constructing the WMAP for the output spectrum.

ycolh [string]

The name of the column used for the Y coordinate when constructing the WMAP for the output spectrum.

gtitxt [string]

The name for any output ascii file listing the final good time intervals after all time filtering.

xronoutfile [string]

The name for any output xronos window file listing the final good time intervals after all time filtering. (This parameter was named xronwn in the Fortran extractor.)

events [string]

The name of the extension containing the events in the input event files.

gti [string]

The name of the extension containing the good time intervals in the input event files.

timeorder [bool]

If the input events are time-ordered then set true.

timeref [double]

TJD time 0 in MJD for the output files.

tpixrpar [double]

Value of TIMEPIXR to use if the keyword is not in the event file.

usetpixrsel [bool]

If true then use the TIMEPIXR correction when selecting events.

eventkey [string]

The name of a file listing all keywords that should not be copied from the input event file to the output event file.

phamax [string]

The keyword specifying the maximum for the ecol column. If a TLMAX keyword is present then phamax does not have to be set (and if set will be ignored).

xfkey [string]

The keyword specifying the maximum for the xcolf column. If a TLMAX keyword is present then xfkey does not have to be set (and if set will be ignored).

yfkey [string]

The keyword specifying the maximum for the ycolf column. If a TLMAX keyword is present then yfkey does not have to be set (and if set will be ignored).

xhkey [string]

The keyword specifying the maximum for the xcolh column. If a TLMAX keyword is present then xhkey does not have to be set (and if set will be ignored).

yhkey [string]

The keyword specifying the maximum for the ycolh column. If a TLMAX keyword is present then yhkey does not have to be set (and if set will be ignored).

copyall [bool]

If true then copy extra extensions from the input event file to the output event file.

ignoredskeys [bool]

If true then the data subspace (DS) keywords on the input events HDU are treated as documentation only and are not applied as a runtime filter on the input events. If false (the default) the DS keys are applied as a filter, so events that fall outside the cumulative DS selection are discarded. Set to yes to reproduce the historical EXTRACTOR behaviour.

clobber [bool]

If true then any existing output files will be overwritten.

maxgti [integer]

The maximum number of GTIs allowed.

maxccd [integer]

The maximum number of chips allowed.

stride [integer]

The number of rows read at a time from the event file.

CATEGORY