niobsmerge HEADAS help file

NAME

niobsmerge - merge NICER observation products

USAGE

niobsmerge infiles [outdir] ...

DESCRIPTION

The niobsmerge task merges observation products. This can be useful when combining the products of several observation segments of the same target that may span multiple days.

Please note that this task merges the output products of nicerl2, namely the "ufa" file, the cleaned event file, and the filter file. Also, optionally, the task will attempt to merge orbit files. It does not merge all the original Level 1 data files of an observation directory.

The input is one or more NICER observation product output directories. This is often a list of NICER observation directories, but it can also use whatever "cldir" was used to invoke nicerl2.

The list of input directories is given by the "indirs" parameter. Alternatively if indirs=NONE, then the task will expect an explicit list of file names for each file type, as described below.

The output directory is given by the "outdir" parameter, which defaults to the tasks current working directory.

Which files are used as inputs are governed by the inclfiles ("cl" files), inufafiles ("ufa" files), inmkfiles ("mkf" or filter file), and inorbfiles (orbit files) parameters. If indirs=NONE, then these should be a comma-separated list of files to merge, otherwise, then can be a pattern which describes how to find the files within each input directory.

The default pattern are designed to match most commonly used NICER processing outputs of nicerl2.

If no files of a given input type are found, that type is skipped.

The task can use some shorthand templates which allow the task to avoid many hardcoded path names. These templates of the form $NAME and can be used in most input or output file names to indicate certain patterns are present without specifying them explicitly.

The available patterns are:

$INDIR - each input directory as specified by the indirs parameter
$OUTDIR - the output directory name
$OBSID - the observation ID to use for output files as given by the obsid parameter

For example, a pattern of inufafiles="$INDIR/xti/event_cl/ni*_0mpu7_ufa.evt" will look in each input directory in turn, searching for files of the requisite subdirectory and file name pattern.

The output observation ID is given by the "obsid" parameter, and this is used to name the output files. You can specify a different obsid with this parameter, or simply specify output file names explicitly. If any output file name is listed as "NONE" then that file type is skipped.

For each file type, the task will merge the input files that match that type using the appropriate tool, and sort them by time. For event files, NICER FPM selection and GTI information is preserved, so the files can be used downstream for product and response extraction. When you run nicerl3-spect or nicerl3-lc, the "indir" parameter of those tools should be set to niobsmerge's outdir. See below for an example.

By default, the task will not attempt to merge orbit files. Set doorbfiles=YES to request that file type be merged.

PARAMETERS

indirs="NONE" [string]: A comma-separated list of NICER product directories, and @filename.lis which specifies a list of directories, one per line. These directories should typically be where nicerl2 created its output, which is commonly the observation directories themselves. If indirs=NONE, then niobsmerge usees file lists from inclfiles, inufafiles, inmkfiles and inorbfiles.
(outdir=".") [string]: Output directory name where merged products are placed. The directory is created if it does not yet exist.
(inclfiles="{$INDIR,$INDIR/xti/event_cl}/ni*_0mpu7_cl.evt{,.gz})") [string]: Pattern matching string to find "cl" cleaned event files within each indir. Use of standard Unix glob-matching patterns is allowed. Also, certain shorthand patterns are allowed as documented above. If indirs=NONE, then no shorthand patterns are allowed, and the user is expected to provide a comma-separated list of files for this file type, or an @filename.lis file containing a list of files.
(inmkfiles="{$INDIR,$INDIR/xti/event_cl}/ni*_0mpu7_ufa.evt{,.gz})") [string]: Pattern matching string to find "ufa" unfiltered event files within each indir. Use of standard Unix glob-matching patterns is allowed. Also, certain shorthand patterns are allowed as documented above. If indirs=NONE, then no shorthand patterns are allowed, and the user is expected to provide a comma-separated list of files for this file type, or an @filename.lis file containing a list of files.
(inorbfiles="{$INDIR,$INDIR/auxil}/ni*.mkf{,.gz})") [string]: Pattern matching string to find filter ("mkf") files within each indir. Use of standard Unix glob-matching patterns is allowed. Also, certain shorthand patterns are allowed as documented above. If indirs=NONE, then no shorthand patterns are allowed, and the user is expected to provide a comma-separated list of files for this file type, or an @filename.lis file containing a list of files.
(inumkfiles="{$INDIR,$INDIR/auxil}/ni*.orb{,.gz})") [string]: Pattern matching string to find orbit files within each indir. Use of standard Unix glob-matching patterns is allowed. Also, certain shorthand patterns are allowed as documented above. If indirs=NONE, then no shorthand patterns are allowed, and the user is expected to provide a comma-separated list of files for this file type, or an @filename.lis file containing a list of files. Note that you must also set doorbfiles=YES to cause orbit files to be merged.
(obsid="merged") [string]: Observation ID name that will be used in $OBSID pattern matching in output file names. Note that the actual file OBSID_ID keywords are not changed. Note that white space and punctuation characters are not permitted: white space and periods are transformed to underscores, and other punctuation is removed. To facilitate automated usage, specify an obsid only with letters, digits and underscores.
(clfile="$OUTDIR/ni$OBSID_0mpu7_cl.evt") [string]: Name of output "cl" cleaned event file. A value of "NONE" disables this output. Also, certain shorthand patterns are allowed as documented above.
(ufafile="$OUTDIR/ni$OBSID_0mpu7_ufa.evt") [string]: Name of output "ufa" unfiltered event file. A value of "NONE" disables this output. Also, certain shorthand patterns are allowed as documented above.
(mkfile="$OUTDIR/ni$OBSID.mkf") [string]: Name of output filter ("mkf") file. A value of "NONE" disables this output. Also, certain shorthand patterns are allowed as documented above.
(orbfile="$OUTDIR/ni$OBSID.orb") [string]: Name of output orbit file. A value of "NONE" disables this output. Also, certain shorthand patterns are allowed as documented above. Please note that you must also set doorbfiles=YES to enable this output.
(doorbfiles=NO) [boolean]: Set to "YES" to enable orbit file merging. Please note that orbit files must be available in the input directories. If they are not available, this output is skipped.
(cleanup="YES") [boolean]: If yes, then clean up temporary files. If no, temporary files remain. This is typically for debugging.
(clobber = NO) [boolean]: If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.
(chatter = 1) [integer, 0 - 5]: Controls the amount of informative text written to standard output. Setting chatter = 4 or higher will produce detailed diagnostic output; chatter = 1 prints out a basic diagnostic message. The default is to produce a brief summary on output.
(history = YES) [boolean]: If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the niobsmerge task parameters that were used to produce the output file.

EXAMPLES

Merge a set of observation directories data/[0-9]* and extract spectral products. The outputs are placed in the directory "outmerged", and standard file naming is assumed.


  ls -d data/[0-9]* > indirs.lis
  niobsmerge @indirs.lis outdir=outmerged obsid=merged clobber=YES
  nicerl3-spect outmerged obsid=merged clobber=YES

Merge a set of files listed explicitly. Output files will have standard naming.


  ls -d data/[0-9]*/xti/event_cl/ni*_0mpu7_cl.evt* > clfiles.lis
  ls -d data/[0-9]*/xti/event_cl/ni*_0mpu7_ufa.evt* > ufafiles.lis
  ls -d data/[0-9]*/auxil/ni*.mkf* > mkfiles.lis
  niobsmerge indirs=NONE outdir=outmerged inclfiles=@clfiles.lis \
     ufafiles=@ufafiles.lis mkfiles=@mkfiles.lis clobber=YES

Merge orbit files in addition to the other file types.


  ls -d data/[0-9]* > indirs.lis
  niobsmerge @indirs.lis outdir=outmerged clobber=YES doorbfiles=YES

Disable output of UFA files.


  ls -d data/[0-9]* > indirs.lis
  niobsmerge @indirs.lis outdir=outmerged clobber=YES ufafile=NONE

Specify explicit names for output files, and disable UFA output.


  ls -d data/[0-9]* > indirs.lis
  niobsmerge @indirs.lis clfile=cleaned.evt ufafile=NONE mkfile=merged.mkf clobber=YES

LAST MODIFIED