Combining Multiple NICER Observations
It is a common occurrence that multiple NICER observations cover a single scientifically interesting time window. This may occur either because NICER observation of more than one day are naturally broken into segments of one day or less, but it could also occur if the analyst wishes to combine multiple visits into a single data set. This thread gives advice in how to do this.
Read this thread if you want to: Combine multiple NICER observations into a single data set.
Last update: 2024-01-16
It is a common occurrence that multiple NICER observations cover a single scientifically interesting time window. There are several reasons for this to be true.
During the planning and pipeline processing steps, NICER observations are nevel allowed to be longer than a single UTC calendar day (starting at UTC midnight). If a science visit spans more than one calendar day, then the NICER visit can be broken into multiple segments. An analyst wishing to consider these data as a single object must somehow combine multiple segments into a single data set.
It is also possible that the analyst is interested in combining multiple visits of the same target into a single data set that spans it all. Here again the idea is the same: to take multiple NICER segments (a.k.a. sequences, a.k.a. OBS_IDs) into something more easily processed.
This thread gives advice in how to do this operation. There are actually multiple ways to achieve this goal, and we will discuss them, along with the pro's and con's of each.
Here is what is needed:
Each data set is labeled by its observation ID, or OBS_ID. As
retrieved from the HEASARC archive, an observation is stored in a
directory with a ten-digit numerical name, and that name is the
OBS_ID. For the purposes of the examples below, we will assume
that the observations are in the current directory under
the following names:
We also assume that the directories are laid out as retrieved from the HEASARC NICER archive. The steps outlined below will not work if you have rearranged the directory layout.
Method 1: Keep data sets separateThis method is actually not a method at all. NICER data sets come pre-divided into segments. These segments are no larger than a UTC calendar day (days are broken at UTC midnight, with some tolerance so that observations that span midnight are not split into two). Observation segments may be processed using all of NICER's software tools to generate spectra and light curves unique to each of those segments.
Using Method 1 is easiest in the sense that one simply runs the standard tools for each observation, and gets one spectrum or lightcurve for each observation, but may result in many output products for long NICER observations.
When to use: Use this method for quick-look, or if NICER's division into daily segments meets the analyst's scientific needs. For bright or highly variable sources, it is likely that you will want to make spectral or light curves for daily or shorter time periods any way, in order to capture the variability of the X-ray target.
What to do: No up-front effort is required to use this method. However, any products you generate will be naturally split by observation segment. For each data set separately, run nicerl2, followed by nicerl3-spect and/or nicerl3-lc, respectively, in order to produce spectra or light curves.
How to use products: There are a couple of ways to use the resulting products, as detailed below.
Using Method 1 Products
You have a couple of options when using the products from this process.
Method 2: Merge filter file and event lists.
This method involves combining all of the desired NICER data associated with a particular target into a single merged data set. The user can then extract products using the standard product tools.
Using Method 2 provides the most flexibility, but may result in very large merged data sets if count rates are high.
When to use: Use this method for combining long observations, to look for faint sources by increasing exposure, or to make long-term light curves.
When not to use: Combining many data sets into a single merged dataset may result in huge file sizes, which may be cumbersome to use, consume considerable disk space, and take long times to process. The user must be aware that long observations may span different periods of X-ray target behavior as well as different background conditions. Care must be exercised when filtering a merged product to take these into account.
What to do: Use the tool niobsmerge to combine data sets. This tool combines the filter file, screened event lists, and (optionally) the orbit file. The inputs of this tool are:
To construct an observation list, use the "ls" command and redirect
the results to a file. Here is a command which lists the observations
we have outlined in the example above:
We will place the outputs in a new directory named 1234567800. We pick this name since all observations start with 12345678, and the "00" can indicate that this is a "merged" data set. NICER data sets never end with a trailing "00", so using "00" for the last two digits is a unique and helpful indicator of a merged data set.
We run niobsmerge like this,
The result of niobsmerge is a fully merged set of event files (both UFA and CL), a merged MKF file, and (optionally) a merged orbit file. These files are placed in the output directory, and are named using standard naming for NICER files, such as niNNNNNNNNNN_0mpu8_ufa.evt, where NNNNNNNNNN is the obsid specified on the command line.
Note: if you are not using the 3C50 model, you do not need to merge the UFA event files. Not merging the UFA files save considerable disk space and execution time. To disable merging the UFA files, set ufafile=NONE on the command line.
How to use products:
At this stage, you have a combined data set. You can run the standard
nicerl3-spect for spectral analysis
nicerl3-lc for light curve analysis.
After running either of these product scripts, any downstream tools, such as XSPEC or lcurve, can be used as normal.
Other Methods Such as 'addspec' or 'addascaspec'
The NICER team does not recommend using spectrum adding tools such as "addspec" or "addascaspec" to combine spectral data sets. While it may be tempting, there are a number of problems.
First of all, these tools have not been tested with NICER data and can't be verified to produce valid results.
Second of all, if you are using a background model such as SCORPEON, there is no way to combine the "model" files using any of those tools. You will not have a valid background estimate.