NICER Analysis Tips & Caveats


Here we list a number of issues of which users should be aware when analyzing NICER data. These include calibration uncertainties, data processing issues, and other issues.


MPU1 Time Stamp Anomaly from July 8th-23nd 2019

From Day of Year (DOY) 189 (July 8) ~14:18 UT till DOY 204 (July 23) 21:21 UT, NICER experienced a time stamp anomaly due to a Single Event Upset (SEU) that occurred during an SAA passage on July 8th. This caused events collected by MPU1 (detectors with DET_IDs from 10 to 17 inclusive) to have incorrect time stamps. As an example, a power spectrum from these detectors is shown in the figure below.

Figure showing the effect of the incorrect timetags from MPU1 on the NICER data

Figure 1. An example showing the effect of the incorrect timetags from MPU1 on the NICER data.


On July 22nd, MPU1 was power cycled, which resolved the time-stamp issue for MPU1 as indicated in the power spectrum shown below. No other MPUs were affected during this time.

Figure showing the effect of the incorrect timetags from MPU1 on the NICER data

Figure 2. An example showing NICER data after MPU1 was reset.

Internally, NICER timestamps are recorded with an oscillator that runs at ~25 MHz. Normally, every 1 second, these timestamps are registered to a GPS clock. The lower 25-bits of the count are stored on a per-event basis, and higher order timestamp bits are transmitted separately. When the upset occurred, the lower 25 bits became scrambled, while the higher order bits are accurate. Thus, the symptoms of the problem are accurate timestamps at the ~1.3 second level, but inaccurate timestamps at finer time resolution.

Observations for which timing of events is critical analysis (e.g. pulsar light curve modeling, QPO analysis, etc...) will be affected if data from DET_IDs 10 through 17 are included in the analysis for observations in the July 8th-22nd 2019 interval. A table of the OBSIDs for the affected observations is given below (and attached Obsids-unique.xlsx). For these observations, if one is doing timing analysis, MPU1 data should be removed from the combined event file using one of the following methods.

The recommended approach is to run 'nicerl2' with a modified mpulist option:

> nicerl2 ... mpulist=0,2-7 ...

where the mpulist above excludes MPU1.

For "quick and dirty" analysis of an existing event file, one can simply remove data from MPU1 using ftselect.

> ftselect input.evt output.evt expression="DET_ID.LT.10.OR.DET_ID.GE.20"

or

> fselect input.evt ouput.evt expr="DET_ID.LT.10.OR.DET_ID.GE.20"

Valid spectra can be extracted from these observations even if the MPU1 detectors are included. Please note that bright sources might have small deadtime errors, so a comparison of spectra with MPU1 included, and with MPU1 excluded, is advisable. Note that MPU1 normally reads out 7 detectors.

A file listing OBSIDs for the NICER observations affected by this anomany is available here.


Low Energy Noise

In general the useful energy band for NICER is 0.2 < E < 12 keV. The lower bound is set by the detector noise, which peaks near 0.1 keV. However, for data taken during orbit day, the low energy noise peak may intrude into the low energy band. Users should examine carefully the band at the lowest NICER energies. The noise peak tends to increase during orbit day, so that users should be careful about interpreting events in the 0.25 – 0.5 keV range for data taken during orbit day. In some cases, data at E ≤ 0.5 keV may need to be excluded from analysis.


Excluding data from noisy detectors

Analysis of detector behavior on-orbit shows that, under certain conditions, two of the NICER focal plane modules, namely DET_ID = 34 and DET_ID = 14, show episodes of increased detector noise. Users may want to exclude data from these detectors in their analysis. This can be done using fselect:

%  fselect ni1120010115_0mpu7_cl.evt ni1120010115_0mpu7_cl_50.evt "(DET_ID != 34) && (DET_ID != 14)"
where ni1120010115_0mpu7_cl.evt is the cleaned events file with data from all 52 FPMs, and ni1120010115_0mpu7_cl_50.evt is the output cleaned events file excluding FPMs 34 and 14.

Energy Calibration (Gain)

Nominally, the NICER PI detector channels are linearly spaced and 10 eV wide, covering the range 0.0-15.01 keV. The energy boundaries of the PI channels can be found in the EBOUNDS extension of the XTI response matrix file. There remain some residual uncertainties regarding the NICER energy scale (gain scale). Generally speaking, energy assignment is within 5 – 10 eV, except < 1 keV and in the 4 – 7 keV band (energy deviations in the 4 – 7 keV band are generally about 0 – 30 eV). At this time, users should be very careful about claiming red/blue shifts of < 30 eV in the Fe-K range (in particular) based solely on NICER spectra. In-flight testing found no strong dependence of gain on temperature or lighting conditions.


Photon Redistribution (Response Matrix) and Effective Area

Residuals of about 15% remain in the response matrix and effective area, with larger features at specific energy ranges. Users may want to include a systematic 15% uncertainty to spectral analysis.


Timing Calibration

After Level 2 calibration, the timing error is ≲ 100 ns. Note that, before calibration, a 1 second absolute timing offset exists due to the behavior of the on-board clock. As of August 2018, the archived data has a mix of calibration levels applied. For reliable absolute timing results, the user must apply most recent NICER software nicerl2 and use the Calibration Database version 20180711 or later. After processing with nicerl2, the correction for this 1-second offset is recorded in the TIMEZERO keyword in the header of event files, GTIs and filter files. The NICER data extractor will automatically adjust time values using TIMEZERO when creating light curves. Users who have developed custom software for timing analysis will need to account for the TIMEZERO keyword offset.


Deadtime

Deadtime is the time when the detector electronics are "busy" so no events can be recorded. All events, including resets, overshoots and forced triggers, create deadtime. Deadtime is recorded on a per-event basis in the UFA event file. Each MPU is parallel. Deadtime is also recorded in the MPU_DEADTIME column of the .mkf file. The "cleaned" event file (as found, for example, in xti/event_cl/niNNNNNNNNNN_0mpu7_cl.evt) excludes < 0.25 keV pulse heights, 5 forced triggers per second per FPM, undershoot resets (dark current), overshoot resets (typically charged particles).

Deadtime is particularly important for bright sources (≥ 20,000 NICER counts/s using 52 FPM). Very bright sources will cause telemetry saturation and a large increase in deadtime. For bright sources, telemetry saturation can be mitigated by using only a subset of the 52 FPMs to observe the source. This will reduce deadtime, but users should be aware that this will also reduce the total effective area of the XTI, so that this reduction in effective area needs to be corrected when deriving source fluxes. To first order, when N FPMs, are used the effective area is simply reduced by a factor N/52 compared to the standard, 52-FPM effective area normally used.


Fragmented Good Time Intervals

For very bright targets, users must be aware that, at high data rates, telemetry saturation will limit data throughput. The practical result of this is that event data can be fragmented into very short (millisecond) duration data segments. The NICER operations team attempts to avoid this condition, but it is not always possible to predict target intensity or respond quickly to intensity changes. As a result some processed datasets have been archived with large numbers (sometimes thousands or more) of very short good time intervals. Large numbers of short-duration good time intervals can make timing analysis challenging, and may result in identification of spurious peaks in the power spectrum which may be mistaken for periodic or quasi-periodic oscillations if care is not taken. In particular, users creating power spectra must properly deal with the window function created by the good time intervals (this is true for unfragmented observations but especially true for heavily fragmented data).

As noted above, for very high count rates (above approximately 20,000 NICER counts/sec), telemetry saturation can occur. Telemetry saturation can result in many very short good data intervals, with similarly short gaps between them. This information is correctly captured in the good time interval (GTI) table present in all NICER event data files (the GTI extension of the FITS binary tables).

This effect is illustrated in the example below, which uses data obtained from the bright black hole transient MAXI 1820+070. The data presented here were take from obsid 1200120107, using the cleaned event file, ni1200120107_0mpu7_cl.evt. Because of the brightness of this source, the GTI extension of this file shows 5912 individual good time intervals.

Figure 1 shows a continuous light curve of a short portion of the data obtained by binning the events with a bin size of 1/512 seconds. The count rate varies from about 20,000 to greater than 50,000 counts/sec. One can see that at high rates there are many short gaps in the data due to telemetry saturation, resulting in a large number of mostly short GTIs, with short gaps between consecutive GTIs. The presence of very short gaps between consecutive GTIs can result in spurious time signatures in the power spectrum due to the gaps alone, and not the X-ray source of interest.

NICER lightcurve of the bright black hole transient MAXI 1820+070 showing data gaps produced by telemetry saturation for obsid 1200120107

Figure 1. An example of the many short gaps that may result in a NICER light curve
of a very bright source due to telemetry saturation.


As a further example of this, we show a histogram of the time intervals between all GTIs in this event file, where the time intervals are simply the time separations between the midpoints of successive GTIs. The distribution is strongly peaked on timescales from 1/10 to 1/100 of a second.

Distribution of the time intervals between GTIs for obsid 1200120107

Figure 2. Distribution of the time intervals between GTIs for obsid 1200120107 emphasizing
the large number of very short good time intervals for this observations.


A power spectral analysis that includes the data gaps will very likely show spurious (non-source) excess power on these timescales. We show two power spectra from these data, one from a light curve which includes time bins within the data gaps, and one which, correctly, only uses continuous GTIs (in this case, bins which are at least 16 seconds long).

Figure 3 shows these two power spectra. The red curve shows the correct result, using only continuous GTIs, and the black curve shows the (incorrect) spectrum which included gaps in the data. One can clearly see excess power above the red curve, and indeed, a spurious "QPO" near 55 Hz. For timing studies of bright sources, the NICER team strongly recommends double checking the GTIs for each data file.

Power spectrum showing the influence of data gaps on timing analysis

Figure 3. Black: power spectrum of the NICER obsid 1200120107 with data gaps included;
Red: power spectrum of the NICER obsid 1200120107 with data gaps excluded.
Note the spurious peak which is seen in the power spectrum due to short gaps
between the good time intervals.


Pointing Jitter and Spurious Timing Signatures

NICER's X-ray concentrators were co-aligned to within 1 arcmin in order to optimize the X-ray throughput. Each concentrator optic is described by a response function that falls with off-axis angle. Additionally, each optic focuses X-rays onto a detector which has a 2 millimeter-diameter aperture. The combination of each optic's off-axis response and the fixed detector aperture sets the total throughput as a function of the off-axis pointing angle to a target. At about 3 arcmin off-axis the response is reduced by half, mostly due to the aperture stop. For smaller off-axis angles the response is only slightly reduced; for example, within 1 arcmin the response drops by only 2-4%.

NICER points by aiming its "boresight," a fiducial vector specified in the frame of reference of the NICER instrument that maximizes the X-ray throughput, to commanded target positions in the sky. NICER's pointing system is designed to maintain pointing stability to targets of interest at the 1 arcmin level. The summed response of all 52 X-ray concentrators and detectors is approximately "flat-topped" such that angular deviations associated with pointing system jitter at this level are not sufficient to introduce spurious, non-source variability into the observed X-ray flux. This is true for targets with sky coordinates known to high precision (much better than ~1 arcmin), and the NICER team is not aware of any such cases where substantial spurious variability may have resulted from pointing system jitter.

However, for newly identified targets with less precise coordinates (positional uncertainties larger than the arcmin level) users should be aware of the possibility of pointing-related variability being introduced into the observed count rates. The issue here is that when the target coordinates are not precise enough, the source may no longer be sampling the "flat-topped," on-axis response. In such cases, the off-axis variations due to pointing jitter can introduce detectable variations in the flux. A recent example of this effect was seen in observations of MAXI J1348-630 from February 8, 2019 (ObsID 1200530107). Relatively strong (~10% rms) count rate modulations were seen with a quasiperiod near 0.2 Hz. It was determined that for these observations NICER was pointing about 2 arcmin away from the subsequently derived (through independent imaging observations) correct source position. The feature disappeared once the correct pointing coordinates were adopted. The observed 0.2 Hz frequency is a known mechanical resonance associated with the NICER pointing system. The brighter the target, the easier it will be to detect such modulations.

Thus, users should be very cautious when interpreting apparent low-frequency (below 0.5 Hz) variability for bright targets where accurate, sub-arcmin coordinates may not yet be available.

Angular offset (arcmin) from the true source coordinates

Figure 4. The power spectrum of the angular offset (arcmin) from the true source coordinates for observation 1200530107 of MAXI J1348. For this observation, the pointing position is offset from the actual source coordinates by approximately 2.15 arcmin. This power spectrum shows a strong signal near 0.2 Hz (a period of ~5 seconds), which is a known mechanical resonance associated with the NICER pointing system. In mispointed observations like this, the off-axis variations due to the pointing jitter can introduce detectable variations in the observed count rate, producing a spurious (non-source) QPO at 0.2 Hz in the observed count rate. Note that other timing peaks are also seen in the power spectrum.

South Atlantic Anomaly

Particles in the region near the South Atlantic Anomaly (the SAA) produce non-cosmic background which can interfere with interpretations of data. Particle event rates can be very high and swamp source X-ray events. NICER processing uses a boundary defined by a parameter called NICER_SAA which is recorded in the .mkf file. Normally data are excluded from the cleaned event list when NICER_SAA = 1. For analysis of weak sources, or any analysis which is particularly sensitive to particle background, users might instead filter times based on the value of the SAA column in the .mkf file, since the SAA boundary region is considerably larger than the NICER_SAA region. Use of the SAA region for filtering will result in excluding more data than filtering with NICER_SAA, and it may be that some of the excluded data includes valid science events, so care should be taken when filter for times near SAA.


Last Updated: July 26, 2019