NICER Light Leak Overview

Overview

Starting on May 22, 2023, NICER experienced a light leak that significantly affects operations and scientific analysis. This page provides an overview of what occurred, its affects on day-to-day operations, and its impacts to NICER data.

Read this thread if you want to: Understand the impacts of the NICER light leak

Last update: 2024-01-25

Introduction

NICER is an X-ray observatory mounted on the International Space Station or ISS. It is designed to observe astronomical X-ray targets with sensitive silicon X-ray detectors.

The silicon detectors that NICER uses, which are sensitive to X-rays, are also sensitive to optical light. Optical light can degrade the performance of detectors used to sense X-rays. For this reason, NICER's design contains several levels of optical blocking which remove a large fraction of optical light incident on the NICER apertures.

The ISS is an optically bright environment. The optical light present in low earth orbit due to the sun and bright earth limb affects any X-ray observatory in a similar orbit. In addition, the ISS structure itself is highly reflective, typically white, which creates an optical bright ambient environment, mostly during orbit day.

On May 22, 2023, NICER developed an optical light leak in one or more light-blocking films mounted in front of the NICER optics. The overall effect is to increase the amount of optical light available to NICER's detectors. This light leak has had a significant impact on NICER's operations and NICER data.

This page provides an overview to what happened, the overall impacts to operations, and its effects on NICER data. The prospects for a repair are also discussed.

The Nature of NICER's Optical Light Leak

On May 22, 2023, between 13:15 and 13:45 UTC, a light leak developed in one or more of NICER's thermal films. These films primarily protect NICER's X-Ray Concentrator (XRC) optics from large temperature variations due to the low earth orbital environment. However, the films also have an aluminized layer that provides a significant degree of optical light blocking.

Time(s) (UTC)	Description
2023-05-22T13:10	ISS orbit night begins
2023-05-22T13:45	ISS orbit night ends - light leak first detected
2023-05-22 - 2023-05-29	Discovery and diagnosis of light leak
2023-05-30 - 2023-06-06	Implementation of operational mitigations
2023-06-06 -	Continued observations of NICER targets

Table 1. Timeline of NICER light leak

As can be seen from Table 1, the light leak developed sometime during orbit night on May 22. As will be discused further below, the performance of NICER detectors is almost entirely unaffected during orbit night, so the impacts of a light leak were not experienced until the beginning of the next orbit day.

The next ~two weeks were spent discovering the light leak, diagnosing it, and devising a revised operational strategy for observations going forward. During this entire time, NICER continued to observe X-ray targets.

Despite the optical light leak, NICER continues to observe X-ray targets and obtain valuable observations of X-ray targets. The portion of data observed during orbit night continues to be excellent; data observed during orbit day may be more challenging to analyze but still can contain high quality data.

Discovery of Light Leak

The light leak was discovered as the system began to produce out-of-family telemetry, namely very high count rates. In addition, NICER's automatic detector protection systems automatically disabled detectors more frequently than usual.

A primary indicator of optical light is the undershoot reset count rate. A high rate of this type of event indicates that a detector is receiving enhanced levels of optical light, which in turn increases the leakage current. Higher leakage current leads to more frequent detector undershoot resets.

For a more detailed overview of undershoots and optical loading please see our Undershoot and Optical Loading discussion page.

If we examine the undershoot rate as a function of time, a clear change in behavior is visible on May 22nd (Figure 1).

Figure 1. Plot of NICER's undershoot rate (FPM_UNDERONLY_COUNT) spanning the six days around the light leak onset.

From Figure 1 we can see that a clear change in behavior occurred around mid-day (UTC) on May 22nd. Figure 2 below shows a zoom of the plot around the onset.

Figure 2. Plot of NICER's undershoot rate (FPM_UNDERONLY_COUNT) spanning two hours around the onset of the light leak. Orbit night is shown (green shaded).

This figure clearly shows low undershoots during orbit night (green) are nominal and then a greatly enhanced undershoot rate after the next orbit day begins. Note that the standard NICER filtering excludes undershoots greater than 500 ct/s/FPM, so the undershoot rate did truly increase to extreme levels.

High undershoot rates after the onset were the first indication that optical lighting conditions for NICER detectors had changed.

Overview of NICER Exterior Surfaces

To put the light leak in context, we need to understand how NICER was built. NICER consists of 56 independent X-ray telescopes, each consisting of an X-Ray Concentrator (XRC) optic, and a corresponding Focal Plane Module (FPM) X-ray detector. These individual telescopes are mounted in a mechanical structure similar to a box as the X-ray Timing Instrument (XTI), which is then wrapped in protective blanketing.

Below is an image of the XTI from before launch, showing all 56 XRCs and 56 FPMs in the mechanical structure, before blanketing has been installed.

Figure 3.Pre-launch image of NICER's XTI before thermal blanketing was installed.

Each X-ray telescope is protected by some structures, as shown below.

Figure 4.A single XRC thermal film (left) and sun shade (right), before installation.

Figure 5.Exploded view of the assembly of an XRC, sun shade and thermal film for an individual NICER telescope module.

The sunshade provides some shading from direct sunlight on the XRC optics, for pointing angles between 45 and 90 degrees from the sun. In Figure 4, the single sun shade is black composite material; in Figure 3 56-telescope assembly, the sun shades have been painted white on their exteriors.

In Figure 3, the thermal films have been installed but are not easily visible. They consist of a very thin layer of aluminized mylar which serves as a thermal insulator against temperature variations in the orbital environment. The aluminization also provides an optical light barrier. Owing to their thinness, the XRC thermal films are extremely prone to damage and puncture.

NICER's thermal films, which also serve as optical light blocking filters, are extremely fragile because they are thin.

Imagery of NICER Confirms Optical Leak

NICER obtained images of NICER in orbit after the onset of the leak. With the context above we can interpret these images.

With the support of the ISS team, we were able to obtain images of NICER of varying quality.

Figure 6. Imaging survey of NICER (right) taken on May 30, 2023. The inset (left) shows a blow-up of FPM 5-5 (DET_ID=44).

Figure 6 shows an image from a video survey taken by the ISS on May 30, 2023. The image shows that DET_ID 44 (FPM location 5-5) no longer has one "pie wedge" of its thermal films. Additional imaging has confirmed the damage to the thermal film of this module.

In addition to FPM 5-5, there is evidence of other damage that may be related to this light leak episode. DET_ID 15 (FPM 2-6) also shows some degradation or damage, and some other detectors may have smaller "pinhole" leaks.

NICER is working with the ISS team to obtain better imagery.

In conclusion, NICER experienced a puncture of at least one thermal film on May 22, 2023, leading to enhanced optical light loading.

The cause of a puncture is likely to be some form of debris. The ISS is not always a clean environment. Owing to the fragility of the thermal films, even a small or slow-moving piece of debris has the potential to create a puncture. Also, note that NICER is not the first observatory to suffer the effects of debris: Swift XRT experienced micrometeoroid impacts which damaged the X-ray CCD detectors; and RXTE PCA also lost the propane layers of two PCUs due to debris punctures.

An important point needs to be made. It is not entirely important which specific thermal film is damaged. The interior of the NICER mechanical structure is open, meaning optical light that enters through any large enough opening will illuminate the entire interior structure. All detectors will be affected when light enters. Some detectors are affected more than others, but there are no detectors that are immune from loading problems when optical light enters.

The optical light leak can create problems for all detectors, regardless of which specific thermal film (or films) were damaged.

Impacts of the Light Leak

Here we discuss the impacts of the light leak for NICER observers.

Impacts can be divided into two areas:

• Operational impacts - changes to NICER operations to mitigate the effects of the light leak.
• Analysis impacts - changes to NICER analysis workflows or calibration.

We will discuss each of these below.

Operational Impacts

NICER has changed its operations to accomodate the effects of the optical light leak. These may be further changed as the NICER team gains more experience and information about the issue.

Orbit Night: No Major Changes

There are no restrictions during orbit night. NICER observes targets and obtains excellent quality data.

NICER continues to observe targets during orbit night with its long-time observing constraints and detector configuration. The calibration remains unchanged.

Orbit Day: Observing Constraints

During orbit day several constraints are in effect.

• NICER only observes targets >100 deg from the sun.
• The earth limb constraint is >45 deg instead of 30 deg.

These constraints reduce the amount of ambient and direct optical light that NICER experiences, and thus reduces the impact of the light leak. However, these contraints also reduce the number of visible targets visible to NICER, which in turn may reduce overall observing efficiency as well as the availability of NICER to perform monitoring of targets on their specified cadences.

The specific visibility of a given target under the new optical light leak conditions will depend on a number of factors, especially its location in the sky and its angular distance from the sun. This angle will vary seasonally. In addition, the fraction of orbit day and orbit night in a given orbit vary seasonally due to precession of the ISS orbital plane. The NICER Enhanced Visibility Tool provides and indication of when a target is visible, taking into account the detailed ISS structure and other observing constraints. This tool will be modified over the coming weeks to better reflect NICER's true observing constraints after the light leak.

During orbit day, NICER avoids targets within 100 deg of the sun as well as 45 deg of the earth limb.

Orbit Day: Instrument Configuration Changes

In addition to observing constraints, some detector configurations have changed as well.

• low energy threshold raised by about 120 eV to ~370 eV
• 7-10 detectors disabled

Additional detectors are disabled during orbit day to reduce the overall telemetry load. These are detectors that have known optical light sensitivity, and thus disabling them produces a strong benefit.

The low energy threshold is nominally at about 250 eV. During the high optical loading, the noise peak can broaden and actually shift to higher energies. Under the new configuration during orbit day, the NICER team automatically raises the low energy threshold by about 120 eV to ~370 eV. This reduces problems with telemetry saturation due to the properties of the noise peak.

During orbit day, some NICER detectors are disabled and the low energy threshold of all detectors is increased by command to reduce telemetry load.

Overall Operational Impacts

As can be seen these changes will change how NICER observes targets, which targets are available to be observed, and the properties of the data obtained.

Observations during orbit night are not affected. Averaged over a solar year, NICER spends 34.2% of its time in orbit night, and those data will continue to behave as before.

It should be noted that due to the precession of the ISS's orbital plane (i.e. the "beta" angle), NICER experiences seasonal changes in the durations of orbit day and night. During some periods of time there may be several weeks of no orbit night at all ("high beta angle" periods), and during other periods, orbit night will last longer than the average 34.2% of the orbit. This will complicate the planning of long term monitoring and time critical observations.

Thus we estimate that the light leak may cause a current science capability loss of 25-35%, but several key investigations may lose closer to 50% of the capability needed.

Also, due to higher telemetry usage during orbit day, it may not be possible for NICER to observe bright X-ray targets during that time. The combination of high telemetry rates due to optical light loading, plus the X-ray count rates of the target itself, increases the chances of large data losses. Observers should coordinate with science planners before an observation begins, in order to obtain the best science return from NICER.

In addition, the operation of OHMAN, the On-orbit Hookup of MAXI and NICER had been curtailed while the team investigates the light leak issue, and NICER has less visibility during orbit day. As of August, 2023, OHMAN activities are increasing.

The optical light leak may hamper NICER observations of bright targets during orbit day, as well as long term monitoring and time-critical ToO observations.

Data Impacts

Now we discuss impacts to NICER data once they have been received and processed by the NICER pipeline.

High Telemetry Rates

NICER experiences significantly higher telemetry rates after the optical light leak onset.

The impact of this is that file sizes are larger and processing times are increased, compared to pre-light-leak.

Data Loss Issues

During orbit day, event rates can become very high. At these high rates, the detector processing units, known as MPUs, may become overloaded and discard data. This will lead to lost (or possibly degraded) science.

There are two possible forms of data loss that the MPUs record:

• Lost events. Lost events occur when an internal event buffer overflows before events can be packaged and transmitted.
• Lost science packets. Lost packets occur when event packets have been created but the downlink buffer overflows.

Lost events occur when FPMs provide event inputs at a faster rate than they can be processed for downlink. During the normal course of observations, lost events can occur very occasionally, but this is not expected for the large majority of science observations.

Lost events are particularly tricky because they are not accounted for in the event file's good time intervals (GTIs).

Instead, lost events are tallied in the MPU housekeeping, which are then transferred to the filter file in column MPU_LOWMEM_FIFO_DELTA. This column is a 7-vector that reports the number of discarded events for each MPU.

Starting in HEASoft 6.32 (NICERDAS 11), a new column is calculated in the filter file called TOT_LOWMEM_FIFO, which is the array-total number of lost events. In addition, HEASoft 6.32 changes the format of the filter file slightly to prevent numerical overflows of this column. For this reason, it is recommended to reprocess all data with HEASoft 6.32 or later if your observations include the time when light leak is active.

Lost events are best recognized by reprocessing your data with HEASoft 6.32 or higher.

Lost packets occur when an MPU is able to package events into CCSDS packets, but the outgoing buffer used to transmit these packets is full. The result is an entire lost CCSDS packet. Since many events are lost in this scenario, this kind of loss may seem worse than lost events, and in some ways it is, but lost packets have one benefit.

There is enough packet accounting to allow these lost events to be recognized at the GTI level, which allows the downstream software to correctly account for the loss using normal science tools. Thus, no additional data processing or screening steps are required to properly account for this kind of loss.

However, if many packets are lost, this can lead to a "GTI shredding" appearance to your data. Here, "shredding" means that there are many small GTIs corresponding to many individual lost packets. In principle, however, data should still be good in this situation; it just has less than 100% exposure efficiency.

In addition to this method of accounting, lost packets are also tallied in the MPU housekeeping, which are then transferred to the filter file in column MPU_LOWMEM_SCI_DELTA. This column is a 7-vector that reports the number of discarded packets for each MPU.

Starting in HEASoft 6.32 (NICERDAS 11), a new column is calculated in the filter file called TOT_LOWMEM_SCI, which is the array-total number of lost packets.

Lost packets automatically yield correct exposure results.

Energy Scale Shifts

High optical loading can produce extremely high undershoot count rates. Optical loading also tends to distort the energy scale. At the calibrated range of undershoots (0-500 ct/s/FPM), this distortion tends to be a nearly linear shift in the energy scale with increasing undershoots, and is well-understood.

However, at the extreme undershoot count rates NICER can sometimes experience during orbit daytime (>500 ct/s/FPM), the energy scale can shift in non-linear ways. This can be exacerbated by the the "event loss" problem noted above. This effect tends to discard undershoots as well as X-ray counts, leading to an underestimate of the true undershoot rate, and thus an underestimate of the true energy scale shift.

Scientists should continue to use the 0-500 ct/s/FPM range for undershoots (or tighter) to avoid energy scale calibration issues.

Commanded Threshold Changes

As of June 2, 2023, the NICER team routinely adjusts the low energy threshold during orbit day observations.

The impact of this operational change is to change the effective energy range over which NICER observes X-ray targets. Before the light leak, and during all orbit night observations, the X-ray threshold is set to be effectively ~250 eV, although the shape of the trigger efficiency curve is more complicated than a simple sharp cutoff at low energies (see FPM Detector and XRC parameters for more details).

During orbit day after the light leak, the low energy threshold has been increased by approximately 120 eV to a new centroid of about 370 eV.

The figure below shows threshold variations as a function of time.

Figure 7. Plot of NICER array-averaged threshold setting (filter file column DELTA_SLOW_LLD) for 20000 seconds at the beginning of UTC day 2023-07-09.

As the plot shows, the threshold steps up and down, corresponding to transitions between orbit day and night, respectively. HEASoft 6.32 adds a new column to the filter file called DELTA_SLOW_LLD which aids in tracking array-averaged threshold adjustments such as this.

Note that the units of this setting are arbitrary, and correspond to the commanded setting offset value from nominal. The actual value in eV is still under investigation.

This change will complicate analysis of X-ray targets observed during orbit day, or a mix of orbit day and orbit night. Currently it is not recommended to mix post-light-leak orbit day and orbit night data, since they contain a mix of two different thresholds. The NICER software is not capable of handling this type of change.

Instead, it is recommended that users analyze orbit night and orbit day data as two separate products. As of HEASoft 6.32 (NICERDAS 11), NICER provides some tools to select data taken with different threshold settings.

Scientists should strongly consider reprocessing data with HEASoft 6.32 if their observation contains light leak data. This is true if even a portion of the observations overlap with the light leak, since the newly added features in this release change the format of some filter file columns; reprocessing all data will give consistent filter file information for all data.

The thresh_range parameter of nicerl2 allows selecting a threshold range for analysis.

"Noise Ringers"

Starting in early 2023, the NICER team became aware of a phenomenon that we are calling "noise ringers."

Noise ringers are a particular detector phenomenon that manifest preferentially during times of high undershoot levels. The greatest symptom of noise ringers from a science analysis point of view is to create additional noise in the 0.2-0.6 keV pulse height range. This noise is outside of the normal range of the noise peak, but interactions between undershoot resets and the noise peak tends to produce a visible tail that extends to higher energies.

NICER spectra can experience noise ringers at any time there are high optical loading conditions, typically above about 80 ct/s/FPM. This is true both after and before the optical light leak of May 22, 2023. However, since optical loading has become such a greater effect after May 22, the chances of experiencing noise ringers is higher.

Please see the Noise Ringers documentation page for more detailed information about this effect.

Noise ringers can produce unwanted noise tails which extends to about 0.6 keV. The optical light leak of May 22, 2023, enhances the channces of having noise ringer effects. HEASoft 6.32 contains mitigations to reduce or even entirely remove the effects of noise ringers, even up to the maximum calibrated limit of 500 undershoots/s/FPM.

What's Next for Observers and Analysts

Please continue on to the Analysis Recommendations After the Light Leak page to find more guidance and recommendations for users that have data taken during the optical light leak.

What's Next for NICER (Repair and Mitigation Options)

The NICER team continues to investigate the light leak problem, and implement operational and data processing strategies to mitigate the problem where possible.

The NICER team is currently working with the ISS program and other partners to establish feasibility of a repair of the optical leak, and to improve NICER's performance. The Update on the Repair of the NICER Light Leak page has a more in-depth discussion of the repair options as of this writing.

While the full repair option is being developed and executed, the optical light leak remains. It is likely that NICER will have to live with the optical light leak problem for the near to intermediate future (through a large portion of 2024).

Therefore, it benefits scientists to understand the impacts and how to adjust their observing and data processing strategy to get the most out of their observing program. Please see the Analysis Recommendations After the Light Leak page for a discussion of how you can address the effects of light leaks on your data.

Modifications

• 2023-07-14 - initial draft
• 2023-08-18 - light editing for readability
• 2024-01-25 - add further discussion of repair progress