PCA SAA History and Background Problems


C. B. Markwardt, J. Swank, P. Boyd, G. Rohrbach, M. Tripicco
Last Updated: 2008-02-11

Table of Contents

  1. Summary
  2. Introduction
  3. Problems with the SAA History File
    1. Data Gaps
    2. Daily Missing SAA Passes
    3. Repeated Data
  4. PCABACKEST Bug
  5. Discussion of Impact
    1. Faint Model Details
    2. Bright Model Details
  6. RXTE Standard Products
  7. Conclusions for Observers
  8. Preventative Measures
  9. Change History

Summary

An investigation in Sep 2007 found that the SAA history file had a number of problems, including: data gaps (including one 15 day gap); repeated data; missing data at the beginning of the day. The SAA history problems occur throughout much of the mission, but the worst effects are concentrated on a few tens of days.

There is a completely separate bug in the 'pcabackest' background estimation program (version 3.0). The bug affects observations taken from 2004-Jan-01 to present using the 'faint' model pca_bkgd_cmfaintl7_eMv20051128.mdl (which has been available since 2006-Aug-06). The result is that the SAA and long-term temporal components are by default ignored for the given time range, regardless of the state of the SAA history file.

For users of the faint model the combination of these two problems will primarily cause an error of the background for all data taken after 2004 processed with the given model file (with amplitude <10%).

For users of the bright model the problems will primarily cause an underestimate of the background for observations taken within 50 minutes of an SAA, when there is a gap in the SAA history file (with amplitude of <30% [approx.]).

A new SAA history file was created using archival HEXTE particle monitor data. The process for creating the SAA history file has been improved, including the algorithm, and error checking.

Recommendations to observers:

  1. Unfortunately, since the errors are a complicated function of time, SAA history and energy, there is no simple way for an observer to know if their observations are affected, other than to reprocess their data with the correct methods.

  2. When running pcabackest, use the parameter "maxmodels=2000".
    The problem occurs with pcabackest version 3.0 and earlier. The problem is solved by pcabackest version 3.1, released 21 Sep 2007 with HEASOFT release 6.3.2, but there is no harm in using maxmodels=2000.)

  3. Use the new corrected SAA history file. (See PCA Digest page)

Introduction

The PCA background is a function of time since SAA. There is radioactivation within the instrument, which decays on various timescales. The famous "240" minute timescale has never been identified with a particular element, but it is used in the L7/240 faint model. The bright model actually uses two activation components. To get an accurate estimate of the amount of activation, the HEXTE particle monitor is used, which remains on even during SAA passages. The PCA instrument itself is off during SAA passes, but the activation persists for several hours afterward.

Estimates of the PCA background are computed using the pcabackest task, a model file, and a summary of the SAA passages. The SAA passes are stored in the "SAA history" file. This file is nominally updated daily in the XTE SOF using HEXTE particle monitor data. The history file records for each SAA pass the cumulative dose (in units of particle monitor counts) and the time of peak count rate.

Problems with the SAA History File

In September 2007, we looked at the SAA history file to try to "clean" it up (i.e. put in time order, remove accumulated HISTORY keywords, etc). However, we found several other more major problems.

Data Gaps

There were several data gaps in the SAA history file. The table below shows when the gaps were. In some cases there is a known reason for the gap.

The "HEXTE single event upset" was a problem with HEXTE which prevented particle monitor data from being recorded. During that time we do not have valid measurements, although we can estimate them based on the "predicted" file.

The large gap from 30 Apr - 14 May 2007 was not always present. The SAA history processing ran properly at first, but on 25 May a portion of the file was deleted, causing the 15 day gap which has persisted until this analysis.

Impact: users with observations during this time, who run PCABACKEST, will find the background model is underestimated because the SAA component is missing. See below for a detailed discussion of impacts and corrective actions users should take.

Start Time (UTC)

End Time (UTC)

Reason (if known)

1998-10-16T19:48:16

1998-10-17T19:58:08

Spacecraft single event upset

2000-02-18T23:57:20

2000-02-20T01:33:52

unknown

2001-11-26T22:15:28

2001-11-29T19:59:12

Spacecraft ACS anomaly

2003-03-31T10:02:40

2003-04-01T21:40:48

HEXTE single event upset & reboot

2003-08-27T22:37:04

2003-08-29T13:32:00

unknown

2003-12-02T22:44:00

2003-12-04T12:01:20

unknown

2004-07-03T22:57:52

2004-07-05T01:51:12

unknown

2005-09-01T15:01:36

2005-09-03T04:13:52

unknown

2006-08-14T20:10:08

2006-08-16T09:21:52

unknown

2007-04-30T22:33:15

2007-05-14T07:23:15

unknown [*]

Table 1. Data Gap Table

Note [*] - the large 15 day gap from 30 Apr to 14 May was lost because of an apparent processing crash in the XTE SOF. Files with time-stamps 20070514 through 20070524 contained the SAA passes, but after 20070525 did not.

Several corrective actions have been taken.

Where possible, gaps have been filled by reprocessing the archived HEXTE particle monitor data. In cases where the spacecraft or HEXTE data were not available, predicted values have been used, based on the mean HEXTE particle monitor behavior and the orbital path through the SAA region.

We have also gone through the resulting SAA history file and searched for times that an SAA history pass was expected based on the orbit, but none was recorded. In those cases, predicted values were used as well. Predicted values are generally reliable at the ~20% level; the error in the new prediction may contribute at most ~2% of the overall background amount.

Daily Missing SAA Passes

We also found that throughout most of the mission, SAA passes at the beginning of the UTC day are often missing. This is especially noticeable within the first three hours of the day, where approximately 50% of SAA passes are missing (see Figure 1).

saa-vs-dayfract-sm.png
Figure 1. Histogram of recorded SAA passes as a function of the fraction of day. Note the ~50% deficit in the first ~3 hours of the day.

Impact: users with observations after those particular SAA passes will have a systematically underestimated background model. Figure 2 shows the effect of the missing passes. The effect is periodic because of a complex beat between the spacecraft orbital period, the earth rotation period, and the begin-of-day sampling window.

lost-daystart-saa.gif
Figure 2. Plot of SAA passes (and doses) that occur in the first 3 hours of the day. About 50% of these passes were discarded in the original SAA history file.

A periodogram of Figure 2 is shown in Figure 3. The periodic (or quasi-periodic) structure is quite evident. It should be pointed out that most users will not have XTE observations that are systematically scheduled at the start of each day, so the periodic effect of the missing SAAs will be significantly reduced in practice. Still, users should be aware of the listed periodicities in their data.

lost-daystart-lomb.gif
Figure 3. Lomb periodogram of the time series in Figure 2. Periods at 15, 22 and 45 days are apparent. Visual inspection of the time series suggests that the 45 day period is the most significant one.

As above, we have attempted to correct for the loss of these passes. Where possible we have used existing HEXTE particle monitor data, and in a small number of cases, predictions were used.

We believe we have corrected the algorithmic error which produced the missing passes.

Repeated Data

There were a number of cases where SAA dosages were reported twice. The sample literally appeared twice in the file. The table below shows the time intervals with problems.

Impact: users with observations during this time, who run PCABACKEST, will find the background model is overestimated because the SAA component is counted twice. See below for a detailed description of impacts.

Start Time (UTC)

End Time (UTC)

Reason (if known)

2003-09-01T12:27:12

2003-09-11T18:48:32

XTE SOF new computer installation

2005-06-16T02:42:56

2005-06-16T04:21:36

unknown

2007-03-29T06:33:52

2007-03-29T16:30:56

unknown

Table 2. Time Intervals of Repeated Data

Repeated data have been removed from the SAA history file.

The algorithm has been improved so that repeated data should not occur in the future.

PCABACKEST Bug

While investigating the SAA history file problems, we have discovered a bug in the behavior of the PCABACKEST program (version 3.0). This program has a parameter called "maxmodels," which places a limit on the number of models it is capable of using. The default setting is 600, while the "combined mission" faint model file has 660 model components. The result of this problem is that for the last background interval (Epoch "5c"; covering data 2004-01-01 to present) the background estimate is missing several components including the SAA component.

Thus, regardless of the state of the SAA history file, users of the "combined" faint model for data observed from 2004-Jan-01 to present (background epoch 5C) will have background errors. The problem only occurs with the model file pca_bkgd_cmfaintl7_eMv20051128.mdl, which was released on 2006-Aug-06.

This problem does not affect the bright model, nor does it affect the faint models for observations taken before 2004.

Observers can correct the problem by invoking pcabackest with maxmodels set to a large number, such as "maxmodels=2000". For those observers that use runpcabackest, they must set the parameter value before calling pcabackest, with a command like this:

pset pcabackest maxmodels=2000

As noted above, pcabackest version 3.0 will function properly as long as the maxmodels parameter is set to a large number.

A fix to the problem is available with pcabackest version 3.1. This version was released on 21 Sep 2007, with HEASOFT 6.3.2. However, there is no harm in continuing to use maxmodels=2000 on the command line.

Discussion of Impact

Since the SAA history is an integral part of the PCA background estimation process, the problems noted above may have an effect on guest observers' data analysis.

Because the radioactivity effect in the PCA is persistent over several hours, observations after the SAA passage can be affected. The main problem is "missing" SAAs, either caused by data gaps in the SAA history file, or because the SAA component was not applied by pcabackest due to the bug. The effects of missing SAAs can be categorized as follows.

Energy Range

Faint Model
(worst case)

Bright Model
(worst case)

2-5 keV

-9.3%

-27%

5-10 keV

-0.4%

-21%

10-20 keV

-2.1%

-18%

20-50 keV

-0.8%

-23%

Table 3. Worst case errors in the faint and bright models because of missing SAAs. For the faint model the typical errors are about half those shown. The bright model error distribution is very wide. Although the worst case can have errors of up to ~30%, about three fifths of observations have errors less than 2%.

Faint Model Details

For users of the faint model, large numbers of missing SAAs lead to an error of the background, which has a daily temporal cycle that peaks approximately during the ~5 orbits that pass through the SAA. Most of the error occurs in the 2-5 keV band. Table 3 shows the worst-case impacts (the typical impact is about half of the worst case). To repeat: Table 3 represents the impact to faint model users for data after 2004 and during 1-day SAA history gaps. Single missing SAA passes will cause a much smaller effect.

l7_saa_resid.png

l7_tdrift_resid.png

Figure 4. Spectral dependence of the L7 (faint) model errors for PCU 2. The two components shown depend on SAA (left) and long term temporal drift (right). The errors will be proportional to the templates shown.

The spectral dependence of the faint model errors is shown in Figure 4. The SAA component underestimates are concentrated mostly in the 2-5 keV band, although there is a weak overestimate effect at higher energies (10-20 keV and 30-70 keV). The actual amount of the SAA component error will depend on how recently the RXTE satellite passed through the SAA and the cumulative particle dose (as recorded in the SAA history file). The "long term temporal drift" component tracks gradual changes in the PCA background performance. The negative values in Figure 4 indicate that the background is slowly decreasing over time. Because of the pcabackest bug, this component would be ignored, leading to a separate overestimated background at lowest energies, an effect which grows with time. Because these components are both positive and negative, the amount of over(under)-estimate depends on the energy band and time of observation.

Users of the faint model with observations on the specifically noted days (and up to ~12 hours later) should in principle check to see whether the new SAA file has an effect (see "Conclusions" below).

Bright Model Details

For users of the bright model, the pcabackest bug is not an issue, but the problems with the SAA history file still apply. The worst impact of SAA history gaps occurs immediately after exiting the SAA (especially within 50 minutes of passing through SAA), and all energy bands are affected. The worst case shown in Table 3 represents the effect immediately after coming out of SAA. Times far from SAA are minimally impacted (factor of ~10 times less). To repeat: Table 3 represents the impact to bright model users during SAA history gaps only.

vle_saa1_resid.png

vle_saa2_resid.png

Figure 5. Spectral dependence of the VLE (bright) model errors for PCU2. Both components depend on SAA, with different exponential decay timescales: 240 minutes (left) and 24 minutes (right). The errors will be proportional to the templates shown.

Figure 5 shows the spectral dependendence of the bright model. The bright model has two different SAA components, representing two different (phenomenological) exponential decay time-scales. The 240 minute (slow) time-scale mimics the SAA component of the faint model, in that it gradually builds during the part of the day with SAA orbits, and is dominated by the 2-5 keV band. However, the 24 minute (fast) time-scale decays rapidly after exiting SAA, and is strong in all energy bands. It is this component that can contribute up to ~20% of the background errors.

The problem of missing single SAAs at the beginnings of days will affect primarily users of the bright model. Since the effects are distributed over hundreds of days, it is impractical to list all of the affected observations. Again, if there is any question, the user can check if their results are changed by using the new SAA history file and a proper maxmodels setting.

RXTE Standard Products

The RXTE Standard Products are also affected by the problems described in these reports.

The standard pipeline processing used a contemporary SAA history file at the time of processing, which means that the gaps and duplicated data noted above will be present. The SAA history problems affect data from the whole RXTE mission, although the effects are most concentrated on a few date ranges.

Regarding the bug in pcabackest version 3.0, the problematic models which triggered the bug were first installed 05 Feb 2007 (RXTE pipeline v6.3). The problem was fixed on 18 Sep 2007 (RXTE pipeline v6.5), using the maxmodels=2000 technique. The pipeline processing version is included in each standard product file, in the FITS keyword RXTESP.

Users interested in the most sensitive scientific analysis are definitely recommended to reprocess their data using the newest versions of FTOOLS, including pcabackest and the SAA History file. As noted in the RXTE Standard Products Guide, the standard products allow the user to get a quick, accurate overview of each archival observation, and are not intended to take the place of careful analysis by the user.

Conclusions for Observers

First and foremost, users either upgrade to HEASOFT 6.3.2 or later (which contains pcabackest version 3.1 or later); OR should change how they call pcabackest, by adding the "maxmodels=2000" parameter to the command line.

Observers should download a revised SAA history file from the standard location on the PCA Digest Page. This file contains the best available information, taken from either the archival HEXTE particle monitor data, predictions, or the original SAA file. We have done numerous consistency and quality checks on the new file. Also, we have searched the file for cases where an SAA pass should have appeared (based on the XTE orbit) but did not, and have corrected the discrepancies.

Users who are unsure of the provenence of their file should check the CREATOR keyword. Files created by the new method will be created by 'PMHIST'.

Preventative Measures

A new version of pcabackest, version 3.1, has been released, which corrects the problems present in version 3.0.

For the future, we have implemented new procedures for the ongoing creation of the SAA history file. Extensive error checking is now occurring to prevent future problems from occurring. In particular, the new file is regularly checked for data gaps, missing SAAs based on a predictive model, and duplicate data. Discrepancies are flagged for further human action.

The raw HEXTE particle monitor data is now being kept indefinitely. For the future, if problems should occur, this data can be used to investigate and correct the problems.

Change History

  • 2007-09-18 - Initial draft.

  • 2007-09-19 - Minor clarifications.

  • 2008-02-11 - Update regarding released version of pcabackest; also discussing the standard products archive.




If you have a question about RXTE, please send email to one of our help desks.

This page is maintained by the RXTE GOF and was last modified on Wednesday, 24-Aug-2022 11:10:33 EDT.