INTEGRAL Science Data Centre (ISDC)
                  ===================================

                   http://www.isdc.unige.ch/integral


                            Known issues
                            ============

      Package:      OSA
      Version:      11.0
      Rel. Date:    19-Oct-2018

-------
General
-------

 1. Values of parameters of "real" type with more than 7 digits are
    handled badly by the Graphical User Interface (GUI). This can lead
    to errors when the analysis scripts are run. In particular, using
    the JEM-X GUI, if the user wants to enter timeStart and timeStop
    values through the GUI, the values will be truncated if, after you
    edited the fields, you close and reopen the hidden parameter
    window. Therefore, in case you want to enter accurate time
    constraints, make sure you reenter the timeStart and timeStop
    values each time you have to reopen this window.

 2. The "spe_pick" executable combining spectra of several science
    windows does not properly write the source position in the header
    of the resulting file.

----
IBIS  
----
  PICsIT
  ******
      
  1. The spectra extraction with the PIF method is not reliable for
     the moment (executable "ip_spectra_extraction"). The user should
     extract the spectra from images (count rates from intensity maps
     and errors from significance maps) and then convolve them with
     the RMF/ARF.

  ISGRI
  *****

  1. (OSA11) energy boundaries for the response matrix are taken from the 
     table in the file ic/ibis/rsp/isgr ebds mod 0001.fits. 
     They are chosen as close as possible from the boundaries
     defined int the input parameters. No boudaries from matrices in 
     previous versions of OSA should be used !
  2. The PHA2 files produced by spe pick contain a reference to the 
     response matrix used for the particular
     science window. These are referenced in the file system used for the 
     analysis. Therefore, the PHA2
     files cannot load a matrix when moved on another file system, unless 
     the user takes care of manually
     copying the single response matrices and update the entry in the 
     PHA2 file with other tools.
  3. In general, systematic uncertainties of about 1% should be added to 
     ISGRI counts, fluxes. However,
     this needs to be checked accurately by scientists performing their 
     analysis (see also below).
  4. Since 2016, ISGRI occasionally experiences particularly rapid and 
     unpredictable changes in the detector
     response, at the scale of up to 5%, which are not corrected in the 
     energy reconstruction and response
     computation.
  5. In the mosaic build with the option spread=1 the source flux is 
     slightly reduced (?10%) compared to
     the weighted average of the fluxes measured in the Science Window.
  6. The maximum number of sources handled by ii spectra extract is 200 
     but it is strongly recommended to
     only fit spectra of the sources that are effectively active 
     (visible, detectable) during the Science Window.
     In some cases, especially in the later part of the mission (as the 
     number of usable pixels has decreased),
     the maximum number of sources which can be meaningfully 
     reconstructed with ii spectra extract can
     be as low as 50.
  7. With OSA10.2, calibration files have been produced including a 
     correction for the variation of gain
     across the entire mission, as observed in previous OSA versions. 
     However, on single revolution time
     scale, a drift in counts is still observed. For the latest part of 
     the mission, spectra extracted at the
     beginning and end of a same revolution can therefore show an 
     artificial difference in counts. The secular
     drift observed in all bands over the mission life-time is known and 
     due to the evolution of gain: this
     effect is accounted for by the set of ARFs available in the IC tree. 
     With OSA11, this effect has been
     corrected, but other instabilities set in at instrumental level with 
     unpredictable paste (see above).
  8. The position of low-energy threshold is increasing with time (see 
     Sect. 12.4.1), due to the detector
     gain variation, which causes the energy scale to be more compressed 
     when expressed as function of the
     electric signal in the detector. 
     A safe lower limit for the response is 18 keV until revolution 848.
     Between revolutions 848-1090, we recommend to ignore data below 20 keV.
     From revolutions 1090 on, we recommend the user 
     to ignore data below 22 keV.
     Since revolution 1600, we recommend to exclude data below 30 keV 
     from the spectral analysis.
     For imaging, lower energy ranges can be used, down to the low 
     threshold of the instrument, which
     evolves from about 15 to 25 keV along the mission. However, the low 
     threshold and active status of
     pixels is changed at the beginning of each revolution, according to 
     the instrument status. Using energy
     ranges at the limit of detector sensitivity might introduce a strong 
     decoding noise in the image, due
     to the rapid variation of a pixel response with energy. To obtain 
     clean images and optimize detection
     of weak sources, we recommend to use a low threshold similar to the 
     one recommended for spectral
     analysis, which, however, slightly reduces the sensitivity to soft 
     sources. We note that a signal from
     the source is present al lower energies (see Fig. 32), but with 
     unstable response. Therefore, if one does
     not need an accurate determination of a source flux, but wants to 
     optimize sensitivity, more inclusive
     energy cuts can be made. The same considerations apply to light 
     curves.
  9. Merging products of OSA 10.2 and OSA 11 is not recommended. Spectra 
     should be fit separately,
     while images could be mosaicked only in overlapping energy ranges, 
     but with the caveat that count
     rates of a stable source might be different along the mission (see 
     Sect. 12.4.1). For instance, light curves
     could possibly show a discontinuity when passing from OSA 10.2 to 
     OSA 11. The analysis strategy for
     data sets overlapping with the transition should be decided on a 
     case-by-case basis, following scientific
     needs.
  10. A problem on-board IBIS causes event times to be shifted by 2 
     seconds under some circumstances (this
     is rare). The software tries to correct the data. The keyword 
     TIMECORR found in the event files (*-
     *-ALL or *-*PRP extensions), indicates whether the correction was 
     done. If you are doing an accurate
     timing analysis and your data contains TIMECORR>0 please take great 
     care: If TIMECORR=1 or 2, the applied correction should be OK. 
     If TIMECORR=3 you should better not use these data. If
     TIMECORR=4 contact ISDC.
  11. The lightcurve extraction (ii lc extract) is performed by building 
     shadowgrams for each time and energy
     bin. It potentially takes a large amount of CPU time and there is a 
     minimum usable time bin. The
     time bin must be such that the total number of maps in the file 
     isgr-corr-shad does not exceed 2 GB
     worth of disk space. The product of the number of time bins in a 
     science window, and the number of
     energy bands must be less than about 9942.
  12. ii_pif will crash if the input catalog inCat contains more than 500 
     sources.
  13. At large off-axis angles the IBIS response is not well known and 
     strongly energy dependent. Therefore,
     the user should be careful when analyzing observations performed at 
     large off-axis angles, above ?10
     degrees, since systematic flux variations might be introduced. The 
     systematic flux variations are energy
     dependent, and therefore the user should be careful both with 
     photometric and spectral analysis of
     sources at large off- axis angles.
  14. Due to lack of a consistent calibration information and the lower 
     end of the energy scale, absolute
     energy scale of ISGRI is poorly defined below ?60 keV. We recommend 
     systematic uncertainty on the
     absolute energy scale of 1 keV at 30 keV.
  15. Specifying energy range with (e.g. 20.1 - 40 keV) crashes the ISGRI 
     pipeline. It is recommended to
     choose round energy boundaries, e.g. 20 instead of 20.1 keV.

---
SPI 
---

  1. SPI is a complex gamma-ray instrument almost always dominated
     by background contributions. The scientific validation of the SPI
     data analysis going on at ISDC and at different instrument team
     sites is as of today far from complete. Users are encouraged to
     look critically at any result obtained with the ISDC software,
     and to use external comparisons and simulations when
     possible. Spurious results can be derived, for example, when
     using a wrong set of parameters and/or an incorrect background
     modeling.

  2. The SPI instrument is equipped with a Pulse Shape Analysis
     (PSD) electronic which carries out a parallel processing of the
     single detector events in order to provide additional information
     about their pulse shape. The PSD information was intended to help
     reducing the background. Unfortunately, the in-flight background
     conditions are such that even the best experts have failed to
     achieve significant improvements with the PSD. Consequently, all
     the PSD related processing is currently disabled in the analysis
     pipeline.  PSD events are simply used as standard single
     events. The basic user choice is then to analyze only single
     (+PSD) events specifying detector list of 0-18 in the analysis,
     or to consider double and triple detector interaction with pseudo
     detectors 19-84.
       
  3. Different instrumental responses are now included in our system,
     characterizing SPI before, between, and after, the detector 2, 17,
     5, and 1 failures. The spi_science_analysis pipeline cannot
     currently handle a time variable response. The easiest is to
     analysis the possible cases independently (our software then selects
     automatically the appropriate response), and to combine the results
     later on. It is possible however, to analysis different mixtures of
     different data together using one of the three responses as they are
     not too different. Great care should be taken in this case anyway to
     avoid possible biases (see the Tips and Tricks section of our
     documentation). The spimodfit analysis can instead handle multiple
     responses which are appropriately used during the data processing.
     The final response accounts for the multiple responses accordingly.

  4. The "spiros" imaging software is quite a complex tool with many
     different options and parameters. Not all possible cases have been
     fully scientifically validated. The best tested modes include
     "imaging" and "spectral extraction". Other modes such as "timing"
     and "spectral timing" and other background methods are being further
     tested and validated. The spimodfit software is an on-going project
     which has now reached a stable configuration, but not all the
     features have been scientifically tested.

  5. At least in one case, a long (staring) pointing which is split
     up into several science windows in the ISDC system is not handled
     correctly in the SPI data analysis. It concerns: ScWs
     008200220010.001 008200220020.001 008200220030.001. Only the
     first pointing is properly included in the analysis, while the
     subsequent ones are ignored. Please report if you find any other
     such cases.

  6. The "spiros" lightcurve production has shown some crashes when
     running large data-sets and a time binning of one ore more days is
     selected. The program handles correctly a resolution of one Science
     Window, however, so the user is encouraged to use this finer time
     binning and merge the results afterwards in case he/she finds
     similar problems during the analysis of a long data-set.

  7. "spimodfit" handles time variability through the use of splines.
     The spline order can be 0 to 5, 0 corresponding to a piecewise
     constant function (with one scaling parameter per interval) and 3
     corresponding to cubic splines. In many cases, when using n-order
     splines (with n equal or greater than 1), the fitting algorithm
     fails to find the optimal parameters. This is thought to be due to
     over-parametrized time variability because of the additional
     parameters of the splines. In addition, crashes of "spimodfit" have
     been reported when using a large dataset, with about 1000 pointings
     or more, their origin is still unclear and is under investigation.

-----
JEM-X
-----

ISSUES can be classified as

1. Detector gain variations.
2. Near-Real-Time data.
3. Cross-talk between sources.
4. Grey-filter artefacts.
5. Flux extraction methods.
6. PIF-option and mosaic_spec.
7. Data from early revolutions.
8. Restricted imaging data. 


  1. Detector gain variations.
     The JEM-X detector gain varies significantly for a few hours after the 
     instrument has been switched on. These effects are normally taken care of 
     for the CONS(consolidated) data by IC Gain History Tables which are now 
     produced on a revolution by revolution basis by the JEM-X support team. 
     CONS generally has an energy determination within +-2% for spectra 
     integrated over an entire science window and can be used for all 
     energy-sensitive applications.  Noisy revolutions with poorer energy 
     determination do occur occasionally and users should check the JEM-X 
     calibration page to get an overview of the goodness-of-energy 
     -determination for the revolutions they are looking here .

     For help fitting data in complicated revolutions contact Dr. Carol Anne 
     Oxborrow at Ôoxborrow@space.dtu.dkÕ.

  1. Near Real Time data.
     When analysing Near Real Time data (i.e. within a few hours from the data 
     transmission to ground, and well before the entire revolution is 
     completed) the JEM-X gain calibration is not available. The NRT data from 
     JEM-X are produced prior to the generation of these IC-tables and can 
     therefore only be used for non-spectral applications: e.g. lightcurves 
     covering all energies, source location, mosaicking with a single energy 
     bin etc. For NRT data we suggest to run the analysis with parameter 
     "COR_gainModel=2" to force the use of a simplified fitting model. The 
     default model (COR_gainModel=-1) can be used again at revolution 
     completion.

  2. Cross-talk between sources.
     It is a fundamental property of coded mask instruments, that every source 
     in the field of view contribute background for all other observed 
     sources. In this sense cross-talk is unavoidable. Specifically, strong 
     bursts from one source will introduce Poisson noise in the light curves 
     for all other sources. The new light-curve extraction method introduced 
     with the OSA 11 release provides very good separation of the sources, but 
     cross talk cannot be fully eliminated. An important new feature is an 
     automatic search for bursts in the count rate records. This allows to 
     identify the bursting source (or provide evidence that the burst is 
     caused by a fluctuation in the instrument background). (See further 
     details in the JEM-X Analysis User Manual). The JEM-X lightcurves are 
     deadtime corrected. The DEADC parameter in the light curve files are 
     therefore set to 1.0 (for XRONOS compatibility).


  3. Grey filter artefacts.
     A count rate limiting mechanism, the grey filter, is activated when the 
     total count rate exceeds the telemetry capacity. (With the current 
     telemetry allocation this corresponds to about 100 counts/s or about 1.5 
     Crab). At the beginning og each science window the grey filter is set to 
     zero (no action). If needed, the grey filter will adjust itself 
     automatically, according to the filling level of the onboard telemetry 
     buffers. Ideally, the grey filter should reject events in a completely 
     random way. However, the mechanism implemented is only pseudo-random. 
     Therefore, some care should be taken in interpreting power spectra of 
     arrival times of events from bright sources affected by significant grey 
     filtering, as QPO-like artefacts may show up. As the grey filter varies 
     over a science window and the artefacts are specific to each particular 
     grey filter setting there may be some "averaging" out of the power 
     spectra artefacts. Anyway, if noticing transient features in the power 
     spectra of very strong sources it should be checked if this is limited to 
     a period of a specific grey filter setting. Please check the JEM-X 
     Observers Manual for further explanations.

  4. Flux extraction methods.
     The flux of a given source can be obtained either with the "standard" 
     extraction or with mosaic_spec. In cases when the fluxes obtained with 
     the two methods differ, it is advised to consider the one obtained from 
     the standard extraction (SPE level).

  5. PIF-option and mosaic_spec.
     For the time being it is not trustable to extract spectra of strong 
     sources with mosaic_spec from images obtained with the PIF-option.  

  6. Data from early revolutions.
     Due to changes of the on-board configuration at a number of occasions, 
     the detection efficiency has changed significantly several times during 
     the mission history. In particular, for all pointings between revolutions 
     26 and 45, this means that the measured fluxes of sources -  in 
     particular fluxes at low energy - will strongly depend on the specific 
     data taking time. 

  7. Handling of Restricted Imaging data. 
     The use of the ÔRESTÕ telemetry format has been discouraged since 2004. 
     It has turned out to be difficult to interpret data taken with this 
     format, specifically the energy resolution is very poor. Analysis of 
     REST-format data is only supported in OSA-5 or earlier versions. 


---
OMC 
---
     
  1. The automatic extraction of fluxes and magnitudes produces
     reliable results only for point-like sources.

  2. For extended sources or high-energy sources with
     large uncertainties in their position, the OMC planning assigns
     multiple adjacent sub-windows to cover the whole area. In that
     case, multiple boxes are found with different ranks but with the
     same OMC ID.

     
     From OSA 6.0 onwards these mosaics of sub-windows can be correctly 
     analysed by using IMA_wcsFlag=yes (default in current OSA), once the 
     coordinates are well defined (e.g., from X-ray observations). 
     In this case, o_src_get_fluxes creates a virtual 11x11 pixel 
     sub-window inside the whole area centred at the source position given
     in the OMC Input Catalogue. After that, OSA works on this new sub-window 
     and ignores the previous windows of the mosaic. 
     This is an internal software trick; these virtual sub-windows 
     do not exist as standard sub-windows (o_ima_build, for example, 
     will not create these virtual sub-windows as images of 11x11 pixels). 
     Note that with IMA_wcsFlag=no, these mosaics of sub-windows will not 
     be analysed correctly as the software treats each box individually. 
     Users should also note that even with IMA_wcsFlag=yes, for those 
     new sources which are not yet included in the OMC Input Catalogue, 
     these virtual sub-windows cannot be created because the software 
     extracts the coordinates from the OMC Input Catalogue. 
     In addition to this method, 
     the observer may extract the optical photometry manually from 
     the corrected images produced by the analysis pipeline.

  3. If the source coordinates are inaccurate by more than 2 OMC
     pixels (~35"), the analysis software will not be able to
     re-centre the target. As a consequence, 
     fluxes and magnitudes derived
     using default parameters will not be correct.

  4. If another star is within a few pixels from the source of
     interest, it can introduce systematic errors in the derived
     fluxes and magnitudes. The strength of this effect can be
     different for different pointings, since the relative position in
     the sub-windows will change slightly for different rotation
     angles.

  5. Some of the bright sources
     slightly saturating one or a few pixels might not be detected as
     saturated sources. As a consequence, their derived magnitudes might
     not be correctly computed. The observer should check whether the
     source could be saturating the CCD for a given integration time,
     and re-analyze the data rejecting the shots with the longest
     integration times.

  6. Due to thermoelastic deformations, the alignment of the OMC 
     optical axis with the spacecraft attitude reference (after 
     correcting for the known OMC misalignment) may diverge 
     by up to 30" (~2 pixels). 
     This is corrected for in the analysis (OSA 5 upwards) using the photometric 
     reference stars, giving an accuracy better than 2" in most cases.