Hitomi processing version : List of Changes

  • Pipeline "Version 03.01.006.008 Oct 02, 2017"
  • Pipeline "Version 03.01.006.007 Apr 25, 2017"
  • Pipeline "Version 03.01.005.005 Dec 23, 2016"
  • Pipeline "Version 02.01.004.004 Aug 20, 2016"
  • Pipeline "Version 01.02.003.003 Jul 08, 2016"
  • Pipeline "Version 01.01.003.003 Jun 24, 2016"


  • Current Pre-pipeline/Pipeline version


  • Pipeline Release Note Oct 02 2017
  • 
     The input data to the pipeline for the final processing are the outputs 
     of the pre-pipeline run in Dec 2016 
    
    =====================
    Pipeline 
    =====================
      - Uses the updated CALDB for SGD (sgd20170926) 
      - Uses the Hitomi software version 6 and the latest HeaSOFT release
      - Only the sequences containing the SGD data were processed. The others
        are from the processing run in Aprl 25 2017. The sequences affected are :
           100050020
           100050030
           100050040
           100043050
           100043060
           100044010
    
    
    
    

  • Pipeline Release Note Apr 25 2017
  • 
    The input data to the pipeline for the final processing are the outputs 
     of the pre-pipeline run in Dec 2016 
    
    
    =====================
    Pipeline 
    =====================
      - Uses the updated Caldb for the SXI and SGD and the new SGD screening criteria
      - Uses the Hitomi software version 6 and the latest HeaSOFT release
    
    
    
    

  • Pre-Pipeline & Pipeline Release Note Dec 28 2016
  • =====================
    Pre-Pipeline 
    =====================
    
      -  Attitude file is updated. The aberration function of STT telemetry is corrected. See the description of attitude
         (jaxa_hitomi_memo_2016-001_v30.pdf) for detail. The format of L32TI in ATTITUDE extension was corrected.
     
       - The TIME column in the SXI event fits is now corrected; SEQSTARTTIME is newly defined as SEQSTART_TLM - 4.0 [sec] and
         now aligned with the SXI EXOPOSURE. The event file now contain a new column SEQSTART_TLM defined as the telemetry value 
         of the packets associated to the event data.
     
       - Minor bugs in pre-pipeline process are fixed
    	a) TIME and DATE-OBS/DATE-END in SXI hot pixel fits (MZDYE) are filled.
    	b) DATE-OBS/DATE-END in SGD burst fits are properly filled. (in 2nd run it was set to be ‘2014-01-01T00:00:00’)
    	c) Remove HXI/SGD Shield FITS with blank extension.
    	d) Fix fverify errors on the HK_PSP_SYSLOG extension of SXS HK1 fits.
    	e) Incorrect duration is excluded from GTIPOINTING extension in
    	   OBSID=000008030.
    
       - Fix the telemetry saturation GTI so that durations with no science events due to telemetry packet skips are 
         taken into account. Following OBSIDs are affected (numbers in parentheses are the excluded exposure times 
         in unfiltered events).
    	* SXS: 000006020 (~68 s), 100040060 (~32 s), 100041020 (~5451 s), 100043010 (~3 s), 100043020 (~803 s),
    	* SGD: 100043050 (~1 s), 100043060 (~3 s), 100044010 (~1 s), 100050030 (~1 s), 100050040 (~0.03 s)
    	* SXI, HXI: none
    
       - Update the calculation of the telemetry saturation gti for HXI to include the time when CdTe pixel show flaring phenomena.
         The flaring activity appeared only one pixel (RAWX, RAWY)=(97,55) in HXI2 for few seconds (i.e., the GTI of the clean events
         becomes a few seconds shorter than the previous).
    
       - TSTART/TSTOP are properly updated and resulted in the longer exposures of G21.5-0.9 3rd/4th observations (100050030, 100050040) 
          by 8 ks and 19 ks, respectively.
     
       - Fix the SXI event treshold in the RXJ observations  (partially in the pre-pipeline, and partially in the pipeline)
    
    
    ==================
     Pipeline 
    =================
    
       - GTI calculted within th eprocessing are now appended to the uf file:
          * EHK, MKF, and NXB GTI to uf files for all instruments
          * EHK DRK and ADR OFF GTI to SXS uf files
          * EHK DAY GTI to SXI uf files
    
       - Create an additional SXS antico light curve with a bin size of 20 seconds in the PI range 60-12200; 
         create a GTI from this lightcurve using the expression,"RATE_SAA <= 3" and append it to the SXS uf files
    
       -  Bug-fix: fix creation of a fake SXS lost GTI file when the file was not included in the ingest telemery data and now used in the 
          screening ; Add the PIXEL column is fake SXS lost GTI
          
       - Behavior change: perform proximity flagging in the second run of sxsflagpix instead of the first run
    
       - Behavior change: remove the EHK2 output file; the EHK file is replaced by EHK2 if the latter is successfully created
    
       - Behavior change: change default values for SXS expressions; PIXELALL2 to PIXELALL3 and PIXELNXB2 to PIXELNXB3
    
       - Behavior change: decrease default tolerances for sxsgain and ahgainfit; r2tol from 0.01 to 0.001; 
         bisectolshift from 0.1 to 0.001; bisectolwidth from 0.2 to 0.001
    
       - Bug-fix: change the cpkeyword parameter value in ahscreen calls from 'yes' to 'ALL'; the former value is not valid
    
       - Behavior change: change Perseus stop date from 2016-03-08T00:38:00 to 2016-03-04T00:41:00; with the new stop date, 
         only the first two Perseus sequences run the sxsperseus task
    
       - Behavior change: write the following keywords to all output HDUs:TIMESYS, TIMEUNIT, TIMEREF, TASSIGN, GPSOFFET, CLOCKAPP
    
       - Behavior change: have ahgtigen update timing keywords on output GTI extensions.
    
    


    Past Pre-pipeline/Pipeline versions


  • Pre-Pipeline & Pipeline Release Note Aug 20 2016
  • =====================
    Pre-Pipeline 
    =====================
    
    Common
      - Version number TLM2FITS='001.013.003p4.003'
      - OBSID 100050050 is merged into 100050040
      - OBID 000004020 is splitted in  000004021 000004022 000004023
      - Illegal DATE-OBS are fixed in all extensions 
      - running new code for sxssamcnt (to assign time of lost events in science file, and account for WFRB 
        in the first lap and TRIGGER_LP in the last lap) ahtime and akmktim to handle the L32TI reset  
      - Fix the slew/pointing GTI for OBSID= 100041010
      - Use gen gti prepared by SCT GSFC for OBSID=100042030 
      - Delete all the empty FITS files listed in 1st run 
      - Time coverage of ATT/ORB are checked (Orb is made with -1/+1 Day from the START/STOP of the attitude) 
      - S_TIME fixed in all the HK which was 1 sec late
      - Extensions that have columns useful for check the STT status are included in ahXXXXXXXXgen_a0.hk 
      - Sort the HK lookup table by the local time  
      - New attitude released 
    
    SXI
      - hot pixel files are created for all sequences even when no dark frame was taken
      - bug fixed in generation of hot pixel fits for OBSIDs=100041010,100043010-100043050 and 000008010-000008040
      - fix sxi gti for MZDYE 
      - fix sxi gti for OBSID= 100040060 to exclude times with no events 
      - fix keywords in SXI exposure FITS for OBSID= 000006020 and 000007020 
      - fverify errors in SXI exposure FITS are fixed 
      - SXI : changed the header timing keywords and mode to properly process the Crab data
    
    SXS
      - pxfff files are not longer created. These files contain events with TIME set to null. The new sxssamcnt assigns time for these events
      - Fix the content of the DATAMODE keywords in the Pixel WFRB FITS that prevent to assign time correctly 
      - Added the keywords INSTRUME/DETNAM/DATAMODE/TSTART/TSTOP/EXTNAME in SXS EL of OBSID=100040030 and 100040040 
      - MXSONOFF keyword in SXS pixel FITS of OBSID=100044010 was fixed 
      - Fix a bug that prevent columns in the SXS diagnostic files to be populated correctly
    
    HXI
      - Run ahtime with the proper parameters for 'lookupfile' for CAM SCL HK extension of HXI 
      - Fix bug in generation of HXI events FITS for OBSID=000007020 and OBSID=100042040 
    
    OPEN 
    - SXS HK1 non-ascii code
    - attitude description and ATTGTI
    
    ==================
     Pipeline 
    =================
    
    General:
       - Update the software version to Hitomi vs 4  and calibration version Hitomi vs 4 
         SOFTVER= "Hea_09May2016_V6.19_Hitomi_18Aug2016_V004"
         CALDBVER= "gen20160812_hxi20160812_sgd20160614_sxi20160614_sxs20160715"
       
       - Use the exit status to ouput diagnostic messages for ahgtigen and ahscreen to 
         distinguish an error in the GTI creating from a normal execution that produces empty GTI files
        
      HXI:
       - Calculate a new column, HXI#_TRIGGER_RATE, in the extension name HK_HXI#_CAM_SCL of the HXI HK files
       - Used new screening criteria to account the high background region and PI cut
    
       SGD:
       - No changes
    
       SXI:
       - No longer screened event files with erasing mode (Dataclass=100000b0 and 100000b1)
       - No longer screen NXB, DE or cal source for minus z day-earth (MZDE) data (Dataclass ending in 1). They are always empty. 
       - Set the new sxiflgpix paramaters to flag cosmic ray echo events  
         (echoflag=yes, echomap=yes echonbr=2, echomin=6 echospth=15 echofrac=0.7). Bit 37 in the STATUS column ID the echo event.  
       - Calculate the MZDE gti and the gti extension to bad image files (output of sxiflagpix) 
       - Change : sximodegti is run only if an SXI exposure file is present in the sequence
       - Use the new screening criteria to include the MZDE  and status bit 37  
        
       SXS:
       - Rename GTILOSTOFF extensions to GTIFOUNDALL (this GTI includes times to when all pixels are on
         constructed from the pixel dependent LOST GTI) 
       - Remove GTILOSTOFF screening for pixel 12 and Fe55 as for the science event data files
       - Changed name of SXS pixel 12 file, e.g. px12 changed in pxcal (to not confuse submode numbering to pixel 12 numbering) 
       - No longer produce pixel dependent (36 extension) GTI file when running sxsgtipix. The 36 gti exension file is not used
         in the pipeline. 
       - Use PIXELALL2  and PIXELNXB2 labels for cleaning events.The new screening includes: 
         a) add event selection on TICK_SHIFT, b) SLOP_DIFFER selection is energy dependent, c) add new STATUS selection to eliminate 
         event within the "lost event gti", d) PI >600  
       - Calculate the PI column also for pixel 12 event file 
       - New paramaters and new  setting for sxsflagpix. New setting: antphathr=71  antdurthr=2 calctl2=yes gtifile=" set to the
         gti losts"  Newparameters: pxpithr=600 and usepithr=ALL, Change resetflag from boolean to string and set to All  
       - New paramater in sxssecid pxpithr=600 and usepithr=ALL
       - Change the paramater setting from no to yes of calccttel2 when running sxsflagpix 1st time  
       - Calculte the SXS antico lightcurve with PI > 60 binsize =1 sec and add the columns (ANTRATE and ANTERROR) 
         in the ehk file so to be plotted together with the SAA.
    

  • Pipeline Release Note Jul 8 2016
  •     - sxsgain:
          * Behavior change: added time validity check for setting 'extraspread=100' 
            parameter in sxsgain when processing Perseus data (sequence 100040010)
    

  • Pipeline Release Note Jun 24 2016
  • First release