Xronos
LCSTATS (Jan96)               xanadu.xronos              LCSTATS (Jan96)



NAME
    lcstats   --  calculates  statistical  variables  for  1  input time
    series and             prints on the screen the results
    
    
USAGE
    lcstats file(s)+options window dtnb nbint
    
    
DESCRIPTION
    This task performs statistical analysis for  one  times  series  and
    prints  the results on the screen (no output is produced). The input
    file format is FITS using the BINTABLE extension. Both  binned  data
    format  and event format are input. Data can be rebinned and divided
    into  Intervals  (See  GENERAL  XRONOS  TERMINOLOGY).  Time,  Phase, 
    Intensity   and   Exposure  windows  (See  WINDOW)  allow  for  data 
    screening. The quantities calculated are : newbin integration  time;
    interval  duration;  number  of  good  accepted newbins in interval;
    average (and its error); standard  deviation;  minimum  and  maximum
    count   rate  per  interval;  variance  (amnd  its  Gaussian  error) 
    evaluated from the data scatter; expected variance from  a  constant
    source  (and  its  Gaussian  error), as evaluated from the errors in
    the newbin count rate; third  moment;  average  absolute  deviation;
    skewness  (and  its  Gaussian  error); RMS fractional variation (and
    its Gaussian error) or a 3 sigma upper limit if the variance is  not
    larger  than the expected variance at a confidence level higher than
    99.86%; Chi-square  and  the  corresponding  number  of  degrees  of
    freedom;  constant  source  probability associated to the Chi-square
    value; constant source probability from a Kolmogorov-Smirnov test.
    
    
GENERAL XRONOS TERMINOLOGY
    Within XRONOS tasks, BINS and NEWBINS control the  binning  used  in
    the  analysis,  INTERVALS  the  subdivision  of  the time series and
    FRAME the grouping of the output results:
    
    BINS : these are the time bins of the time  series  being  analysed.
    More  than one input file can have different bin durations, e.g. two
    consecutive time series, one with 0.5 s bins and the other with 2  s
    bins.  The  original  bin time is the value stored in the input file
    in the keyword TIMEDEL. If the data are stored in  each  row  as  an
    array  with  1CTYPn  =  'TIME', the original bin is set to the value
    stored in the keyword 1CDLTn (where n is the column number).
    
    NEWBINS : these correspond to  the  time  resolution  at  which  the
    analysis  is  carried out.  Note that: (i) newbins cannot be shorter
    than the longest bin duration of the  time  series  being  analysed;
    (ii)  in many XRONOS applications (e.g. powspec, autocor, crosscorr)
    the newbin duration is forced to  be  an  integer  multiple  of  the
    longest bin duration.
    
    INTERVAL  :  an  interval  is  defined by the number of newbins over
    which the analysis is carried out. Note that in  applications  using
    FFT  algorithms  (e.g.  powspec,  autocor  and  crosscor set in fast
    mode) the number of newbins in an interval is a power of 2.
    
    FRAME : a frame consists of  the  average  of  the  results  of  the
    analysis   of  one  or  more  contiguous  intervals.  Note  that  in 
    'lcurve', 'efsearch' and 'lcstats' a frame consists  always  of  one
    interval.
    
    
WINDOWS 
    If  any  window  is  required  during  the  analysis,  a window file
    containing  the  relevant  windows  must   be   created   with   the 
    application  XRONWIN,  before  running  a  XRONOS task.  There are 4
    different types of windows :
     * Time Windows  : consist of up to 1000 time intervals                  
     * Phase Windows : consist of an Epoch, a Period and up to 10 phase intervals
     * Intensity Windows : consist of up to 10 intensity in bin, newbin and interval
     * Exposure Windows : consist of up to 1 exposure in bin, newbin and interval 
    Intensity and Exposure Windows can be specified  independently  for:
    (i)  Bins  , (ii) New Bins , (iii) Intervals. When dealing with more
    than one  time  series,  Intensity  and  Exposure  Windows  must  be
    specified  separately  for  each series.  Time and Phase windows are
    applied to Bins. Intensity and Exposure windows  are  applied  first
    to  Bins,  then  Newbins and finally to Intervals as specified.  For
    time and phase windows, only those bins whose center time is  within
    the  start  and  stop  of  a  time  window  or  phase  window (for a
    specified epoch and period) are accepted. Intensity windows must  be
    ordered  with  increasing  intensity  and  if set for newbins can be
    used in conjunction  with  "Special  Newbin  Windows"  (see  below).
    Exposure  Windows consist of a minimum and a maximum exposure level.
    Units are such that 1 means 100% exposure. The  Newbin  Exposure  is
    obtained  by  propagating  the  bin  exposures  to  each newbin. For
    example, if in a 30 s newbin the total exposure (due to the  sum  of
    the  individual  exposure  of  the  bins  contributing  to the given
    newbin) is 18 s then its exposure is 60%. The Interval  Exposure  is
    the  ratio  of  accepted  to expected newbins: for example, if a 128
    newbin long interval contains only 32  accepted  newbins,  then  its
    exposure  is  25%. Many XRONOS application use some default exposure
    windows, which are designed to avoid analysing data sets  which  are
    too  inhomogeneous with respect to their statistical properties. The
    minimum default Exposure windows in an Interval is  set  to  0.0  in
    the  lcurve,  efold  and efsearch and to 0.5 (i.e. 50% exposure ) in
    all the other tasks. Note that exposures can  be  higher  than  100%
    (e.g.  if  the  newbin  time is not a multiple of the bin time, then
    "beats" are generated which might bring the exposure of a newbin  to
    values  >100%;  or  if  two  or  more  input files for the same time
    series overlap in part, some of the newbins will be more  than  100%
    exposed).   IMPORTANT  NOTE  WHEN TIME WINDOWS ARE SET IN THE WINDOW
    FILE: The time used within XRONOS tasks  is  Truncated  Julian  Days
    (TJD=  JD-2440000.5)  if either (1) the keyword MJDREF is present in
    the header or (2) if the TIMESYS  value  is  one  of  the  following
    strings  MJD  or JD or TJD.  If (2), the time values are expected to
    be stored as JD, MJD or TJD in the header keywords and in  the  TIME
    column  in  which case the MJDREF keyword is not used (it should not
    be present).  When Time windows are set using XRONWIN, they must  be
    compatible  with  the values in header of the timing keywords and/or
    the values in the TIME column.
    
    An additional window type called "Special Newbin Window" can be  set
    directly  from  the  parameter file. Special Newbin Windows are used
    to exclude the parts of a light curve which  immediately  follow  or
    precede  a  burst  or  a background event which has been rejected by
    intensity  windows  in  newbins.  The  Special  Window  operates  on 
    newbins  in  conjunction with intensity windows (in newbins) and are
    specified  by   changing   to   positive   values   the   parameters 
    'spwinbefore'  and  'spwindowafter'.  Their use is the following: if
    e.g. spwinbefore is set =10,  all  newbins,  whose  center  time  is
    within  10  second  before  the  center time of a newbin rejected by
    intensity windows, will also be rejected; if e.g.  spwindowafter  is
    set  =20,  all  newbins, whose center time is within 20 second after
    the center time of a newbin  rejected  by  intensity  windows,  will
    also be rejected.
    
    
FILELIST and INPUT FILE OPTIONS
    To  input multiple files for each time series, a file containing the
    list of files is needed (Filelist). The Filelist  is  input  in  the
    program  as  '@Filelist'.  The format of this file list is ascii and
    contains one filename+options per line. Files from  different  times
    series  are  separated  by  '///'  mark.  Below is an example of the
    Filelist containing 2 files for 3 different times series.
     file1_ser1
     file2_ser1
     ///
     file1_ser2
     file2_ser2
     ///
     file1_ser3
     file2_ser3
    The Input File Options (up to 10) can be specified for each file  in
    the  same  input string.  They consist of 2 characters followed by a
    numerical constant (up to 8 character long).  There are  two  groups
    of   options.   The  first  allows  data  selection  within  a  FITS 
    extension. The available options within this group are :
     frN= start reading input file from row number N (first row)
     lrN= stop reading input file from row number N (last row) 
     vxN= use column number N as x-axis (i.e. time axis, default name is TIME)
     vyN= use column number N as y-axis (default names are COUNT or RATE)
     vsN= use column number N as error for y-axis (default name is ERROR)
     veN= use column number N as exposure (default name FRACEXP).
          If the input file is an event list, exposure is by default
          calculated using the GTI extension. In this case, N=0
          turns off the usage of the GTI extension for the exposure
          calculation, and N > 0 specifies the GTI extension to use.
     feN= select data (either binned or events) from channel number N (First Energy). 
          For an event list channel selection is made using the column named 'PHA'
     leN= select data (either binned or events) to channel number N (Last Energy). 
          For event list the default column channel name searched is 'PHA'.
          The option 'vcN' allows the choice of a channel column name different from 'PHA' (es. 'PI').
     vcN= use column number N for channel selection (valid only for event lists). 
     rtN= use extension N of the FITS file to read the data. The first extension 
          is N=1 (the primary array is irrelevant). To specify the extension the 
          following also can be used: filename[N] or filename+N.  
     of = The MJDREF keyword is not used. The time is calculated using the
          TIME column and the TIMEZERO keyword.
    
    The  second  group  of  options  performs  algebraic  operations  on 
    individual  input  files.   They  are  applied  in the same order in
    which are specified. For event files  they  are  applied  after  the
    data are binned. The available options within this group are:
    
     stX = Shift all Time in input file by X days 
     ssX = Shift all times in input file by X Seconds 
     muX= multiply data and errors by X (MUltiply)
     mdX= multiply data by X (Multiply Data) 
     meX= multiply errors by X (Multiply Errors
     maX= as muX but exposure is divided by X 
     diX= divide data and errors by X (DIvide)
     ddX= divide data by X (Divide Data) 
     deX= divide errors by X (Divide Errors) 
     daX= as diX but exposure is multiplied by X
     aaX= add data and errors with X (Add All) 
     adX= add data with X (Add Data) 
     aeX= add errors with X (Add Errors) 
     saX= subtract data and errors with X (Subtract All)
     sdX= subtract data with X (Subtract Data) 
     seX= subtract errors with X (Subtract Errors) 
     qaX= add to data the square of data muliplied by X and add to errors 
          the product of data and error multiplied by X 
     qdX= as above but for data only
     qeX= as above but for error only 
    Below  is  an  example  of  the  Filelist  containing  2 files for 3
    different times series where the different options  are  applied  to
    the input files for different time series.
    
     file1_ser1 aa4           add to data and error 4 
     file2_ser1 aa4            "      "        "    "
     ///
     file1_ser2 rt2 aa2       read 2nd extension; add to data and error 2 
     file2_ser2 rt2 aa2        "    "     "           "      "        "    "   
     ///
     file1_ser3 rt2 vy4 vs5   read 2nd extension; use column 4 and 5 for Y-axis and Error
     file2_ser3 rt2 vy4 vs5    "    "    "         "    "    "     "  "   "          " 
    
    
PARAMETERS
    
    cfile1 (filename(s) first series+options) [string]
        Input  filename(s)  for  the  first  time  series + options. The
        valid  input  files  are  in  FITS  format  using  the  BINTABLE 
        extension.   Xronos   tasks  read  for  each  time  series  many 
        consecutive input files (up to 50).  Additional  flexibility  is
        provided  by  Input  File  Options  which  are  used  to perform
        algebraic operations on individual input files  (either  on  the
        'times'  or  on the 'count' or 'count/s' values). The Input file
        Options are also used to select columns and rows within  a  FITS
        file.   If  the  first character of the input string is '@', the
        rest of the string is taken to  be  a  filename  containing  the
        list  of  input  files  (Filelist).  The  Filelist  can  contain 
        filenames  for  more  than  one  series.  See   description   of 
        "FILELIST and INPUT FILE OPTIONS".
    
    window (name of window file) [string]
        Filename  of the xronos window file. The window file is an ASCII
        file and by default a standard window file is used,  where  only
        exposure  windows  are  set.   To  modify  the  standard file or
        create a new file used the script XRONWIN.
    
    dtnb (integration time) [double]
        The duration in seconds of the NEWBIN  time.  For  binned  input
        files  the  NEWBIN  duration can not be shorter than the longest
        bin duration in the input file. In a number of XRONOS tasks  the
        NEWBIN  time  must  be an integer multiple of the minimum newbin
        time.   The  task  internally  calculates  (and  prints  on  the 
        screen)  a default value such that a single interval is produced
        with a fixed number of newbins (typically between 128  and  4096
        depending  on  the task and on the time interval length see also
        "nbdf" parameter).  Typing 'INDEF' forces the  task  to  use  as
        NEWBIN  time  the  value  calculated  by  the  program. NOTE: By
        pressing return the task will  use  as  NEWBIN  time  the  value
        found in the parameter file used in a previous run.
    
    nbint (number of points per interval) [integer]
        The  number  of  newbins  per interval used in the analysis. The
        "nbint" together with the NEWBIN duration determines the  length
        in  time  of  an  interval  and  therefore  the  total number of
        intervals  within  the  start  and  stop  time  over  which  the 
        analysis  will  be  carried out.  Typing 'INDEF' forces the task
        to use the default  value  (see  parameter  "nbdf").   NOTE:  By
        pressing  return  "nbint"  is  set  to  the  value  found in the
        parameter file used in a previous run.
    
    itre (Flag for trend removal) [integer]
        A polynomial trend, up to 4th-order, can be removed  from  input
        time  series.  Setting the parameter "itre" equal to 1 or 2 or 3
        or 4  remove  a  1st,  2nd,  3rd,  4th-order  polynomial  trend,
        respectively.  The  trend  is  determined  separately  for  each 
        interval of each series by using a least-square  technique.  The
        value  0  does not cause the removal of any trend from the input
        series and is the default value.  NOTE The trend removal is  not
        available for the efold and efsearch tasks.
    
    itremo (Mode for trend removal) [integer]
        Specify  how the trend removal is applied to the data (available
        only id "itre" is higher than 0). The trend can  be   subtracted
        from  the  time  series  (itremo =1) , or the time series can be
        divided by the trend (itremo = 2), or the  time  series  can  be
        replaced  with  the  trend (itremo = 3). By default the trend is
        subtracted (value set to 1).
    
    tchat (terminal chattiness) [integer]
        Set  terminal  chattiness:  (0-4)  only  little  information  is 
        output  in  running  XRONOS  task  ; chattiness 5 is the default
        value; (6-7) more details on  input  files,  windows,  intervals
        statistics, etc.; (<8) mostly for debugging purposes.
    
    lchat (log file chattiness) [integer] 
        Set  log  file  and chattiness in the log file: = 0 the log file
        is not written; for all other values, information is written  in
        the  log  file.  The  chattiness  levels are the same as for the
        terminal.
    
    logname (log filename) [string]
        Name for the log file. The default name is xronos.log.
    
    clobber  [boolean]
        Flag specifying whether or not  a  pre-existing  file  with  the
        same  name  as  that requested for an output file in the current
        task will be overwritten. Default value = yes.
    
    (dpath = XRDEFAULTS) [string]
        This string parameter gives the path to  the  Xronos  'defaults'
        directory,  which  contains  the  default  '.pco' file (used for
        plotting)  and  the  defaults  window   file   'default_win.wi'. 
        Ordinarily,  the user may leave this parameter set to the string
        'XRDEFAULTS',  which  causes  Xronos  to  use  the   environment 
        variable  XRDEFAULTS to locate these files. XRDEFAULTS is set by
        the mkftools script to point to the  appropriate  directory  for
        the  current  distribution  of  Xronos  (for FTOOLS v3.6 this is
        /ftools/xronos/defaults/).  If the user wishes to  modify  these
        files,  he  or  she  may  make  and  edit copies, and change the
        XRDEFAULTS  variable  appropriately  using   setenv,   but   the 
        original files should not be changed.
        
        gapfill (running mean gap filling) [integer]
            Replace  gaps in input series with running mean.  If =0 (the
            default) data gaps are not filled.  If =n newbin  data  gaps
            in  input  series  are  filled  in  with running mean values
            calculated over n newbins. Note that a gap newbin is  filled
            in  only  if  the  corresponding  running mean is calculated
            over n/4 points at  least  (this  means  that  in  order  to
            bridge  a  gap  of m newbins n must be >1.35m).  This global
            parameter is ignored in epoch  folding  applications  (efold
            and efsearch).
        
        forcestart (flag for start time) [boolean]
            If  =  yes the first interval will be forced to start at the
            time of the first time window otherwise  (=no  default)  the
            center  time  of  the  first qualified newbin is used as the
            start time.
        
        errorbars (Error bar Evaluation) [integer]
            This parameter defines the way in which the  error  bars  of
            the  analysis  results  are calculated. If the number of the
            intervals per frame ("nintfm") is  higher  than  "errorbars"
            value  (default=5),  the  error  bars are evaluated by using
            the  standard  deviation  of  the  average  (based  on   the 
            measured  scatter).  Otherwise  the error bars are evaluated
            by  propagating  the  theoretical  error  bars  through   an 
            averaging   process.    Note   that   for   several   XRONOS  
            applications (e.g. `autocor`, `crosscor`)  only  the  former
            way  of evaluating error bars is available.  For example, if
            "errorbars" is 5 in the  application  `powspec`:  (a)  if  a
            frame  contains  the  average  of  5 or fewer power spectra,
            then the error bars in the average power  spectrum  will  be
            calculated   by   propagating   through   the   average  the 
            theoretical error bars associated with each  power  spectrum
            (in    turn    obtained   from   the   relevant   chi-square  
            distribution); (b) if a frame contains the average of  6  or
            more  power  spectra,  then  the  error  bars in the average
            power  spectrum  will  be  calculated  by   evaluating   the 
            standard  deviation of the average power for each frequency.
            By adjusting the value of "errorbars" it is  possible,  e.g.
            to  evaluate error bars as in (a), also in the case in which
            a large number of intervals per frame  has  been  specified.
            Values  <  5  are not recommended (at least 5-6 measures are
            necessary to reliably evaluate  the  standard  deviation  of
            the   average   from  the  scatter  around  it).  NOTE:  not 
            applicable for 'lcurve', 'lcstats' and 'efsearch'.
        
        exposure (flag for analysis of exposure profile) [boolean]
            If =yes the exposure profile(s) (i.e. newbin values are  set
            =1,  gaps  and  rejected newbins are set =0) is/are analysed
            (instead of the input series). Default value is = no.
        
        normalization (type of normalization) [integer]
            Flag to specify the type of normalization to  apply  to  the
            results.   This parameter is only relevant for the following
            tasks: powspec, autocor,  efold,  crosscor,  timeskew).  The
            standard  normalization  corresponds  to  a  value of 1 (the
            default) in all  XRONOS  applications.  Other  normalization
            value   flags   are  described  for  each  application  (See 
            normalization).  NOTE:  not  applicable  for  'lcurve'   and 
            'efsearch'.
        
        simultaneous (flag for simultaneity) [boolean]
            If  =yes  a  strict simultaneity is forced between the input
            series in applications which use more then one series  (i.e.
            if  the  n-th  newbin  of  a series is a gap or is rejected,
            then the n-th newbin  of  all  other  series  will  be  also
            rejected).  This  flag is ignored in the efold applications.
            Default value is = no.
        
        spwinbefore (special window start) [double]
            Special newbin window :  number  of  seconds  before.  If  a
            value  >  0 is used , e.g. 10.0, then all the newbins within
            10.0 seconds  before  a  newbin  rejected  by  an  intensity
            window  will  also  be  rejected. The default (=0) is not to
            apply this type of special newbin window.
        
        spwinafter (special window stop) [double]
            Special newbin window : number of seconds after. If a  value
            >0  is  used  ,  e.g. 10.0, then all the newbins within 10.0
            seconds after a newbin rejected by an intensity window  will
            also  be  rejected.  The  default  (=0) is not to apply this
            type of special newbin window.
        
        rescale (rescaling for results) [double]
            Rescaling factor applied to  result  variables  and  errors.
            The  rescaling  is  applied  just  before writing the output
            file (this to avoid affecting the statistical variables  for
            the frame). Default value for "rescale" is set to 1.
        
        offset (additive constant for results) [double]
            Additive  constant  summed to result variables. Result error
            bars are left unchanged.  The  additive  constant  is  added
            just  before  writing  the  output  file  (this  is to avoid
            affecting the statistical variables  for  the  frame).  Note
            that  if  a  rescaling  factor  is also specified (different
            from 1), then  the  results  are  first  multiplied  by  the
            rescaling factor. Default value for "rescale" is set to 0.
        
        fast (Flag for fast algorithm) [boolean]
            This  parameter  sets  the  type  of  algorithm used for the
            Fourier transform.  IMPORTANT NOTE: This  parameter  can  be
            set  by  the  user  only  in  the  `powspec`,  `autocor` and
            `crosscor` tasks (it is a query parameter in  these  tasks).
            In  all the other tasks "fast" is a non-query parameter, and
            should not be changed from the default setting.
        
        ipow2 (Flag if power of 2) [integer]
            Internal Flag used to decide if the current task  must  used
            with  a  power  of  2  of  number  of  points  (ipow2=1) per
            interval or not (ipow2=0). IMPORTANT  NOTE:  This  parameter
            should not be changed by the user.
        
        iavgreb (Flag if average interval) [integer]
            Internal  Flag used to decide if the current task allows the
            averaging of intervals in frame and/or the rebinning of  the
            analysis  results.  If "iavgreb" is set to -1 the results in
            an interval can not be either averaged or rebinned,  if  set
            to  -2,  the  results in an interval can be averaged but not
            rebinned. IMPORTANT  NOTE:  This  parameter  should  not  be
            changed by the user.
        
        nbdf (Default No. Bins) [integer]
            Set  an internal default value for the number of newbins per
            Interval.  This value  is  used  to  calculate  the  default
            newbin  integration  time  to  have  one  interval with nbdf
            points. Different "ndbf" values have been set for  different
            XRONOS  task.  IMPORTANT  NOTE:  With caution this parameter
            can be changed by the user.
            
            
EXAMPLES
    1. Calculate statistical variables for an input time series  with  a
    binning  of  100  seconds  and     a  start-stop=5000 seconds in one
    interval (50 newbin per interval)
       > lcstats cfile1="mydata.lc" window="-" dtnb=INDEF nbint=50 
    2. For an input time series (consisting of several  files)  with  an
    original  binning  of  400    seconds and a total length of 6 hours,
    calculate statistical variables using a  newbin     of  800  seconds
    over  2-hour  intervals  (9  newbin  per  interval  for a total of 3
    intervals).
       > lcstats cfile1="@all.lis" window="-" dtnb=800 nbint=9
       The parameter cfile1 is a file containing the list  of  filenames
    for one time series    (see "Filelist and Input File Options").
    
    
    
SEE ALSO
    efold,  efsearch,  crosscor,  autocor,  powspec,  lcurve,  listdata, 
    timeskew xronwin, fits2qdp, ascii2lc.
    
    
BUGS
    Report     problems     to     angelini@lheavx.gsfc.nasa.gov     and    
    xanprob@athena.gsfc.nasa.gov.   Provide  a  detailed  description of
    the problem (with a log file if possible).

Xronos Home Page Xanadu Home Page Xanadu ftp site

Please send reports of errors to : xanprob@athena.gsfc.nasa.gov

HEASARC Home | Observatories | Archive | Calibration | Software | Tools | Students/Teachers/Public

Last modified: Thursday, 06-May-2004 13:47:11 EDT