crosscor -- Calculates cross correlation using a fast or slow algorithm for 2 input time series
crosscor file(s)+options window dtnb nbint nintfm plot plotdev outfile
This task calculates the cross correlation of 2 input time series, plots and outputs the results (in a FITS file). The cross correlation is computed either by an FFT algorithm or a direct slow algorithm. The paramater "fast" can be used to select the type of algorithm (the default is set to fast). The input file format is FITS using the BINTABLE extension. Both binned data format and event format are input. It requires two simultaneous input time series (See FILELIST and INPUT FILE OPTIONS). Time, Phase, Intensity and Exposure windows (See WINDOW) allow for data screening. Input data can be rebinned and divided into Intervals and Frames (See GENERAL XRONOS TERMINOLOGY). The results of the cross correlation from several intervals can be averaged in a frame. The standard plot output is cross correlation versus time delay (see PLOT). The time delays refer to the 1st series: therefore a cross correlation function peaking for positive time delays indicates that variations in the 2nd series lag those in the 1st series. The cross correlation error bars are from the standard deviation of the average of the cross correlations in each delay bin from different intervals, if the specified number of intervals per frame is larger than the value of "errorbars" parameter (default =5); otherwise error bars are set =0. For the "slow" method only, error bars are also obtained by propagating the theoretical error bars of the cross correlations from individual intervals (in turn obtained by propagating the newbin errors bars through the cross correlation formula). Different normalizations can be used for the auto correlation by changing the parameter "normalization" (See normalization).
The "normalization" paramater has the following meanings (and values) :
* =0 cross correlations are normalised by dividing by the square root of the product of the number of good newbins of the two series in each interval, i.e. they are cross covariances. * =1 (d/f); cross correlations are normalised as in the case =0 and by dividing also by the square root of the product of the newbin variances of the two series in each interval. * =2 cross correlations are normalized as in the case =0 and by dividing also by the square root of the product of the newbin excess variances of the two series in each interval. Problems arise when, due to statistical fluctuations, the excess variance of an interval is negative for any of the two series. To avoid negative normalization, `crosscor` adopts automatically normalization 1. * =3 cross correlations are normalized by dividing by the square root of the product of the average newbin excess variances of the two series in each frame after averaging the cross covariances. If the excess variance in a frame is negative, for any of the two series, 'crosscor' adopts automatically normalization 1.
If other values than those listed above are used, they are treated as =0.
Within XRONOS tasks, BINS and NEWBINS control the binning used in the analysis, INTERVALS the subdivision of the time series and FRAME the grouping of the output results:
BINS : these are the time bins of the time series being analysed. More than one input file can have different bin durations, e.g. two consecutive time series, one with 0.5 s bins and the other with 2 s bins. The original bin time is the value stored in the input file in the keyword TIMEDEL. If the data are stored in each row as an array with 1CTYPn = 'TIME', the original bin is set to the value stored in the keyword 1CDLTn (where n is the column number).
NEWBINS : these correspond to the time resolution at which the analysis is carried out. Note that: (i) newbins cannot be shorter than the longest bin duration of the time series being analysed; (ii) in many XRONOS applications (e.g. powspec, autocor, crosscorr) the newbin duration is forced to be an integer multiple of the longest bin duration.
INTERVAL : an interval is defined by the number of newbins over which the analysis is carried out. Note that in applications using FFT algorithms (e.g. powspec, autocor and crosscor set in fast mode) the number of newbins in an interval is a power of 2.
FRAME : a frame consists of the average of the results of the analysis of one or more contiguous intervals. Note that in 'lcurve', 'efsearch' and 'lcstats' a frame consists always of one interval.
If any window is required during the analysis, a window file containing the relevant windows must be created with the application XRONWIN, before running a XRONOS task. There are 4 different types of windows :
* Time Windows : consist of up to 1000 time intervals * Phase Windows : consist of an Epoch, a Period and up to 10 phase intervals * Intensity Windows : consist of up to 10 intensity in bin, newbin and interval * Exposure Windows : consist of up to 1 exposure in bin, newbin and interval
Intensity and Exposure Windows can be specified independently for: (i) Bins , (ii) New Bins , (iii) Intervals. When dealing with more than one time series, Intensity and Exposure Windows must be specified separately for each series. Time and Phase windows are applied to Bins. Intensity and Exposure windows are applied first to Bins, then Newbins and finally to Intervals as specified. For time and phase windows, only those bins whose center time is within the start and stop of a time window or phase window (for a specified epoch and period) are accepted. Intensity windows must be ordered with increasing intensity and if set for newbins can be used in conjunction with "Special Newbin Windows" (see below).
Exposure Windows consist of a minimum and a maximum exposure level. Units are such that 1 means 100% exposure. The Newbin Exposure is obtained by propagating the bin exposures to each newbin. For example, if in a 30 s newbin the total exposure (due to the sum of the individual exposure of the bins contributing to the given newbin) is 18 s then its exposure is 60%. The Interval Exposure is the ratio of accepted to expected newbins: for example, if a 128 newbin long interval contains only 32 accepted newbins, then its exposure is 25%. Many XRONOS application use some default exposure windows, which are designed to avoid analysing data sets which are too inhomogeneous with respect to their statistical properties. The minimum default Exposure windows in an Interval is set to 0.0 in the lcurve, efold and efsearch and to 0.5 (i.e. 50% exposure ) in all the other tasks. Note that exposures can be higher than 100% (e.g. if the newbin time is not a multiple of the bin time, then "beats" are generated which might bring the exposure of a newbin to values >100%; or if two or more input files for the same time series overlap in part, some of the newbins will be more than 100% exposed).
IMPORTANT NOTE WHEN TIME WINDOWS ARE SET IN THE WINDOW FILE: The time used within XRONOS tasks is Truncated Julian Days (TJD= JD-2440000.5) if either (1) the keyword MJDREF is present in the header or (2) if the TIMESYS value is one of the following strings MJD or JD or TJD. If (2), the time values are expected to be stored as JD, MJD or TJD in the header keywords and in the TIME column in which case the MJDREF keyword is not used (it should not be present). When Time windows are set using XRONWIN, they must be compatible with the values in header of the timing keywords and/or the values in the TIME column.
An additional window type called "Special Newbin Window" can be set directly from the parameter file. Special Newbin Windows are used to exclude the parts of a light curve which immediately follow or precede a burst or a background event which has been rejected by intensity windows in newbins. The Special Window operates on newbins in conjunction with intensity windows (in newbins) and are specified by changing to positive values the parameters 'spwinbefore' and 'spwindowafter'. Their use is the following: if e.g. spwinbefore is set =10, all newbins, whose center time is within 10 second before the center time of a newbin rejected by intensity windows, will also be rejected; if e.g. spwindowafter is set =20, all newbins, whose center time is within 20 second after the center time of a newbin rejected by intensity windows, will also be rejected.
To input multiple files for each time series, a file containing the list of files is needed (Filelist). The Filelist is input in the program as '@Filelist'. The format of this file list is ascii and contains one filename+options per line. Files from different times series are separated by '///' mark. Below is an example of the Filelist containing 2 files for 3 different times series.
file1_ser1 file2_ser1 /// file1_ser2 file2_ser2 /// file1_ser3 file2_ser3
The Input File Options (up to 10) can be specified for each file in the same input string. They consist of 2 characters followed by a numerical constant (up to 8 character long). There are two groups of options. The first allows data selection within a FITS extension. The available options within this group are :
frN= start reading input file from row number N (first row) lrN= stop reading input file from row number N (last row) vxN= use column number N as x-axis (i.e. time axis, default name is TIME) vyN= use column number N as y-axis (default names are COUNT or RATE) vsN= use column number N as error for y-axis (default name is ERROR) veN= use column number N as exposure (default name FRACEXP). If the input file is an event list, exposure is by default calculated using the GTI extension. In this case, N=0 turns off the usage of the GTI extension for the exposure calculation, and N > 0 specifies the GTI extension to use. feN= select data (either binned or events) from channel number N (First Energy). For an event list channel selection is made using the column named 'PHA' leN= select data (either binned or events) to channel number N (Last Energy). For event list the default column channel name searched is 'PHA'. The option 'vcN' allows the choice of a channel column name different from 'PHA' (es. 'PI'). vcN= use column number N for channel selection (valid only for event lists). rtN= use extension N of the FITS file to read the data. The first extension is N=1 (the primary array is irrelevant). To specify the extension the following also can be used: filename[N] or filename+N. of = The MJDREF keyword is not used. The time is calculated using the TIME column and the TIMEZERO keyword.
The second group of options performs algebraic operations on individual input files. They are applied in the same order in which are specified. For event files they are applied after the data are binned. The available options within this group are:
stX = Shift all Time in input file by X days ssX = Shift all times in input file by X Seconds muX= multiply data and errors by X (MUltiply) mdX= multiply data by X (Multiply Data) meX= multiply errors by X (Multiply Errors maX= as muX but exposure is divided by X diX= divide data and errors by X (DIvide) ddX= divide data by X (Divide Data) deX= divide errors by X (Divide Errors) daX= as diX but exposure is multiplied by X aaX= add data and errors with X (Add All) adX= add data with X (Add Data) aeX= add errors with X (Add Errors) saX= subtract data and errors with X (Subtract All) sdX= subtract data with X (Subtract Data) seX= subtract errors with X (Subtract Errors) qaX= add to data the square of data muliplied by X and add to errors the product of data and error multiplied by X qdX= as above but for data only qeX= as above but for error only
Below is an example of the Filelist containing 2 files for 3 different times series where the different options are applied to the input files for different time series.
file1_ser1 aa4 add to data and error 4 file2_ser1 aa4 " " " " /// file1_ser2 rt2 aa2 read 2nd extension; add to data and error 2 file2_ser2 rt2 aa2 " " " " " " " /// file1_ser3 rt2 vy4 vs5 read 2nd extension; use column 4 and 5 for Y-axis and Error file2_ser3 rt2 vy4 vs5 " " " " " " " " " "
The array of results from each XRONOS task can be plotted. The PLT routine (See also QDP/PLT manual) provides the interactive plotting and fitting functions. The plotting function is available for the following tasks : autocor, crosscor, efold, efsearch, lcurve, powspec and timeskew. The 'crossocor' plot is the cross correlation function versus time delay (positive and negative) given in seconds. A number of commands can be entered from the 'PLT>' prompt to allow plot customisation (e.g. Add/remove labels; Plot data with various combinations of lines, markers, and error bars; Change text fonts; Change X-axis and/or Y-axis; Change number of plotted panels; Define and Fit models to data). Useful QDP Commands are:
PLT> r x xlow xhigh * Rescale X axis PLT> r y ylow yhigh * Rescale Y axis PLT> r xlow xhigh ylow yhigh * Rescale both PLT> log x * X axis is plot in logarithmic scale PLT> log y * Y axis is plot in logarithmic scale PLT> log off * Turn off the logarithmic axes (both X and Y) PLT> dev /xxx * Change the current plot device PLT> mo ? * List available model PLT> mo cons linr * Define model= constant plus linear plt> fit * Fit the defined model PLT> hardcopy filename * Hardcopy of the current plot (postscript) PLT> exit * EXit: to exit from the PLT subroutine type:
The QDP/PLT software is provided and maintained by Allyn Tennant (Marshall Space Flight Center). The PLT software uses the PGPLOT Graphics Subroutine Library for plotting, written by T.J. Pearson (California Institute of Technology).
The analysis results are output in a FITS file. Two different FITS layouts are available (see parameter outfiletype). The first stores one interval (or frame) of interval results per FITS table row, and the output file will have a single extension. The second stores one interval (or frame) per FITS extension, and the output file will have as many extensions as the number of intervals (or frames). The output file contains, besides the array of results, a number of statistical variables (and errors if appropriate) associated with each interval (or frame) for each time series. These are: 1- average count/s in frame (in a frame this is the average of the averages in intervals); 2- fractional exposure in frame (from average of fractional exposures in intervals, the latter is the ratio of good newbins to the total number of expected newbins/interval); 3-variance in frame (average of variances in intervals); 4- expected variance in frame (average of the expected variances in intervals, the latter is calculated from the error bars of the newbins); 5- 3rd moment in frame (average of 3rd moments in intervals); 6- minimum count/s in frame; 7- maximun count/s in frame; 8- excess variance in frame (average of excess variances in intervals, the latter is calculated as variance-expected variance); 9- chi-square in frame (average of chi-squares in intervals); 10-rms fractional variation in frame (average of rms fractional variation in intervals, the latter is calculated as the square root of the excess variance divided by the average.
When "outfiletype"=2, the output file for the 'crosscor' task contains in each extension the following columns: time delay, half width of time delay bin, cross correlation and error. Not many applications can currently deal with the output files produced when "outfiletype"=1. The format will be described in future releases.
1. Given 2 time series with an original binning of 1 second and length 1.5 hours compute the cross correlation in intervals of 1024 points and average all the results per interval in a frame. Make a plot and output the results.
> crosscor cfile1="mydata1.lc" cfile2="myfile2.lc" window="-" dtnb=1 nbint=1024 nintfm=INDEF plot=yes plotdev="/xw" outfile="-"
2. As before but using as input file a list of filenames (@all.lis) (see "Filelist and Input File Options").
> crosscor cfile1="@all.lis" window="-" dtnb=1 nbint=1024 nintfm=INDEF plot=yes plotdev="/xw" outfile="-"
3. Given 2 time series with an original binning of 1 second and length 1.5 hours compute cross correlation using all the data in one interval (nintfm=1). The number of points in such interval length are 5400 and for the fast method this requires that the number of points be a power of two. The closest value is 8192.
> crosscor cfile1="@all.lis" window="-" dtnb=1 nbint=8192 nintfm=INDEF plot=yes plotdev="/xw" outfile="-"
efold, efsearch, autocor, powspec, lcstats, lcurve, listdata, timeskew xronwin, fits2qdp, ascii2lc.
Report problems to email@example.com and firstname.lastname@example.org. Provide a detailed description of the problem (with a log file if possible).