ASCA Rev1+ Data Processing

On August 15, 1996 the ASCA data processing team will start using a new implementation of processing called rev1+. With a few exceptions, the data products in rev1+ are the same as those in rev1.

Why a New Processing version?

Rev1+ will provide a bridge between rev1 (basically just filtered event lists) and rev2 (including source detection, spectra, and exposure maps). Rev1+ uses the software for rev2 (a set of ksh scripts) but produces nearly the same data products as rev1. This way the user community can enjoy some of the benefits of rev2 while its full implementation is being worked out. Rev1+ will also allow thorough testing of the new software before the entire archive is reprocessed under rev2.

CD-ROMS

With the advent of rev1+ the ASCA processing team will be able to distribute data on CD-ROMS. This will eliminate the hassles of reading data from FITS format tapes, and allow users to analyze their data without having to dedicate hundreds of megabytes of disk space. The CD-ROMS will include an HTML interface to the data and contain documents such as the ABC Guide to ASCA Data Reduction and The ASCA Getting Started Guide.

If you would like to receive your data on CD, send e-mail to ascamat@ssvs.gsfc.nasa.gov with the subject line "CDROM". At first we may not be able to accommodate all requests, but eventually we expect to distribute all datasets on CD.

What are the Differences?

This section describes the differences between rev1 and rev1+ in detail.

No More Raw Files

In rev1, the raw telemetry was unpacked into a set of several hundred raw event files (ft*.fits). These were calibrated individually using the ascalin FTOOL, and FAINT mode files were converted to BRIGHT and BRIGHT2 modes. The calibrated raw files were then grouped by mode into ten or so unfiltered event files (ad*.unf). The unfiltered event files were then screened using xselect to produce the filtered event files (ad*.evt).

In rev1+, the raw event files (ft*.fits) are combined immediately into unfiltered event files (ad*.unf), and the raw files are deleted. This has several advantages:

  1. The number of files for a given sequence is reduced by an order of magnitude.
  2. The typical volume of a dataset is reduced by a couple hundred megabytes.
  3. Processing time is decreased because the overhead of opening each file and reading its header, etc. is reduced by a factor of 10.
All of this should make ASCA data easier to process, distribute, and analyze, without any loss of useful information.

To use the unfiltered files (ad*.unf) in your data reduction, you need to run ascascreen -e unf and specify the unscreened directory as your data directory. Future versions of the FTOOLS (3.6 and higher) should do this by default. If you are running xselect directly, set the data directory to the unscreened directory with the command set datadir [unscreened directory path] and then load the files individually with the command read events [filename], where [filename] represents the name of the file you wish to load.

HTML Documentation and Logs

Rev1+ will include several HTML documents describing your observation, the distributed files, and the processing of the data. Hardcopy versions of these will accompany your distribution.

GIF images

In rev1 high and medium bit-rate event files with identical modes were combined and run separately through xselect. Color GIF format images were then created for each combined bit-rate filtered event file.

In Rev1+ the high + medium bit-rate combined event files are eliminated. FITS format images are created for each filtered event file (just as they were in rev1) and these images are added directly to produce a single GIF image for each of the four instruments. This is a more efficient way of processing the data and produces smaller datasets.

Calibration Files

Rev1+ will do a more thorough job of distributing the calibration files used in processing.

Quick Analysis Output

The SIS quick analysis FTOOL, sqaplot, will be run on filtered event files which have had hot and flickering pixels removed. Also, hardcopies of the ASCII output from sqaplot and gqaplot will no longer be distributed, although the files themselves will still be included on the tape or CD.

ascatape and File Names

The current version of the FTOOL ascatape (FTOOLS 3.5.1 and earlier ) may corrupt the names of wrapped (non-FITS) files. Specifically, some underscores may be changed to decimals, and some files may be given an extension of ".unwrapped". This will not affect any of the actual data products. The only significant effect will be that the links between HTML documents will be broken. This can be fixed by renaming these files by hand. The correct names of the HTML files are:
  1. ad[sequence]_[ver]_hdr_page.html
  2. ad[sequence]_[ver]_event_info.html
  3. ad[sequence]_[ver]_file_info.html
  4. ad[sequence]_[ver]_error.log
  5. ad[sequence]_[ver]_index.log
  6. ad[sequence]_[ver]_job.log
where [sequence] is the 8 digit sequence number and [ver] is the 3 digit processing version number.

Other Information

Here are some links to related documents: If you have any questions which these documents don't answer, you can send e-mail to ascahelp@ascasrv.gsfc.nasa.gov