6. SCREENING

6.1 Introduction

The purpose of screening is to create, for each instrument, events lists that contain good data, suitable for scientific analysis.

There are three methods of applying screening criteria, and three starting points which differ in the amount of pre-screening performed by the ASCA Data Facility. These will all be described below.

After screening, users are ready to filter the parts of the data they wish to analyze and subsequently extract spectra, light curves, and images.

6.2 The Basics of Screening

6.2.1 Screening Using XSELECT, ASCASCREEN or TKASCASCREEN

Screening is a routine process and done by using XSELECT as the main workhorse to apply screening criteria to the FITS data files. Screening is made even easier with the scripts ascascreen and tkascascreen which drive XSELECT with convenient user-interfaces. The three methods of screening can be summarized as follows.

1. XSELECT Running XSELECT in stand-alone mode, screening can be performed by entering the appropriate Boolean expression using the mkf parameters and their desired ranges of values, with the command select mkf. example:

   xsel:ASCA-GIS2-PH > select mkf "ACS==0 && COR>4 && ELV>5"
   > Enter the filter file directory >[.]
   

   Screening criteria may be written in Fortran style (e.g., ELV.gt.5) or in C style (ELV$>$5). The select mkf command prompts the user to enter the directory where the mkf file or files is/are located. Alternatively the mkf directory can be specified on the command line by assigning it to the parameter mkf_dir (for example, mkf_dir=../aux/).

2. ascascreen This is a perl script which essentially creates a script which runs XSELECT by prompting the user for various inputs. By answering questions at the screen, users do not have to remember the corresponding XSELECT commands, the standard selection criteria, nor their recommended values. If the default criteria are selected in ascascreen, the result will be a set of cleaned events lists identical to those produced by the standard GSFC processing (see §5.5). One thing to be aware of is that introducing non-standard criteria has to be done outside ascascreen, but this is straight-forward.

3. tkascascreen This is a GUI version of ascascreen. This provides a convenient, user-friendly method of screening.

The first stage of screening involves determining which data files correspond to the instrument, mode and bit rates you want to analyze and then applying screening criteria. For this, ascascreen makes using XSELECT easier. However, XSELECT must still be used for the final stage of defining and applying filters and extracting products.

6.3 Suggested Screening Strategies

As mentioned in §2.4, ASCA data are made available to users, on tapes and in the Archive, in three different levels of processing. These are UNSCREENED, SCREENED and RAW (but the latter are discontinued in REV1 and later revisions; see §2.4). From the point of view of screening, these levels represent three different starting points. The most common will be to read the unscreened files into XSELECT into XSPEC in order to apply screening criteria suited to the user's particular data set. On the other hand for a quick-look analysis the user can read the screened files into XSELECT and proceed immediately to filter and extract images, light curves and spectra.

If for some reason, the user cannot obtain the unscreened, *.unf files from REV1 (or later) processing, XSELECT and ascascreen will still accept RAW files.

Whether to use XSELECT, ascascreen, tkascascreen, unscreened files or screened files, is a matter of taste, convenience and what you want to do with your data. Here is a (non-exhaustive) list of data reduction scenarios matched to suggested screening strategies:

• Producing a screened spectrum as fast as possible. Since the default settings of the screening criteria applied by ascascreen are the same as those used in producing the screened events lists, the quickest route is to skip screening altogether. Start XSELECT, read in the screened events list and proceed to apply filters and to extract a spectrum.

• Relaxing the settings of the standard criteria to increase signal-to-noise. The most convenient way to re-apply the standard criteria with different settings is to run ascascreen on the unscreened data files.

• Applying non-standard criteria. Selection criteria can be applied to any column in the data files or mkf file - not just to the standard set used by ascascreen. If, for example, you want to select data based on the altitude of the satellite (the SAT_ALT column in the mkf file) then we recommend you run ascascreen on the raw files with the -q option and then edit the *_mkf.sel file to include the criterion. Alternatively, you can read into XSELECT the screened events list and apply the additional criterion with the select mkf or select events commands - whichever is appropriate.

• Two-stage reduction. After producing a screened events list, ascascreen leaves you at the XSELECT prompt, giving you the opportunity to continue the reduction process, i.e., filtering and extraction. If, however, you'd prefer to stop and continue later, quit the XSELECT session. To continue, type XSELECT and read in the screened events list.

6.4 ASCASCREEN

6.4.1 Before You Begin

It is generally a good idea to keep your mkf files, screened events lists and data products in separate directories.

6.4.2 About ASCASCREEN

The automatic screening script ascascreen simply carries out, with a minimum of user interaction, what the ASCA GOF consider to be standard screening criteria. Rather than executing the command XSELECT followed by a set of XSELECT commands, the script prompts the user for inputs which are then used by the script to produce an XSELECT command file (a file with a .xco extension) which is then run by XSELECT. The command file is not discarded and can be used again - edited, if you like - for a truly automatic XSELECT session. It can also serve as a reminder of the sequence of XSELECT commands if you're using XSELECT xselect interactively.

The ascascreen script comes automatically with the FTOOLS package, and can be found in the directory $FTOOLS/bin.perl (for example, $FTOOLS may be /ftools/ALPHA/release) which is produced by the installation process.

Please note that ascascreen is written in Perl and therefore requires that Perl be installed for it to run. If you do not have Perl, type ascascreen at your system prompt to see a set of instructions on how to obtain Perl.

Finally, it is assumed that the user has read chapter 5 and fully understood the various selection criteria that are used in analysing ASCA data.

6.4.3 Running ASCASCREEN

First create a directory for your screened products beneath the one containing the screened events files. Then type ascascreen which will cause the following series of directions and questions to appear.

1. Enter data directory: ascascreen understands relative directory names, so typing .. is sufficient to specify the directory above (if that is indeed where your data files are). Note that ascascreen assumes that the mkf file is either in the same directory as the data files or in a directory called aux at the same level as the data.

2. Which instrument do you want to process? ascascreen examines the contents of your data directory and, based on what's in there, asks you to choose an instrument. Answers can be lower case, e.g., gis2. If more than one data mode is present, you'll be offered a choice.

3. Enter product filename root [ xsel ]: The names of the screened files produced by ascascreen will be prefixed by whatever characters you enter at this prompt. Note that ascascreen does not add an instrument descriptor to the name, so it's a good idea to specify prefixes like a1060_g2, for example.

4. Which Bit Rates do you want to use? It's possible to enter a combination here, such as HIGH, MEDIUM, as well as a single bit rate.

   Note that for GIS data:

   • In GIS PH mode, temporal resolution depends on bit rate.

   • There is usually very little LOW-bit rate data, but it is OK to use except for sources brighter than 4 count/s/GIS (the telemetry limit in LOW bit rate).

   For SIS data:

   • In general SIS temporal resolution does not depend on bit rate.

   • There is usually very little LOW-bit rate data, which, because of hot and flickering pixels, is invariably saturated and can be excluded from analysis.

5. After looking in the data directory for the specified bit rates, ascascreen displays on the screen a summary of the data, like this:

        Index NEVENTS  ONTIME     RAWXBINS RISEBINS TIMEBINS PHA_BINS POS_DET
           0  123721   66181.8      256       32        1     1024       FLF
   

   for the GIS and

        Index NEVENTS  ONTIME  S0CCDMOD S0CCDLST S0_ARENA
            0  872848  73264.8        4     0123        0
   

   for the SIS. The quantities listed (apart from NEVENTS and ONTIME) correspond, for the GIS, to the PH mode bit assignments and for the SIS to various mode-dependent settings. Data for which these crucial modal parameters are different and should not in general be analyzed together. The user is thus forced to chose a data set by specifying the index (the first column). In this example only one index is present, so no choice is available.

Next, the user is prompted for actual mkf parameter selections.

1. Enter Maximum allowed angular deviation [ 0.01 recommended ]:
   This refers to the deviation from mean pointing and is measured in degrees. See §5.2.2 for more details.

2. Enter minimum elevation angle [ 5-10 recommended ]:
   This refers to the angle between the target and the Earth's limb and is measured in degrees. See §5.2.1 for more details.

3. Enter angle from Bright Earth [ 20-40 recommended ]: (SIS ONLY)
   This refers to the angle between the target and the Earth's bright limb and is measured in degrees. See §5.4.3 for more details.

4. Enter minimum cutoff rigidity [ 6 recommended ]:
   This identifies periods of high background based on cut-off rigidity. It is measured in GeV/c. See §5.2.4 for more details.

5. Enter PIXL rejection threshold [ 50 recommended ]: (SIS ONLY) This identifies periods when the count rate per pixel exceeds a threshold due to background flares. See §5.4.4 for details.

6. Start up SAOimage, and remain in XSELECT at the end? (y/n):
   Self-explanatory, but note that if you answer `no', you'll need to start a new XSELECT session to continue the reduction process (i.e., filtering and extracting).

7. Remove ring and calibration source? (y/n): (GIS ONLY)
   Non-X-ray events occur most often in the outer ring of the GIS FOV. It is therefore a good idea to exclude this part of the FOV, as well as the internal monochromatic calibration source which also appears at the edge of the FOV. See §5.5.2 for details.

8. Use Rise Time window background rejection? (y/n): (GIS ONLY)
   The final question refers to using the Rise Time Invariance to reject background events. It should almost always be applied. Note that the requisite RTI column may be present but not filled. REV0 users are prompted on whether to have ascascreen fill it before proceeding with the screening. See §5.3.2 for details.

9. Remove hot and flickering pixels? (y/n): (SIS ONLY)
   Unless you want to study them, hot and flickering pixels should always be removed. See § 4.4.1 and §5.4.1 for more details.

10. Select based on grade ( this will keep only grades 0,2,3 and 4)?: (SIS ONLY)
    By choosing these grades, you'll be get a good combination of spectral resolution and signal-to-noise. See §5.4.2 for more details.

Note that for SIS data, ascascreen applies two additional criteria which, because they are usually fixed, are not prompted for. These are (1) time after passing through the SAA (T_SAA) and (2) the time after passing through day-night terminator (T_DY_NT). These selection criteria are fully described in § 5.4.6 and §5.4.7 respectively. Note, however the bug in FTOOLS v3.6, described in §5.4.8.

Taking GIS data as an example, ascascreen produces the following files (a1060_g2 was used as the root of the product file names in this example):

• a1060_g2.evt, the screened events list. If you choose not to run SAOimage as part of your ascascreen session, then this is the file you should read into XSELECT to continue filtering and extracting.

• a1060_g2.xco, the XSELECT command file used to drive the XSELECT session. If you type @a1060_g2.xco at the XSELECT prompt, you can re-run the screening.

• a1060_g2_mkf.sel, the file containing the mkf-based selection criteria. In the XSELECT command file, it is activated by the command select mkf @a1060_g2_mkf.sel.

• a1060_g2.img, a detector image extracted from the screened events. If you chose to run SAOimage as part of your ascascreen session, then this file is displayed on the screen. If not, you can look at it with SAOimage: by typing saoimage -fits a1060_g2.img.

• a1060_g2_randc.reg, the SAOimage region filter file corresponding to the high background ring and internal calibration source. In the XSELECT command file, it is activated by the command filter region a1060_g2_randc.reg.

• xsel.log, the XSELECT log file. If you have a problem running ascascreen or XSELECT, please send this file, with an account of the problem, to ascahelp@athena.gsfc.nasa.gov.

• a1060_g2_obscat.sel, the obscat selection file, contains the criteria which identify the input data files.

• a1060_g2.cat, a FITS file containing a catalogue of the input data files chosen.

An analogous set of files is produced with SIS data.

6.4.4 After Running ASCASCREEN

ascascreen stops after creating a screened events list. What you do next depends on which of the following three scenarios applies in your case.

1. If you answered `yes' to the question about starting SAOimage and remaining in XSELECT, then you can continue the reduction process. The next steps are filtering, and extracting spectra, light curves or images, described in chapters 8, 9 and 10 respectively.

2. If you answered `no' to the question about starting SAOimage and remaining in XSELECT, then you have to restart XSELECT. Type XSELECT at your system prompt. You'll first be prompted to specify the session name (used to identify temporary files). Next, you have to tell the program where the data (in this case, the screened events list) and the mkf files are. Use the commands set mkfdir and set datadir. Finally, read in the events list with the read events command and proceed to filter and extract.

3. If you answered `yes' to the question about starting SAOimage and remaining in XSELECT, but then change your mind and want to exit XSELECT, type exit at the XSELECT prompt and follow the instructions for case 2 above.

6.4.5 The -q Option

As mentioned above, ascascreen creates an XSELECT command file which is used to drive XSELECT. By typing ascascreen -q, users can stop ascascreen at the point just before XSELECT is invoked and the command file executed, thereby returning to the system prompt. The advantage of such a procedure is that it enables the user to change the screening process without having to run XSELECT from scratch: users can simply modify the command file and or the mkf selection file and then use the modified file to run XSELECT automatically.

For example, if you want to create a screened events list from 4-CCD mode data which includes events from only one chip (SIS0, chip 1, say), run ascascreen with the -q option and then use a text editor to add the following line to the XSELECT command file produced by ascascreen (sis0.xco, say):

select events ccdid.eq.1

Then type XSELECT , give a session name, and then enter the command

XSELECT>@sis0.xco.

6.5 TKASCASCREEN

tkascascreen is a GUI version of ascascreen which does exactly the same things as ascascreen. It is very easy to use, but to understand the questions, please refer to the instructions for running ascascreen.

There is one minor difference: the session name which the user should supply on the first screen is the prefix added to the names of the output files.

6.6 XSELECT

6.6.1 Getting Acquainted

This section explains how to use XSELECT manually to achieve the same results as ascascreen. To get better acquainted with XSELECT, we also suggest the following.

• Re-reading the section on ascascreen above. It tells you what XSELECT will do, although not how XSELECT will do it.

• Looking at a .xco file produced by running ascascreen. The file contains a set of XSELECT commands in the order in which they should be issued.

• Typing help at the XSELECT prompt. XSELECT is equipped with extensive on-line help.

• Reading the XSELECT User's Guide, which can be obtained from the URL

  http://heasarc.gsfc.nasa.gov/docs/software/ftools/xselect/xselect.html.

6.6.2 Before You Begin

Again, it is a generally a good idea to keep your mkf files, events files and data products in separate directories.

6.6.3 Running XSELECT

To start up XSELECT, type XSELECT at your system prompt. In this example, we'll be reducing GIS data from the PV observation of Abell 1060. Since the data are public, you can get the same data and mkf files yourself via FTP from the corresponding unscreened and aux directories. After giving the session a name, the mkf and data directories should be set:

tarquin>XSELECT

                         **  XSELECT V1.3   **

>Enter session name>[xsel] A1060

 Notes: XSELECT set up for      ASCA
 Keywords for time and pha are  TIME       PI
 Units of time are              s
 Default timing binsize =   128.00
 Keywords for images are          DETX       DETY
 Keywords for the WMAP are        DETX       DETY

A1060:ASCA> set mkfdir ..
A1060:ASCA> set datadir ..

Setting data directory to /usr/day/asca/data/a1060/
A1060:ASCA>

Notice that XSELECT displays its initial default settings. The mission is ASCA; light curves, spectra and images will be extracted from the TIME, PI and DETX/DETY columns, respectively. At this point, we specify which instrument we want to analyze and which data mode:

A1060:ASCA> set instrument gis2

Setting: IMAGE binning    =    1
         WMAP binning     =    1
         Energy Column    =  PI
         Energy rebinning =    1

A1060:ASCA-GIS2> set datamode ph

As before, another set of default settings is displayed: extracted images will not rebinned, nor will the WMAP (§8.2.2) and spectra.

The next, crucial step is to identify which of the data files in the data directory correspond to GIS2 PH mode. This is one of the important functions of XSELECT. For *.unf files it is trivial but any users who are still using RAW data files are spared from having to `deconstruct' the filenames. It is done by making what is called an obscat:

A1060:ASCA-GIS2> make obscat
> Default filter for the catalogue (NONE for none, DEF for default) >[DEF]
Using default selection expression: 
ONTIME>100&&NEVENTS>0&&HV_RED=='OFF'&&HVH_LVL==3&&HVL_LVL==4

******************** Observation Catalogue ********************

Data Directory is: /usr/day/asca/data/a1060/
HK Directory is: /usr/day/asca/data/a1060/

        OBJECT   DATAMODE BIT_RATE DATE-OBS TIME-OBS ONTIME   NEVENTS  PHA_BINS
      1 A1060    PH       MEDIUM   28/06/93 23:46:14 .146E+04     2225     1024
      2 A1060    PH       MEDIUM   29/06/93 00:20:06 .114E+04     2060     1024
      3 A1060    PH       HIGH     29/06/93 00:49:00 .136E+04     3254     1024
      .     .     .            .          .        .        .        .        .
      .     .     .            .          .        .        .        .        .
      .     .     .            .          .        .        .        .        .
     39 A1060    PH       MEDIUM   29/06/93 23:57:42 .262E+04     3458     1024
     40 A1060    PH       HIGH     30/06/93 00:41:32 .106E+04     2595     1024
     41 A1060    PH       HIGH     30/06/93 01:04:46 .392E+03      918     1024

A1060:ASCA-GIS2>

The `filter for the catalogue' offers the user the opportunity to pre-select files based on certain header keywords, some of which are displayed in the obscat itself. The default is fine for most purposes.

Each line in the obscat, not all of which are shown here, corresponds to an individual data file. Specifying which ones to merge and screen is done with the choose command. However, before choosing, users might want to edit the obscat by selecting, for example, all the HIGH bit-rate files. Generally, such selections are performed with the select command. In this case, the first argument is obscat and the second argument is a selection criterion, as in the following example.

A1060:ASCA-GIS2 > select obscat BIT_RATE.eq.'HIGH'

Notice that the selection criterion is specified in Fortran style. It is also possible to use C style. The parameter selected on, BIT_RATE is a column in the obscat (a FITS file, incidentally). Since its values are character strings, HIGH is enclosed in quotation marks: numerical values, such those in the ONTIME column do not require quotation marks. To see the new, selected obscat, type show obscat.

We can now choose which files to work on. To choose all of them, type choose 1-** (** stands for the number of the last file):

A1060:ASCA-GIS2 > choose 1-**

   Setting datamode to PH

Got the minimum time resolution of the chosen data:  0.62500E-01,

Getting Min and Max for Energy Column...
Got min and max for PI:     0   1023

Number of files read in:   25

Files currently in use:

   1   ft930628_2345_1426G200770H.fits
   2   ft930628_2345_1426G201770H.fits
   3   ft930628_2345_1426G201970H.fits
   .                                 .
   .                                 .
   .                                 .
  23   ft930629_1426_0123G206370H.fits
  24   ft930629_1426_0123G206970H.fits
  25   ft930629_1426_0123G207170H.fits

As can be seen, the screen output shows the data files we've chosen.

We are now ready to apply selection criteria which are described in detail in chapter 5. For the GIS, these fall into three categories: (1) mkf-based criteria, (2) removing the high background ring and calibration source, and (3) applying RTI-based background rejection. Each of these uses a different XSELECT command.

To apply mkf-based selection, type select mkf followed by the selection criteria which can be entered separately, one per command, or strung together. The latter is better because each select mkf command creates a set of good-time intervals (GTI) which are combined with a logical `AND' operation to the current set. For large files, this can be CPU-intensive:

A1060:ASCA-GIS2-PH> select mkf
>Boolean expression for filter file selection >[] 
elv.gt.5.and.cor.gt.6.and.g2_l1.gt.0.and.saa.eq.0

The high background ring and calibration source are removed by applying a region filter. You can either extract an image and define the filter yourself, use a pre-existing one left over from a previous GIS2 ascascreen session or simply copy the following lines into an ASCII file to create an SAOimage file (which we'll call ring.reg):

 CIRCLE(124,132,81.00)
-CIRCLE(166.00,221.0,24.00)

See also §5.5.2. The filter is entered with the following command:

A1060:ASCA-GIS2-PH> filter region ring.reg

Applying RTI-based background rejection, unlike using a region filter or mkf file, is mission-specific. Accordingly, the procedure gets its own XSELECT command, gisclean:

A1060:ASCA-GIS2-PH> gisclean
Using table:/tmp_mnt/FTP/software/ascasrv/ftools/DEC/ftools/refdata/rti_gis_1024_040693.fits
Now processing /usr/day/asca/data/a1060/ft930628_2345_1426G201770H.fits
Infile # of rows    Outfile # of rows    # filtered
----------------    -----------------    ----------
        5915                5339               576
           .                   .                 .
           .                   .                 .
           .                   .                 .
Now processing /usr/day/asca/data/a1060/ft930629_1426_0123G207170H.fits
Infile # of rows    Outfile # of rows    # filtered
----------------    -----------------    ----------
         918                 841                77

For each data file, XSELECT displays the number of rows (each event occupies one row) which are accepted and rejected using RTI.

We can now create the screened events list by simply typing extract events. This command initiates the extractor, a program called by XSELECT which applies whatever GTIs and filters are in place to the current set of data files. Depending on what is extracted (events, a spectrum, image or light curve), the extractor creates a temporary file which is deleted upon exiting XSELECT, unless saved.

A1060:ASCA-GIS2-PH> extract events
 Extractor 1.0r
 XPI 2.1u
 Doing file: /usr/day/asca/data/a1060/redux/A1060_work1001.xsl
100% completed
    Total      Good    Bad: Region      Time     Phase       PHA
     5339      2623           1498      1218         0         0
 Writing events file
 .
 .
 .
 Doing file: /usr/day/asca/data/a1060/redux/A1060_work1024.xsl
100% completed
    Total      Good    Bad: Region      Time     Phase       PHA
      841       604            237         0         0         0
 Writing events file
100% completed
=====================================================================
   Grand Total      Good   Bad:  Region      Time     Phase       PHA
         73565     47039          21402      5124         0         0

The extractor displays on the screen a summary of which events have been extracted and which discarded. There are three categories for discarded events: Region, Time, Phase and PHA which correspond to the various kinds of filter. Events screened out using the mkf file are tallied in the TIME column. The events list, made up of the screened events in the data files, is saved by:

A1060:ASCA-GIS2-PH> save events a1060_g2.evt
Wrote events list to file a1060_g2.evt
>Continue to use the filtered events list?>[yes]
Changing Data directory from:
    /usr/day/asca/data/a1060/
to the current working directory.

Note that XSELECT, by default, assumes that the user wants to work with the new, screened events list which is in the current directory rather than the data directory above it - hence the screen message about changing the working directory.

At this point you can continue the reduction process. The next steps are filtering, and extracting spectra, light curves or images - described in chapters 8, 9 and 10 respectively.

Alternatively, you can quit XSELECT, do something else, and then resume. Type XSELECT at your system prompt. You'll first be prompted to specify the session name (used to identify temporary files). Next, you have to tell the program where the data (in this case, the screened events list) and the mkf files are. Use the commands set datadir and set mkfdir respectively. Finally, read in the events list with the read events command and proceed to filter and extract.

6.6.4 Displaying mkf Parameters

It is possible to plot all the mkf parameters from XSELECT using the commands mkfbin and then plot mkf. In the following example, SIS events above threshold (S0_PIXL1 and S1_PIXL3) are plotted against the Bright Earth angle (BR_EARTH). A full list of the mkf parameters can be obtained from XSELECT using the show par mkf command.

 xsel:ASCA-SIS1> mkfbin
 > Enter parameters in filter file >[] BR_EARTH S0_PIXL1 S1_PIXL3
 > Give print-out time interval >[20] 32
 xsel:ASCA-SIS1 > plot mkf

The available parameters are:

1. - TIME
2. - BR_EARTH
3. - S0_PIXL1
4. - S1_PIXL3

 > Enter independent variable ( 0 for TIME  ) >[0] 1
 > Enter dependent variable(e.g., 1-6) >[] 2-3
 PLT>

Standard QDP commands can be used from here on.

Note that the GSFC-supplied mkf files have a bin size of 32s. Using a finer `print-out time interval' than this in the mkfbin command will result in incorrect plots.

Plotting such mkf parameters as:

1. ELV, BR_EARTH, and COR
2. ANG_DIST
3. S0_PIXL1, S1_PIXL3, S0_TEMP, and S1_TEMP
4. G2_L0, G3_L0, and RBM_CONT

is highly instructive, particularly when the default screening criteria lead to a smaller than expected amount of accepted data.