|
|
An FTOOLS Tutorial
1 Introduction
This guide provides a brief tutorial on the use of the FTOOLS software package for analyzing FITS format data files. FTOOLS are the basis for ASCA and XTE data analysis. The purpose of this guide is to highlight some of the more generally useful tasks in the FTOOLS package and to give a step by step introduction on how to use FTOOLS. It is assumed here that the FTOOLS package as already been built on the machine you will be using; if it hasn't, refer to the appendix for help in installing FTOOLS.
FTOOLS is a collection of utility programs to create, examine or modify data files in the FITS (Flexible Image Transport System) format. These tools are not generally designed to perform detailed analysis or do model fitting of data, but instead are useful for examining the contents of the FITS file and for making various modifications to the files to prepare them as input to more involved analysis tasks.
Version 2.8 of FTOOLS was released in February 1994 and new versions are released about every 3 months. The FTOOLS tasks can be run in 2 different environments: as an IRAF package, or as a set of stand alone executable tasks. The IRAF version has been tested on suns and DECstations, while the standalone version also runs on VAX/VMS and on Alpha OSF/1 operating systems. The IRAF package requires a system to be configured with a large swap space of roughly 150 megabytes in order to load the larger CALTOOLS and HEASARC subpackages. The Host installation requires less than the typical 24 megabytes found on systems today. As with all applications, the more memory the better. Best performance is seen on systems with 32 megabytes or more of RAM. This tutorial describes the use of the standalone version of FTOOLS on a Unix machine, but the commands are virtually identical for the other versions. Experienced IRAF users should have little trouble learning to use FTOOLS because they use the same parameter interface and have the same help files that all other IRAF tasks have.
2 General Characteristics of FTOOLS Tasks
All the FTOOLS tasks share certain common properties. The most obvious characteristic is that they all operate on FITS files. With few exceptions, all the tasks take FITS format files as input and produce FITS format files on output. The main exceptions are the tasks that convert information between FITS and ASCII format files (e.g. fdump and fcreate).
All the FTOOLS also share a common user interface based on the concept of a parameter file (identical to the .par file used in IRAF). The parameter file contains the default or current value of all the parameters needed to run the particular task. Basically when the task is run it first reads the parameter file to get all the parameter values that it needs to perform its task. Depending on how the task is run and on the type of parameter, the user may be queried to confirm that the default values are correct or enter an alternate value. There are basically 2 types of parameters for each task: the required parameters that must be supplied every time the task is run, and the so-called `hidden parameters' that silently take their value from the parameter file, unless the user explicitly overrides them.
A third characteristic of the FTOOLS tasks is that a help file is provide for each one which describes the general purpose of the task and gives detailed information about each of the task parameters. The help files can be viewed on-line by typing `fhelp <task_name>' in the stand-alone environment, or `help <task_name>' in the IRAF environment.
3 A Brief FTOOLS Tutorial (Stand Alone Version)
The following section describes a session of using the stand alone version of FTOOLS on a Unix machine. It is recommended that first time users of FTOOLS repeat this sequence of operations on their own machine, to acquaint themselves with the package.
3.1 Getting Help
General help information about FTOOLS as well as detailed help on each individual task can be obtained by the fhelp task. Type
> fhelp ftoolsto get a brief description of all the tasks available in the FTOOLS package. To get detailed help on any particular FTOOLS task type fhelp followed by name of the task you are interested in. For instance, type
> fhelp fhelpto get a description of how to use the fhelp task. Each help file gives a description of what that task does and of all the required and optional (hidden) parameters. It is important to realize that when running an FTOOLS task you will only be required to enter values for the most important parameters, such as the name of the input FITS file. But most of the tasks also have other 'hidden' parameters which have a predefined value unless you explicitly reset them to a new value. These hidden parameters can significantly alter the operation of the task, so you need to be aware of what hidden parameters are available for a given task and what their default values are. One way to determine this is to read the help file for the task with the fhelp command, but a more concise way to simply see all the task parameters and their current values is with the plist command (or use the `lpar' command when using the IRAF version of FTOOLS). For example:
> plist fhelpshows that the fhelp task has 2 required parameters (`task' and `more') and 4 hidden parameters (`path', `nrows', `outfile', and `mode'; by convention, the names of hidden parameters are enclosed in parentheses). Refer to the help file to determine the meaning and use of each parameter. Note that all tasks have a hidden `mode' parameter but it's value should not be changed by the user.
3.2 Running a task
The simplest way to run a task is to enter its name on the input command line followed by a <CR>, as in,
> fhelpThe task will then prompt you for the value of all the required parameters, in this case the name of the task you want help on. When prompting for a parameter, the task will first display the current value of each parameter as taken from the parameter file. If that value is OK then you can simple hit <CR> to accept it, otherwise enter the new value and hit <CR>. Note that you will not be prompted for any hidden parameters and the task will simply assume the default values contained in the parameter file.
If you know the value of any or all of the required task parameters, another way to run a task it to enter all or some of the parameters directly on the command line in the same order as listed by the plist command. The task will then only prompt you for the values of any remaining non-hidden parameters which were not entered on the command line.
You may override the default value of a hidden parameter in one of 2 ways: by specifying the new value on the command line or by using the pset command. In the above example we saw that the fhelp task has a hidden parameter called `nrows' which controls the amount of text shown on the screen. If you type
>fhelp fdump nrows=10then the help file for the fdump task will be displayed 10 rows at a time, rather than the default 22 rows at a time. This command syntax only sets the hidden parameter value for this single run of the task, and the default value in the parameter (22 in this case) is not changed. If you want to permanently change the value of a hidden parameter, then use the `pset' command, as in
> pset fhelp nrows=10Now, every time you run fdump it will just display 10 rows at a time by default. You can also view and reset the value of every parameter for a tasks simply by typing pset followed by the tasks name, as in
> pset fhelpTo reset the all the parameters for a task back to the original system default values, use the `punlearn' task (analogous to the unlearn command in IRAF):
> punlearn fhelp3.3 Creating and listing the contents of a FITS file
In this section we will describe how to create a FITS table and how to view the structure and content of a FITS file with the following FTOOLS tasks:
First, move to a suitable disk directory where you have space to work. We will assume that there are currently no FITS files available to play with, so the first thing we will do is create one using the fcreate task. To see what input parameters we need, type:
> plist fcreate(If your parameter listing does not look like this, then type `punlearn fcreate' to reset all the parameters back to their system default values). This shows that there are 3 required parameters for this task which are (1) the name of an ASCII column descriptor file, (2) the name of an ASCII data template file, and (3) the name of the output FITS file that is to be created. There are also many hidden parameter that we can safely ignore for the present. You can read the fcreate help file at this point if you want to know more about how it works, but we will simply use it here to create a FITS file without too much further explanation. Using any convenient text editor, you now need to create 2 small text files. First create a column descriptor file, called `cd.file' with the following 3 lines of text (the exact spacing of the items in each row is not important):
NAME 16AThis file defines the names and datatypes of the 3 columns in the table. Then create another text file called `data.file' containing 4 rows of values for the table:
Vega 1 0.0Note that the single quotes are required around the names on the last 2 lines because they contain an embedded space character. Now we can create a FITS file called `test.fits' with the following command:
> fcreate cd.file data.file test.fitsWe can determine the basic structure of this FITS file using the fstruct command:
> fstruct test.fits
No. Type EXTNAME BITPIX Dimensions (columns) PCOUNT GCOUNT 0 PRIMARY 16 0 0 1 1 BINTABLE 8 22(3) 4 0 1 This shows that the FITS file test.fits consists of an empty primary array (indicated by Dimensions = 0) followed by a single binary table extension whose rows are 22 characters wide, with 3 columns and 4 rows of data.
Next we can get information about the columns contained in this FITS table extension by using the flcol command:
> flcol test.fits
This displays the name, data format, and data units (none in this case) of each column of the table.
Finally, we can display the actual header and data values in this FITS file using the fdump task. There are may parameters for this task that can be used to control how much of the file to display and how to format the output (read the help file for the details), but by default fdump will display the entire contents of the FITS file, including both the header and table data, by typing
> fdump test.fitsand then hitting <cr> to accept the default value for the remaining parameters:
Name of optional output file (STDOUT): SIMPLE = T / file does conform to FITS standard < BITPIX = 16 / number of bits per data pixel NAXIS = 0 / number of data axes EXTEND = T / FITS dataset may contain extensions COMMENT FITS (Flexible Image Transport System) format defined in Astronomy COMMENT and Astrophysics Supplement Series v44/p363, v44/p371, v73/p359, COMMENT v73/p365. Contact the NASA Science Office of Standards and COMMENT Technology for the FITS Definition document #100 and other FITS COMMENT information. END
This then shows the value of all the keywords in the primary array header, followed by a listing of the table extension header, followed by a listing of the data values in the table.
The flist task also displays FITS header and table data. However, it uses a row oriented format instead of the column oriented format seen above with fdump. Flist has many parameters for controlling the presentation and formatting of the displayed data. See the help page for all the details. Using the defaults results in all the data being displayed
> flist test.fits Name of optional output file (STDOUT): Names of columns (): Lists of rows (-):The same information is given as before with the data in each column labeled and appearing on a single line. This can be a more comprehensible presentation with some data sets.
In this next series of FTOOLS tasks, a set of (X,Y) events will be binned up into an image using fcreate and f2dhisto. This image will then be displayed with fimgdmp. Here is how it all works.
First create a new column descriptor file named cd2d.file to format two floating point data columns for the X and Y position of the events by placing the following lines in cd.file
X E
Then create a data template file named data2d.file with the following list of (X,Y) events
0.5 0.5 As before the FITS file can be created with fcreate and then viewed with either fdump or flist.
> fcreate cd2d.file data2d.file test2d.fitsNow lets bin this data up into a histogram(image) using f2dhisto. F2dhisto has many parameters for determining the exact binning. Lets bin the data up into a 3 by 3 grid with (0 -> 1) x (0 x 1) being the first bin, (0 -> 1) x (0 -> 2) being the second bin, etc., up to the last bin (2 -> 3) x (2 -> 3). This will require resetting two of the hidden parameters in f2dhisto. Resetting hidden parameters can be accomplished in many ways. The simplest is to set them from the command line
> f2dhisto test2d.fits image.fits 1.0 1.0 xrange='0,3' yrange='0,3'Here the first parameter is the input file, image.fits is the output file which will contain the histogram in the primary array, the following pair of 1.0 values give the X and Y bin sizes respectively and finally the two hidden parameters xrange and yrange specifying the minimum and maximum are given. This is the format used to assigning all hidden parameters; parameter=value. The quotes are used to guarantee the whole string is passed to the parameter.
The f2dhisto command above should have binned up the events into a 3 x 3 histogram. This is easily checked using fstruct again
> fstruct image.fits
Finally, we can append this new selected table back onto the end of the original FITS file with the fappend task:
> fappend new.fits+1 test.fitsRepeating the previous fstruct command, we see that the file now contains 2 extensions following the primary header:
> fstruct test.fits No. Type EXTNAME BITPIX Dimensions(columns) PCOUNT GCOUNTThis concludes this tutorial, but we invite you to further explore the FTOOLS package on your own, using your own FITS data files.
Appendix
A. FTOOLS Task Summary
The current release of FTOOLS has the following tasks implemented within the four categories; general, parameters, astrophysics and calibration.
General Purpose Utilities:
fhelp - Display help pages for an FTOOL task
flcol - List column information in a FITS table extension
fstruct - List a description of the structure of a FITS file
fdump - Dump contents of a FITS table to an ASCII file
flist - List contents of a FITS table to an ASCII file
fimgdmp - Dump contents of a FITS image to an ASCII file
ffilecat - Copies keyword values from a list of FITS file to FITS Table
fcatdiff - Compares columns of a fits file and reports row differences
fcreate - Create a FITS table from ASCII input files
fappend - Append a FITS extension onto another FITS file
fextract - Copy a FITS extension from a file into a new file
fmerge - Merge rows from several FITS tables into one FITS table
ftabcopy - Copy specified columns of a FITS table to a new table
fselect - Create a new table from selected rows of a table
fsaoi - Translate an SAOImage region file to an input file for fselect
fsort - Sort a FITS table in place
fmemsort - Fast memory sort of a FITS table
fcalc - Calculates values for a column using an arithmetic expression
flookup - Filter a FITS file from selection based on a lookup table
fstatistic - Calculate mean, standard deviation, min, and max for a column
farith - Perform arithmetic on 2 FITS images
fcarith - Perform arithmetic on FITS image with a constant
fhisto - Make a histogram of a column in a table
fcurve - Make a light curve histogram from a column in a table
f2dhisto - Make a 2-D histogram from 2 columns in a table
fim2lst - Convert a 2D image to a pixel list (inverse of f2dhisto)
fmrgmsk - Merge 2 or more spatial masks
fmaskfilt - Filter an event list based on an input mask image
findex - Create an index file for a FITS table column
fparkey - Copy a parameter value to a FITS header keyword
fkeypar - Copy a FITS header keyword to a parameter
fpartab - Copy a parameter value to a FITS table element
ftabpar - Copy a FITS table element to a parameter value
ftabkey - Copy a FITS table element to a FITS header keyword
fkeytab - Copy a FITS header keyword to a FITS table element
fmodhead - Modify the header keywords in a FITS file
fkeyprint - Display keyword(s) in FITS header(s)
fburst - Remove bursts of events from time ordered event list
fplot - Plot columns in a FITS file using QDP/PLT package ***
Parameter file manipulation tasks:
Appendix B. How to get FTOOLS
The FTOOLS software is being distributed as compressed tar files and is available through anonymous ftp over the internet from heasarc.gsfc.nasa.gov (128.183.8.233). There you will find documentation and the source for Suns, DECstations, Alpha workstations and VAX computers. Here's what a typical ftp session might look like:
myhost> ftp legacy Name (legacy:kent): anonymous 331 Guest login ok, send your complete e-mail address as password. Password: userid@hostname ftp> cd software/ftools ftp> cd release ftp> ls *.ps.Z Install.v2.6.ps.Z Develop.v2.6.ps.Z Miniusers.v2.6.ps.Z Users.v2.6.ps.Z ftp> ls ftools.* ftools.dec.v2.6.1.tar.Z ftools.osf.v2.6.1.tar.Z ftools.sun.v2.6.1.tar.Z ftools.vms.v2.6.1.tar.Z ftp> bin ftp> get ftools.dec.v2.6.1.tar.Z ftp> get Install.v2.6.ps.Z ftp> get Miniusers.v2.6.ps.Z ftp> quit myhost>You should now have a copy of the compressed tar file for the DECstation's FTOOLS source code and enough documentation to get you started using the FTOOLS in your current directory.
If you are using a VMS system, you will need both an uncompress and an untar program. Both are available in public domain in many different versions via anonymous FTP. Several different compress/uncompress programs can be found at ftp.spc.edu in the [.MACR032] directory, and an untar program is available from vmsa.oac.uci.edu. This installation assumes that the untar program translates illegal filenames to legal ones. For example, the UNIX /ftools/bin.host directory is assumed to be renamed to Some untar programs may not correctly untar files with "unusual" structures.
The two files which have previously caused problems, grfont.dat and crabk.fits, are available for direct retrieval in the same directory as the main FTOOLS tar file. Grfont.dat belongs in ftools:[host] and crabk.fits in ftools:[testdata]. Neither are essential to the installation or running of FTOOLS.
Proceed to the next article
Return to the previous article
HEASARC Home | Observatories | Archive | Calibration | Software | Tools | Students/Teachers/Public Last modified: Wednesday, 20-Oct-2021 10:49:28 EDT |
Select another article