Column and Keyword Filtering Syntax

The column filtering specifier can be used to create or modify table columns and/or header keywords in the input HDU. The presence of a column filter causes CFITSIO to create a new virtual file (usually in memory) that contains the specified modifications. Any other HDUs in the input file are also copied to the virtual file. The actual physical input file is not modified.

The column filtering specifier must begin with the keyword 'col' and is enclosed in square brackets following the name of an input FITS file. The column filter can be used to perform the following types of operations. More than one operation may be specified by separating them with semi-colons.

Select a subset of columns from the input table

The list of columns to be included in the virtual table should be separated by semi-colons. For example, 
    file.fits[events][col X; Y]
will create a virtual copy of the input events table that only contains the X and Y columns. The '*' and '$' wildcard characters may be used to match the names of more than one column to be selected. It is sometimes more convenient to list the columns to be excluded from the virtual table, as in: 
    file.fits[events][col -DETX; -DETY]
 or
    file.fits[events][col !DETX; !DETY]
These examples will copy all but the DETX and DETY columns to the virtual table.

Compute the values for a column

Append a new column to the table or recompute the values in an existing column. For example, 
    file.fits[events][col PI = PHA * 1.05; X; Y]
will modify the values in the PI column, (or will create a new PI column if it does not already exist) using the arithmetic expression 'PHA * 1.05' (where PHA is a column in the input table) to calculate the values for the column. The expression that is used to compute the columns or keyword values can be arbitrarily complex and may be a function of other header keyword values and other columns. The full syntax and available functions for the expression are described in the calc_express help file.

In the above example, the virtual file will contain only 3 columns: PI, X, and Y. The '*' wildcard character may be used when specifying the name of the columns to be copied to the virtual file, as in these examples: 

    file.fits[events][col PI = PHA * 1.05; X; Y; *DET]
    file.fits[events][col PI = PHA * 1.05; *]
In the first example, the virtual table will contain the PI, X, and Y columns and any other columns whose name ends with the string 'DET'. In the second example, the virtual table will contain all the columns present in the input table, as well as the PI column.

When a new column is created, an appropriate data type will be chosen by default based on the form of the expression used to calculate the values. Alternatively, one may specify the desired column data type in parentheses, followed the column name, as in '[col PI(I) = PHA * 1.05]'. The datatype is specified using the same code letters that are allowed for the value of the FITS TFORMn keyword (e.g., 'I' for integer*2, 'J' for integer*4, 'E' for real*4, and 'D' for real*8, etc. for binary tables, and 'I8', F12.3', 'E20.12', etc. for ASCII tables). The full range of FITS data types is supported including character string, logical, bit, byte and complex data types. Vector columns are also supported in binary tables.

Create or modify a header keyword

Create a new header keyword or modify the value of an existing keyword by prefixing the keyword name with a pound sign '#'. The expression must evaluate to a constant scalar (i.e., cannot have a column name in the expression). The comment string for the keyword may be specified in parentheses immediately following the keyword name (instead of supplying a datatype as in the case of creating a new column). For example: 
    file.fits[events][col #VEL(radial velocity) = 23.4]
will create the keyword 
  VEL     =                   23.4 / radial velocity

Rename a column

Rename an existing column or keyword with the syntax 'NewName == OldName', as in: 
    file.fits[events][col NEWPHA == PHA]

Using a text file

For complex or commonly used operations, one can also place the operations into a text file and import it into the column filter using the syntax '[col @filename.txt]'. The operations can extend over multiple lines of the file, but multiple operations must still be separated by semicolons. Any lines in the external text file that begin with 2 slash characters ('//') will be ignored and may be used to add comments into the file.

SEE ALSO

filenames, imfilter, rowfilter, binfilter. pixfilter. calc_express.