8.10 Column and Keyword Filtering Specification

The optional column/keyword filtering specifier is used to modify the column structure and/or the header keywords in the HDU that was selected with the previous HDU location specifier. This filtering specifier must be enclosed in square brackets and can be distinguished from a general row filter specifier (described below) by the fact that it begins with the string 'col ' and is not immediately followed by an equals sign. The original file is not changed by this filtering operation, and instead the modifications are made on a copy of the input FITS file (usually in memory), which also contains a copy of all the other HDUs in the file. This temporary file is passed to the application program and will persist only until the file is closed or until the program exits, unless the outfile specifier (see above) is also supplied.

The column/keyword filter can be used to perform the following operations. More than one operation may be specified by separating them with commas or semi-colons.

The expression that is used when appending or recomputing columns or keywords can be arbitrarily complex and may be a function of other header keyword values and other columns (in the same row). The full syntax and available functions for the expression are described below in the row filter specification section.

If the expression contains both a list of columns to be included and columns to be deleted, then all the columns in the original table except the explicitly deleted columns will appear in the filtered table. If no columns to be deleted are specified, then only the columns that are explicitly listed will be included in the filtered output table. To include all the columns, add the '*' wildcard specifier at the end of the list, as shown in the examples.

For complex or commonly used operations, one can also place the operations into an external text file and import it into the column filter using the syntax '[col @filename.txt]'. The operations can extend over multiple lines of the file, but multiple operations must still be separated by commas or semi-colons. Any lines in the external text file that begin with 2 slash characters ('//') will be ignored and may be used to add comments into the file.

Examples:

   [col Time, rate]              - only the Time and rate columns will
                                   appear in the filtered input file.

   [col Time, *raw]              - include the Time column and any other
                                   columns whose name ends with 'raw'.

   [col -TIME; Good == STATUS]   - deletes the TIME column and
                                   renames the status column to 'Good'

   [col PI=PHA * 1.1 + 0.2; #TUNIT#(column units) = 'counts';*]
                                 - creates new PI column from PHA values
                                   and also writes the TUNITn keyword
                                   for the new column.  The final '*'
                                   expression means preserve all the
                                   columns in the input table in the
                                   virtual output table;  without the '*'
                                   the output table would only contain
                                   the single 'PI' column.

   [col rate = rate/exposure, TUNIT#(&) = 'counts/s';*]
                                 - recomputes the rate column by dividing
                                   it by the EXPOSURE keyword value. This
                                   also modifies the value of the TUNITn
                                   keyword for this column. The use of the
                                   '&' character for the keyword comment
                                   string means preserve the existing
                                   comment string for that keyword. The
                                   final '*' preserves all the columns
                                   in the input table in the virtual
                                   output table.