NAME

batclean - read in a DPI and clean out a fit to the diffuse background

USAGE

batclean infile outfile incatalog [detmask] [options]

DESCRIPTION

This tool fits a simple background or source plus background model to a detector plane image. The code contains a default background model which is described below. The user can also specify one or more background models to be fit using the bkgmodel parameter. The output file can be either the cleaned focal plane (default), the fit to the detector plane or the original input file.

The tool can also, optionally, balance the detector plane either before or after or instead of cleaning. See below under the "balance" parameter for more details.

This code reads in a Detector Plane Image (DPI) file as well as an optional detector mask file. If no detector mask file is provided, then all detectors with non-zero data are included in the fit. The tool works on either flight or simulated flight data (sources at infinity) or ground calibration data (sources at a finite distance). See the description of the bat_z parameter for more about cleaning of sources at a finite distance.

Source Cleaning

If one or more sources are to be cleaned from the DPI (default), then the tool reads the source locations from the incatalog file and then calls the routine maskwtimg which uses a forward projection to create a model of the source exposure through the aperture onto the detector plane.

Background Cleaning

The tool creates a simple eighteen parameter background model, which is described as follows:

The code creates a background exposure map based on the model above (which can be optionally output using the backexp parameter). In addition the user can supply his own background model to be fit in addition to or in place of the standard background model.

The tool then uses a singular value decomposition method to fit the source exposure and background exposure maps to the data from the DPI. The coefficients from the fit are written to the output file as keywords.

The fit coefficients are then used to create a model background DPI. This is then subtracted from the original input DPI to produce a cleaned DPI. The code writes keywords giving the fit parameter for each background parameter (simple plus user supplied) and source parameter that is fit. These keywords are BGPARnn for background and SRPARnn for sources. The number nn goes from 0 to the total number of parameters, starting with the hard-coded background parameters.

PARAMETERS

infile [filename]
Input file name. The name of the input DPI file. This is expected to be an image array with dimensions equal to the number of detector columns by the number of detector rows (set to 286 X 173). If there are multiple BAT_DPI image extensions then all are cleaned and multiple cleaned images are written to the output file.

outfile [filename]
Output file name. This will be written in the DPI format. The default is to write out the cleaned image, but other outputs can be selected using the outversion parameter. Precede the output file name with an exclamation point, "!", (or "\!" on the Unix command line), to overwrite a preexisting file with the same name (or set the clobber parameter to YES).

incatalog [filename]
Input source catalog. The format of the catalog file is that output by the BAT tool batcelldetect. In particular the catalog must contain columns "IMX," "IMY," "CATNUM," (or "SOURCE_ID"), "NAME," and "SNR." The tool reads the source position in BAT image coordinates from this file and uses the source position to forward project a model of the source flux onto the focal plane. This parameter may be omitted or set to "NONE" if only background cleaning is required (i.e., the srcclean parameter set to "NO"). The SRPARnn keywords giving the fit parameter for each source contain the source name in the comments field. Allowed extension names for incatalog are "BAT_CATALOG" (default of batcelldetect), "TRIG_SOURCE" (BAT on-board catalogs) and "INPUT_CATALOG."

(aperture) [filename]
Aperture file. The default is the aperture file in the CALDB area. This parameter may be omitted or set to "NONE" if only background cleaning is required (i.e., the srcclean parameter set to "NO"). If several aperture types are available in the CALDB, the user can specify CALDB:apertype. Valid values of apertype are currently: FLUX or DETECTION, for apertures optimized for flux reproducibility or detection sensitivity, respectively.

(detmask) = "NONE" [filename]
Name of a detector mask file. This should be an image file with the same dimensions as the focal plane map. A pixel value of 0 indicates the detector is enabled for imaging, and a value of 1 indicates disabled, noisy, or otherwise selected for elimination from fits. A default value of NONE implies all detectors are on, except for the BAT detector gap regions.

It is strongly recommended to use a detector mask file if one is available. The quality of the fit, especially to calibration data is greatly improved if only working detectors are included in the fits.

(wtmapin) = "NONE" [filename]
Name of an input error map. This is used to provide weights for the fitting in case one is cleaning a DPI which does not have Poisson statistics (such as an already cleaned DPI). The typical sequence would be to create an error map using the wtmapout parameter on the first cleaning and then read this error map in on the second cleaning. If an already cleaned image is cleaned without an input error map than uniform weighting is used.

(wtmapout) = "NONE" [filename]
Name of an output error map. If a filename is specified then the tool writes out a DPI containing the calculated weights for each detector. This can be used on subsequent cleanings as the input wtmapin error map.

(backexp) = "NONE" [filename]
If the name of a file is given here, then the background model will be output to this file. The format is a binary table with one column for each model parameter. The data is written in DPH format, or as a 286 X 173 element array, one number per detector. This table gives the model which was fit to the data, not the actual fit to the data. The final column in the table is the covariance matrix from the fit.

(bat_z) = 0 [real]
This is the z-component of the distance to the source in BAT_Z coordinates (the vertical distance of the source from the origin of the BAT coordinate system). This parameter is ignored if srcclean = "NO." If the incatalog contains a BAT_Z keyword, then the value of the keyword is used and the value of the bat_z parameter is not used. Note that the default value of 0 corresponds to a source at infinity.

(bkgmodel) = "simple" [string]
This optional parameter indicates the background model to be fit. Currently only two models are hard coded: the simple model which is the 14 parameter fit described above and an even simpler test model which was used in testing the fit procedures. As more sophisticated background models are developed, they will be incorporated into the code.

The user can also use this parameter to supply a file or list of files containing user-developed background models to be fit. This can be input as either a comma-separated list of model files or a text file (preceded by the "@" symbol) containing a list of input model files, one per line. These model files must be image files in the same format as a BAT DPI (286 columns by 173 rows). There are no required keywords. If a model file is not found or is not an image file, the code will ignore it.

The comment fields of the BGPARnn keywords contain the model names. If an optional MODNAME is supplied, then its value becomes the model name in the comment field. If this keyword is not present, then the model name is the name of the file (not including the path).

The default behavior of the code is to fit the simple model in addition to any user supplied models. The user does not need to also include the string "simple" in the bkgmodel parameter list. If, however, the user wishes to fit ONLY the user-supplied models and not the simple model, then he can request this by adding the string "useronly" to the bkgmodel parameter list. See examples below.

Finally, it is possible not to fit any model at all, but simply to balance the array (using the balance parameter below. To do this set bkgmodel=none.

Note that if the user selects outversion="fit" or "bkgfit", then no balancing is performed regardless of the value of the balance parameter.

(srcclean) = "YES" [boolean]
Determines whether sources are cleaned from the input DPI or only background. If this parameter is "YES," then an incatalog and aperture file must be supplied.

(cleansnr) = 6.0 [real]
Only sources with a signal-to-noise ratio as read from the incatalog higher than this value are cleaned from the DPI. If no sources have a signal-to-noise ratio above the cleansnr value, then only the background will be cleaned.

(snrcol = "SNR") [string]
Name of catalog column to be used with 'cleansnr' for signal to noise thresholding. By default it is "SNR", but any column can be used to trigger cleaning.

(ignore) = "UNKNOWN" [string]
Unless this parameter is set to "NONE" or "none," the cleaning code will not clean any source in incatalog whose NAME matches the ignore string. The user can either specify a particular source name, the strings "IGNORE" or "NONE", or the name of a file (preceded by "@"). If a file name is supplied, then the file must contain a list of sources to be ignored, once source per line. Note that source names to be ignored must match exactly (including spaces, capitalization, etc.) the names in the catalog.

(balance) = "none" [string]
Unless this parameter is set to "NONE" or "none," the cleaning code will, after cleaning, also balance the entire array or subsets of the array as specified below. Balancing means subtracting the mean for a set of detectors from the values for that set of detectors. In other words, it sets the average for the detectors to zero. If a detector mask (detmask) is given, then only non-masked detectors are included in the balancing.

The balancing that is requested is given by supplied a comma-separated list of any or all of the following values. The balance is performed in the order in which the values are listed. Balancing is not commutative, so order is significant.

Allowed balance values:

There is further illumination of the balancing in the comments in the code.

(balfirst) = "no" [boolean]
Normally, balancing is done after cleaning. However, by setting this parameter to "yes" then balancing can be done before cleaning.

(maskfit) = "no" [boolean]
If this parameter is set to "YES" and outversion="fit" then the output file contains the fit parameters for all detectors even those masked using a detmask. The default behavior is that the fit is output only for unmasked detectors. Detectors will still masked for cleaning and balancing when this option is selected.

(leadedge) = "NO" [boolean]
If this parameter is set to "YES," then the leading edge detectors (those on the two edges of detector module sides facing toward the source) are fit separately from the rest of the detectors. Since the current forward projection code does not correctly handle these edge detectors, setting this parameter does not significantly affect the overall clean.

(eff_edge) = 0.2 [float]
The depth of edge (cm) to consider effective. Range from 0.03 to 0.2

(corrections = "none") [string]
Comma-separated list of corrections to apply to the mask weighting, or "none" if no corrections are to be applied. Note that corrections are only applied to the forward projection for sources. This parameter has no effect if only the background is to be cleaned.

Possible correction types are: "flatfield" (basic flat fielding correction accounting for both cosine-type and r-squared-type effects); "nearfield" (for ground testing results where the source is at a finite distance -- requires bat_z != 0); "cosine" (cosine effects of off-axis illumination only); "rsquare" (r-squared effects only); "opaque" (activates a new algorithm for calculating the mask transmission function); "sides" (effects of sides of detectors XXX not implemented).

(outversion) = "cleaned" [string]
This optional parameter indicates what detector plane image is written to the outfile. The default is to write the cleaned DPI (input minus fit). Other options are:

Choice of outversion parameter dictates what value is written to the HDUCLAS2 keyword. Possible values are:

(clobber = NO) [boolean]
If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.

(chatter = 1) [integer, 0 - 5]
Controls the amount of informative text written to standard output. Setting chatter = 4 or higher will produce detailed diagnostic output; chatter = 2 prints out a basic diagnostic message. The default is to produce no printed output.

(history = NO) [boolean]
If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the ftcopy task parameters that were used to produce the output file.

EXAMPLES

Note that when commands are issued on the Unix command line, strings containing special characters such as '[' or ']' must be enclosed in single or double quotes.

The following examples illustrate ways in which batclean can be used.

1. Fit the simple background model to a DPI with a detector mask file. Output the cleaned DPI and a background exposure map. 

    batclean input.dpi cleaned.dpi detmask=bat.mask backexp=backexp2.dph 
srcclean=NO
2. Fit a set of source models along with a simple background model to a DPI, using the default aperture file. Output the fit to the focal plane instead of the cleaned focal plane. A flat-fielding correction is applied in the forward projection. 

    batclean input.dpi fit.dpi catalog.tbl detmask=bat.mask outversion="fit"
corrections="flatfield"
3. Fit two user-supplied background models in addition to the hard-coded model. 

 
   batclean input.dpi cleaned.dpi detmask=bat.mask bkgmodel=model1.dpi,model2.dpi
4. Same example, except the user gives the two model files as lines in the text file "models.lst." 

   batclean input.dpi cleaned.dpi detmask=bat.mask bkgmodel=@models.lst
5. Balancing is performed after cleaning, applying the Module and InOut balancings. 

    batclean input.dpi cleaned.dpi detmask=bat.mask balance=Module,InOut

BUGS

SEE ALSO

batcelldetect, batmaskwtimg

REFERENCES

Based on Netlib code http://www.netlib.org/c/numcomp-free-c 

Name        : svd.c.Z (8704 bytes)
Where       : in pub/C-numanal on ftp.usc.edu
Description : SVD based on pascal from J. C. Nash book
Author      : Bryant Marks (bryant@sioux.stanford.edu)
              Brian Collett (bcollett@hamilton.edu)
Version     : 14 April 1993

Statistics for low count rates from K. Mighell (ApJ 518, p. 380, 1999)

LAST MODIFIED

Nov 2007