10. FITSification of OSSE Data Files

SDB2FITS & FITS2SDB converters: Need to be operational on low and high- level SDB files.

PDF2FITS & FITS2PDF are having problems since INGRES was removed from COSSC. Apparently the program TARGS.C collects some data from the archive through SQL calls and uses it for adding some keywords to the FITSified PDF files.

The basic converters utilize the ToFU and FITSIO libraries. The ToFU templates for the FITSification can be found in Section 10.1.

10.1 SDB2FITS

The SDB2FITS program converts a single OSSE SDB file into a single FITS file. Each AUX type in the SDB file is converted into an individual binary table extension, i.e. all the individual AUX 00 spectral records will be converted into a single binary table.

Calling Sequence:
SDB2FITS [infile] [outfile] [view_period]

infile:

Input SDB file name

outfile:

Output FITS file name

view_period:

CGRO viewing period number, with decimal place

Platform:

VAX/VMS

Logicals Used:

OSSE_CAT, OSSE_SDB_TEMPLATE

Data Files Used:

Subprograms/Libraries Called:

ToFU, FITSIO

10.1.1 Program Outline

This converter acutally works in two parts. The first part is SDB2PREP which separates the individual AUX types into separate files. The PREP2FITS portion of the code looks for each of these temporary files and constructs a binary table extension for each of them.

1.

10.2 FITS2SDB

Calling Sequence:
FITS2SDB [infile] [outfile]

infile:

Input FITS file

outfile:

Output SDB file

Platform:

VAX/VMS

Logicals Used:

Data Files Used:

Subprograms/Libraries Called:

ToFU, FITSIO

10.2.1 Program Outline

1.

10.3 SDB_COMPARE

SDB_COMPARE compares an original OSSE SDB data file and a reconverted data file (original -> FITS -> reconverted) record by record and byte by byte to determine if the FITS conversion process was correctly performed.

Calling Sequence:

SDB_COMPARE [file1] [file2] [verbose]

file1:

compare this SDB file

file2:

to this SDB file

verbose:

Platform:

VAX/VMS

Logicals Used:

SDB_CONVERTERS

Data Files Used:

SDB_CONVERTERS:SDB_BYTE_EXCEPTIONS.DAT

Subprograms/Libraries Called:

ToFU

10.3.1 Program Outline

SDB_COMPARE compares the first record of each file and exits if not identical. It then grabs the next record from both the original and reconverted files and verfies that both have the same index record key (first 8 bytes of each record). If the index keys are not the same, then it's possible that the original file had a dummy header record inserted which the FITS and reconverted files do not contain; if this is the case then the dummy header record is ignored and the next record from the original file is read; if the original file record is not a dummy header record then SDB_COMPARE exits with an error message. If the index keys match then the record lengths are checked to make sure they are the same. If the record lengths match then the record contents are comparied byte by byte.

The only instance where the condition

byte(original\_data\_file) != byte(reconverted\_data\_file)

is allowed is when the bytes belong to a real*8 field. Due to differences in VAX and IEEE R*8 representations, the reconverted R*8 bytes can differ from the original R*8 bytes. When a byte mismatch is found in a R*8 field, SDB_COMPARE reconstructs the R*8 numbers and compares the relative error between original and reconverted values; the relative error must be within a specified tolerance for the mismatch to be acceptable. the file

SDB\_CONVERTERS:SDB\_BYTE\_EXCEPTIONS.DAT

contains a list of byte start positions for the R*8 values.

SDB_COMPARE writes error messages to a log file. It looks for a log file of the form ORIGINAL_DATA_FILE_NAME.log to append its output too, and creates such a file if it does not already exist.

If the VERBOSE flag is set to .TRUE. then SDB_COMPARE provides the user with extra status information, both to the screen and to the log file. This is useful for debugging.

1.

10.4 SDB_PROCESS

The FITS converters described in the previous sections are called in a batch process, SDB_PROCESS, which takes a delivery of raw SDB data files and converts them to FITS.

Platform:

VAX/VMS

Logicals Used:

Data Files Used:

Subprograms/Libraries Called:

SDB2FITS, FITS2SDB, SDB_COMPARE

10.4.1 Program Outline

1.

Setup DCL logicals.

2.

Build destination path.

3.

Now loop over each SDB file.

(a)

Create the destination directory, if necessary.

(b)

Run SDB2FITS on the native file, creating the FITS file. Record an entry in the log file if it fails.

(c)

Run FITS2SDB on the FITS file creating RECONV.TMP. Record an entry in the log file if it fails.

(d)

Run SDB_COMPARE on the native file and RECONV.TMP. Record an entry in the log file if it fails.

(e)

If the above steps ran without errors, write an entry in the success log and delete the SDB file in the native directory.

4.

Erase the zero-length log files in the input and processing area.

5.

Mail the log files to the host account.

10.5 PDF2FITS

Platform:

VAX/VMS

Logicals Used:

Data Files Used:

Subprograms/Libraries Called:

10.5.1 Program Outline

1.

10.6 TARGS

Platform:

Ultrix

Logicals Used:

Data Files Used:

Subprograms/Libraries Called:

10.6.1 Program Outline

1.

10.7 FITS2PDF

Platform:

VAX/VMS

Logicals Used:

Data Files Used:

Subprograms/Libraries Called:

10.7.1 Program Outline

1.

10.8 PDF_COMPARE

Platform:

VAX/VMS

Logicals Used:

Data Files Used:

Subprograms/Libraries Called:

10.8.1 Program Outline

1.

10.9 PDF_PROCESS

Platform:

VAX/VMS

Logicals Used:

Data Files Used:

Subprograms/Libraries Called:

PDF2FITS, FITS2PDF, PDF_COMPARE, TARGS

10.9.1 Program Outline

1.

Setup DCL logicals.

2.

Build destination path.

3.

Clean out work files from temporary area.

4.

Now loop over each PDF file.

(a)

Create the destination directory, if necessary.

(b)

Run PDF2FITS on the native file, creating the FITS file. Record an entry in the log file if it fails.

(c)

Clean out temporary area.

(d)

Submit targs.c to Ultrix system to build additional FITS primary header info from the INGRES database. (INGRES is no longer supported so this call is a source of much grief.)

(e)

Run FITS2PDF on the FITS file creating DECONV.TMP. Record an entry in the log file if it fails.

(f)

Run PDF_COMPARE on the native file and DECONV.TMP. Record an entry in the log file if it fails.

(g)

If the above steps ran without errors, write an entry in the success log and delete the SDB file in the native directory.

5.

Clean out the temporary area.

6.

Erase the zero-length log files in the input and processing area.

7.

Mail the log files to the host account.

10.10 Basic Operations of the FITS Converters

Here are some basic notes on how the FITS converters operate which may be important if modifications are needed in the future. There are 7 basic classes of aux**_extract routines:

Class 0:

datatype entities are array elements

Class 1:

datatype entities are structures, number of data records is not necessially equal to the number of structures

2.

datatype entities are structures, number of data records must be equal to the number of structures (obsolete)

4.

datatype is undefined

5.

datatype entities are array elments, elements may require I*4 to I*2 conversin (AUX 64 only)

6.

datatype entities are structures, number of data records is not necessially equal to the number of structures, elements within the structures may require I*4 to I*2 conversion (AUX 65 only).

7.

datatypes are classified as "special" and require non-standard processing (AUX 38 and 39)

 

Table G.1: Subroutine summary for OSSE FITS converters (Part 1)

AUX	Extension	Entity	SDB2FITS	FITS2SDB	Comments
Type	name	type	convert	extract
00	SINGLE_SPECT_R	element(0)	custom	custom	(see note 1)
01	SGL_SPEC_STAT_+UNC	element(0)		auxclass0
02	SGL_SPEC_STAT_-UNC	element(0)		auxclass0
03	SGL_SPEC_SYS_+UNC	element(0)		auxclass0
04	SGL_SPEC_SYS_-UNC	element(0)		auxclass0
05	CHNL_LOW_EDGE_ENG	element(0)		auxclass0
06	CHNL_LOW_EDGE_UNC	element(0)		auxclass0
07	CHNL_ENERGIES	element(0)		auxclass0
08	CHNL_ENERGIES_UNC	element(0)		auxclass0
09	CHNL_WIDTHS	element(0)		auxclass0
10	CHNL_WIDTHS_UNC	element(0)		auxclass0
11	PSD_EFFCIENCY	element(0)		auxclass0
12	PSD_EFFCIENCY_UNC	element(0)		auxclass0
13	DET_RESOLUTION	element(0)		auxclass0
14	DET_RESOLUTION_UNC	element(0)		auxclass0
15		undefined(2)
16		undefined(2)
17		undefined(2)
18		undefined(2)
19	MULTI_SPECT_DSF	structure(1)		custom
20	MULTI_SPECT_R	structure(1)	custom	custom
21	M_SPEC_STAT_+UNC	structure(1)		custom
22	M_SPEC_STAT_-UNC	structure(1)		custom
23	M_SPEC_SYS_+UNC	structure(1)		custom
24	M_SPEC_SYS_-UNC	structure(1)		custom
25	CAL_LINE_INFO	structure(1)		auxclass1

Note 1: Used in conjunction with AUX types 64 & 66(formerly) to handle R*4 and I*4 data types in spectra

 

Table G.2: Subroutine summary for OSSE FITS converters (Part 2)

AUX	Extension	Entity	SDB2FITS	FITS2SDB	Comments
Type	name	type	convert	extract
26		undefined(2)
27	PSD_MODEL_INFO	structure(1)		custom
28	RELATED_SPECTRA	structure(1)		auxclass1
29	ORBIT_HISTORY	structure(1)		auxclass1
30	FOOTNOTE	structure(1)		auxclass1
31		undefined(2)
32	EST_BACKGROUND	element(0)		auxclass0
33	EST_BACKGROUND_UNC	element(0)		auxclass0
34	AU_DEFINITION	structure(1)		auxclass1
35	AU_IDENT	structure(1)		custom
36	SELECT_SPECT_LIST	structure(1)		custom
37	COMPOUND_RESP_COEF	structure(1)		auxclass1
38	RESP_MATRIX	special(3)	custom	custom
39	INST_PP_RESP_VECT	special(3)	custom	custom
40		undefined(2)
41		undefined(2)
42	SDB_DIRECTORY	structure(1)	custom	custom
43	RESP_MATRIX_DEF	structure(1)		auxclass1
44	RESP_MATRIX_VECTORS	special(3)	custom	custom
45	COVAR	structure(1)		auxclass1
46		undefined(2)
47		undefined(2)
48		undefined(2)
49		undefined(2)
50		undefined(2)

Note 1: Used in conjunction with AUX types 64 & 66(formerly) to handle R*4 and I*4 data types in spectra

 

Table G.3: Subroutine summary for OSSE FITS converters (Part 3)

AUX	Extension	Entity	SDB2FITS	FITS2SDB	Comments
Type	name	type	convert	extract
51		undefined(2)
52		undefined(2)
53		undefined(2)
54		undefined(2)
55		undefined(2)
56		undefined(2)
57		undefined(2)
58		undefined(2)
59		undefined(2)
60		undefined(2)
61		undefined(2)
62		undefined(2)
63		undefined(2)
64	SINGLE_SPECT_J		custom
65	MULTI_SPECT_J		custom
66	SINGLE_SPECT_I		custom

Note 1: Used in conjunction with AUX types 64 & 66(formerly) to handle R*4 and I*4 data types in spectra