next up previous contents index
Next: 4. Suzaku Function Libraries Up: Suzaku Project Data Management Previous: 2. Observations Types   Contents   Index


3. Software Principles

In this chapter, we present the Suzaku software principles and agreements, which all the software developers need to follow throughout the Suzaku project.

3.1 General Software Design Principles

Suzaku data analysis system should share the same design principles with all the other projects conducted under OGIP. These design principles may be summarized as follows:

  1. Standard and portable data format -- FITS (Flexible Image Transport System) format is adopted for all the binary files. System dependent binary files will never be used. Moreover, the existing OGIP conventions should be followed wherever possible, and new conventions should be submitted to HEASARC FITS Working Group to check for consistencies with other missions. Use of ASCII format is allowed for small files.

  2. Universal and unique software -- There should not be multiple channels of the data analysis. Software releases are controlled, and the same routines used for the instrumental calibrations by hardware teams are used for the scientific data analysis by Guest Observers.

  3. Designed for multimission analysis -- Existing software infrastructures will be utilized as much as possible. Users will be able to analyze Suzaku data with standard high-level X-ray data analysis packages such as XSPEC, XIMAGE, XRONOS, etc.

  4. Easy to install and use -- The software will be easy to install and use, and extensive help, support and documentation will be provided. Suzaku specific software for low-level tasks are distributed in the standard FTOOLS package, providing user friendly interface on most standard platforms (section 3.3.3).

  5. Free and public software -- Users will not have to purchase any commercial software packages (such as IDL), and all the source codes will be open and easily available at free of charge. Users will not have to worry about license issues, and software authors shall not claim any privileges or credits. Users may modify and distribute Suzaku software freely on their responsibility.

3.2 Suzaku Specific Design Principles

In addition to the general design principles above, Suzaku GOF and ISAS propose the following design principles for the Suzaku software/data processing system. They reflect the experiences from the ASCA mission.

  1. The raw telemetry will be converted to FITS format before distribution. There is only one set of software (mk1stfits; section 6.3.1) to access and interpret the raw telemetry data and to convert them to the FITS format (First FITS Files; section 6.3). Mk1stfits, as well as other processing software, must be fully tested and ready before the launch of the satellite.

  2. All the calibration and data processing should start from the First FITS Files. To that end, the First FITS Files should reflect the original structure of the raw telemetry as much as possible.

  3. All the scientific analysis starts from the standard calibrated FITS files. The First FITS Files are further processed by the standard software with instrumental calibration information, and the Calibrated FITS Files (section 6.3) are produced. Scientific outputs are produced always from the official Calibrated FITS Files, and there should not be other routes for scientific analysis.

  4. The same processing system to calibrate the First FITS files should run at GSFC and ISAS. Thereby, US and Japanese Suzaku Observers shall receive the identical Calibrated FITS Files.

  5. Important calibration tools/software should be made promptly available to GOs. At any given time, there shall be always a single version of the official instrument calibration files and software controlled by the Suzaku GOF and instrument teams. This ensures that the Suzaku Observers can apply the latest calibration information to the observing data.

  6. Suzaku software will be written by the Suzaku software and hardware teams at GSFC, ISAS and other institutions in Japan. Tasks which require deep understanding of the Suzaku instruments, spacecraft and telemetry formats will be mainly written by the members of the hardware teams and ISAS. On the other hand, higher level tasks, in which user-friendliness, standardization and conformity with other high energy missions should have a high priority, will be mainly written by the software team at GSFC.

  7. All the software for public release will be delivered to Suzaku GOF before the release. Suzaku GOF will ensure that the software follow the rules presented in this chapter, and will package them in a form which is suitable for general release. Suzaku GOF will be responsible for releasing and maintaining the packages. When modifications or bug fixes are necessary, the Suzaku GOF will be responsible for the fix and the re-release, contacting the original authors as needed. When significant changes are necessary, Suzaku GOF will always consult the original authors in advance.

  8. Tasks required for the Pipe-line processing should run in scripts. In the automated pipe-line processing system (section 6.6), series of data processing tasks are run as background jobs by scripts. Therefore, all the processing tasks including those which make use of GUI are required to run in scripts.

3.3 Suzaku Software Standards

3.3.1 Languages

Suzaku software will be mainly written in C. The use of C++ is allowed, but not encouraged. C++ will not be adopted throughout the project, but may be used within some small independent packages (e.g., ray-tracing program). Fortran77 is allowed, but Fortran90 shall not be used.

In the scripting tasks, use of system independent environments such as Perl or Tcl/Tk is recommended. Use of the shell languages (such as csh, bsh and tcsh) which do not run beside UNIX environment is forbidden.

3.3.2 Coding Rules and Compiler Requirements

Portable coding practices shall be adhered to, including the isolation of system dependencies. C programs will adhere to the ANSI C standard, while Fortran programs will follow the OGIP Fortran standards (Mukai 1993)3.1 for Fortran programming.

The system-independence test for C shall be that the code can be compiled by gcc on the several supported architectures (see section 3.3.3); similarly g77 will be used to test Fortran programs. The cfortran package shall be used to combine C and Fortran routines when necessary.

To write and read FITS files, cfitsio (in C) or fitsio (in Fortran) should be used. The obsolete fitsio C-wrappers, which were developed to call fortran fitsio routines from C-codes, should not be used.

3.3.3 Systems Supported

All the Suzaku software intended to distribute shall run on the most popular systems of Suzaku users. The systems are likely to include Sun/Solaris, DEC/Alpha, Linux (Redhat, Power PC), and Apple/Darwin (Mac-OS X).

3.3.4 Coordination and Version Control

The Software Coordination Group consisting of members from each hardware team, ISAS, and the GOF shall meet regularly (at least twice a year until and soon after the launch) to ensure software coordinations. In addition, the GOF shall have one person attached to each hardware team with responsibility to help coordinate software development. The software coordination group shall also be responsible for ensuring consistency of FITS keyword naming across teams.

The 1st Stage Software (section 6.3) is maintained by ISAS. GSFC keep master copies of all the software except the 1st Stage Software under a control system. This control system shall ensure that a given file is only edited by one person at a time and also that previous versions are archived and can be recovered. The practical way that Suzaku FTOOLS will be developed and maintained globally is explained in section 3.4.

3.3.5 Documentation

All software intended for distribution should be fully documented in English. Comments in the source codes should be written in English, but Japanese translation might be added for convenience and may not have to be stripped when distributed.

All subroutines/programs of general use shall contain a standard header. The GOF will provide a script to strip out these headers and make them available over the Web. The GOF will also provide template routines containing the standard header.

The FITS file format of Suzaku related files is fully explained in a separate document maintained by the Suzaku GOF.

3.4 Suzaku FTOOLS Global Development Scheme

Many scientists and programmers in the United States and Japan are involved in the Suzaku FTOOLS development. Also, Suzaku FTOOLS users are located not only in the two countries, but also in Europe and the rest of the world. Therefore, version control will be very important so that no different flavors of the same FTOOLS be developed and proliferated.

In the early stage of the mission, as understanding of the instruments deepens and new data analysis techniques are getting established, it will be necessary to update and release the Suzaku FTOOLS promptly. We should be ready for the release cycle of a few weeks or less.

To accommodate both requirements of the rigorous version control and prompt release, the following scheme, which is illustrated in figure 3.1, has been proposed and will be practiced for the Suzaku FTOOLS development, version control, and release.

Figure 3.1: Suzaku FTOOLS global development and version control scheme.

  1. At GSFC, the FTOOLS team maintains the FTOOLS ``Repository'', for which only the team members are granted the write permission. The FTOOLS team receives original source codes from the ``Contact'' groups (through ISAS when the Contact groups are in Japan; see 6 below), and put the codes in the Repository, after minimal programmatic changes if necessary. The codes in the Repository should be considered the genuine copy of the latest official FTOOLS.

  2. The entire FTOOLS directory tree is built weekly from the Repository. This FTOOLS is called ``Develop'' FTOOLS, and only available inside GSFC. The Develop FTOOLS are tested at GSFC, and the codes will be fixed if any problems are found, and put in the Repository again. Note that the Develop FTOOLS reflect updates of all the FTOOLS including Suzaku.

  3. From time to time, the entire FTOOLS package is released to public. This package is called the ``Release'' FTOOLS. Frequency of the release is typically once or twice a year.

  4. In order to keep up with short development cycle, whenever Suzaku FTOOLS in the Repository are updated, the FTOOLS team will build the Suzaku FTOOLS against the Release FTOOLS, and install the ``Suzaku add-on''. Interval of the Suzaku add-on build will be as short as one week (= Develop FTOOLS build cycle). The Release FTOOLS with the Suzaku add-on is the one Suzaku users will use for their data analysis. The Suzaku add-on package will be promptly released to Suzaku users, so that they can install it on their own Release FTOOLS.

  5. The Release FTOOLS with the Suzaku add-on will be mirrored daily to ISAS, and will be used for Suzaku data analysis at ISAS. Japanese Suzaku users outside of ISAS may obtain the original package from GSFC or mirrored one from ISAS.

  6. Instrument teams in Japan will test and modify the source codes in the Suzaku add-on package to reflect the latest calibration, and they will deliver the new codes to the Software Development Coordinator at ISAS. The Software Development Coordinator will make sure that the codes from different groups are consistent and can be built cleanly using gcc. After that, he or she will deliver the codes to the FTOOLS team at GSFC (go back to step 1).

  7. The Processing team at GSFC will obtain the Release FTOOLS with the Suzaku add-on, which will become the base of the pipe-line processing. The Processing FTOOLS, as well as the pipe-line processing scripts, will be mirrored to the ISAS processing center from GSFC, so that the data centers at GSFC and ISAS use the identical system to produce standard Suzaku data products.

  8. The processing software should be built-in the software packages HEAsoft, and the processing should be performed with the the latest release (ver. 6.0.3 in November 2005).

3.5 Suzaku Ftools Release Plan

  1. Ftools delivery should include .par files, .hlp files and unit test scripts following the templates provided by HEASARC.

  2. Freeze dates of the Suzaku ftools on the GSFC CVS are the last days of January, April, July and October. After the freeze, only bug fixes may be committed to the CVS.

  3. After the freeze dates, Suzaku ftools go through the multi-platform test at GSFC and ISAS, which takes maximum six weeks.

  4. After the multi-platform test, Suzaku ftools release will be around March 15, June 15, September 15, and December 15. These release dates may be slightly shifted being affected by situations of other ftools development/release, because it is desirable that Suzaku ftools release be synchronized with other ftools release as much as possible.

  5. After the release, new ftools, libraries or new functionalities may be committed to the GSFC CVS. Going back to 1 above, the next release cycle starts.

next up previous contents index
Next: 4. Suzaku Function Libraries Up: Suzaku Project Data Management Previous: 2. Observations Types   Contents   Index
Michael Arida 2007-09-29