Next: 4. Suzaku Function Libraries
Up: Suzaku Project Data Management
Previous: 2. Observations Types
3. Software Principles
In this chapter, we present the Suzaku software principles and agreements,
which all the software developers need to follow throughout the Suzaku project.
Suzaku data analysis system should share the same design principles with all the
other projects conducted under OGIP. These design principles may be summarized as follows:
- Standard and portable data format -- FITS (Flexible Image Transport System) format is adopted
for all the binary files. System dependent binary files will never be used.
Moreover, the existing OGIP conventions should be followed wherever possible, and new conventions
should be submitted to HEASARC FITS Working Group to check for consistencies with other missions.
Use of ASCII format is allowed for small files.
- Universal and unique software -- There should not be multiple channels of the
data analysis. Software releases are controlled, and the same routines used for
the instrumental calibrations by hardware teams are used for the scientific data analysis by Guest Observers.
- Designed for multimission analysis --
Existing software infrastructures will be utilized as much as possible.
Users will be able to analyze Suzaku data with standard high-level
X-ray data analysis packages such as XSPEC, XIMAGE, XRONOS, etc.
- Easy to install and use --
The software will be easy to install and use, and extensive help, support and documentation will be provided.
Suzaku specific software for low-level tasks are distributed in the standard FTOOLS
package, providing user friendly interface on most standard platforms
- Free and public software --
Users will not have to purchase any commercial software packages (such as IDL), and all the source codes
will be open and easily available at free of charge.
Users will not have to worry about license issues, and software
authors shall not claim any privileges or credits.
Users may modify and distribute Suzaku software freely on their responsibility.
In addition to the general design principles above,
Suzaku GOF and ISAS propose the following design principles for the
Suzaku software/data processing system. They reflect the experiences
from the ASCA mission.
- The raw telemetry will be converted to FITS format before distribution.
There is only one set of software (mk1stfits; section 6.3.1)
to access and interpret the raw telemetry data
and to convert them to the FITS format (First FITS Files; section 6.3).
Mk1stfits, as well as other processing
software, must be fully tested and ready before the launch of the satellite.
- All the calibration and data processing should start from the First FITS Files.
To that end, the First FITS Files should reflect the original
structure of the raw telemetry as much as possible.
- All the scientific analysis starts from the standard calibrated FITS files.
The First FITS Files are further processed by the standard software
with instrumental calibration information,
and the Calibrated FITS Files (section 6.3) are produced.
Scientific outputs are produced always from the official Calibrated FITS Files, and
there should not be other routes for scientific analysis.
- The same processing system to calibrate the First FITS files should run at GSFC and ISAS.
Thereby, US and Japanese Suzaku Observers shall receive the identical
Calibrated FITS Files.
- Important calibration tools/software should be made promptly available to GOs.
At any given time, there shall be always a single version of the official instrument calibration files and software controlled by the
Suzaku GOF and instrument teams.
This ensures that the Suzaku Observers can apply the latest calibration information to the observing data.
- Suzaku software will be written by the Suzaku software and hardware teams at
GSFC, ISAS and other institutions in Japan.
Tasks which require deep understanding of
the Suzaku instruments, spacecraft and telemetry formats will be mainly written by the members of
the hardware teams and ISAS. On the other hand, higher level tasks,
in which user-friendliness, standardization and conformity with other
high energy missions should have a high priority, will be mainly written by
the software team at GSFC.
- All the software for public release will be delivered to Suzaku GOF before the release.
will ensure that the software follow the rules presented in this chapter, and will
package them in a form which is suitable for general release.
Suzaku GOF will be responsible for releasing and maintaining the packages.
When modifications or bug fixes are necessary, the Suzaku GOF will be
responsible for the fix and the re-release, contacting the original authors as needed.
When significant changes are necessary,
Suzaku GOF will always consult the original authors in advance.
- Tasks required for the Pipe-line processing should run in scripts.
In the automated pipe-line processing system (section 6.6),
series of data processing tasks are run as background jobs by scripts.
Therefore, all the processing tasks including those which make use of GUI
are required to run in scripts.
Suzaku software will be mainly written in C. The use of C++ is allowed, but not encouraged.
C++ will not be adopted throughout the project,
but may be used within some small independent packages
(e.g., ray-tracing program).
Fortran77 is allowed, but Fortran90 shall not be used.
In the scripting tasks, use of system independent environments such as Perl or Tcl/Tk is recommended.
Use of the shell languages (such as csh, bsh and tcsh)
which do not run beside UNIX environment
Portable coding practices shall be adhered to, including the isolation of system
dependencies. C programs will adhere to the ANSI C standard, while Fortran
programs will follow
the OGIP Fortran standards (Mukai 1993)3.1
for Fortran programming.
3.3.2 Coding Rules and Compiler Requirements
The system-independence test for C shall be that the code can be
compiled by gcc on the several supported architectures (see section 3.3.3);
similarly g77 will be used to test Fortran programs. The cfortran package shall be used
to combine C and Fortran routines when necessary.
To write and read FITS files, cfitsio (in C) or fitsio (in Fortran) should be used.
The obsolete fitsio C-wrappers, which were developed to call fortran
fitsio routines from C-codes, should not be used.
3.3.3 Systems Supported
All the Suzaku software intended to distribute shall run on the most popular systems of
The systems are likely to include Sun/Solaris, DEC/Alpha, Linux (Redhat, Power PC),
and Apple/Darwin (Mac-OS X).
3.3.4 Coordination and Version Control
The Software Coordination Group consisting of members from each hardware team, ISAS, and the GOF
shall meet regularly (at least twice a year until and soon after the launch)
to ensure software coordinations.
In addition, the GOF shall have one person attached to each hardware team
with responsibility to help coordinate software development. The software coordination group shall
also be responsible for ensuring consistency of FITS keyword naming across teams.
The 1st Stage Software (section 6.3)
is maintained by ISAS.
GSFC keep master copies of all the software except the
1st Stage Software under a control system. This control system shall ensure that a given file is only edited
by one person at a time and also that previous versions are archived and can be recovered.
The practical way that Suzaku FTOOLS will be developed and maintained globally is
explained in section 3.4.
All software intended for distribution should be fully documented in English.
Comments in the source codes should be written in English, but Japanese translation might be added for convenience
and may not have to be stripped when distributed.
All subroutines/programs of general use shall contain a standard header. The GOF will provide a
script to strip out these headers and make them available over the Web. The GOF will also provide
template routines containing the standard header.
The FITS file format of Suzaku related files
is fully explained in a separate document maintained
Many scientists and programmers in the United States and Japan
are involved in the Suzaku FTOOLS development. Also, Suzaku FTOOLS users
are located not only in the two countries, but also in Europe and the rest
of the world.
Therefore, version control will be very important so that no different flavors
of the same FTOOLS be developed and proliferated.
In the early stage of the mission, as understanding of the instruments deepens
and new data analysis techniques are getting established,
it will be necessary to update and release the Suzaku FTOOLS promptly.
We should be ready for the release cycle of a few weeks or less.
To accommodate both requirements of the rigorous version control and prompt release,
the following scheme, which is illustrated in figure 3.1,
has been proposed and will be practiced
for the Suzaku FTOOLS development, version control, and release.
Suzaku FTOOLS global development and version control scheme.
- At GSFC, the FTOOLS team maintains the FTOOLS ``Repository'', for which
only the team members are granted the write permission. The FTOOLS team
receives original source codes from the ``Contact'' groups
(through ISAS when the Contact groups are in Japan;
see 6 below), and put the codes in the Repository, after
minimal programmatic changes if necessary.
The codes in the Repository
should be considered the genuine copy of the latest official FTOOLS.
- The entire FTOOLS directory tree is
built weekly from the Repository. This FTOOLS is called ``Develop'' FTOOLS, and
only available inside GSFC. The Develop FTOOLS are tested at GSFC,
and the codes will be fixed if any problems are found, and put in the
Repository again. Note that the Develop FTOOLS reflect updates of all
the FTOOLS including Suzaku.
- From time to time, the entire FTOOLS package is released to public.
This package is called the ``Release'' FTOOLS. Frequency of the release is
typically once or twice a year.
- In order to keep up with short development cycle,
whenever Suzaku FTOOLS in the Repository are updated,
the FTOOLS team will build the Suzaku FTOOLS against
the Release FTOOLS, and install the ``Suzaku add-on''.
Interval of the Suzaku add-on build will be as short as one week (= Develop FTOOLS
The Release FTOOLS with the Suzaku add-on is the
one Suzaku users will use for their data analysis.
The Suzaku add-on package will be promptly released to Suzaku users, so that
they can install it on their own Release FTOOLS.
- The Release FTOOLS with the Suzaku add-on will be mirrored
daily to ISAS, and will be used for Suzaku data analysis at ISAS.
Japanese Suzaku users outside of ISAS may obtain the original package
from GSFC or mirrored one from ISAS.
- Instrument teams in Japan will test and
modify the source codes in the Suzaku add-on package to reflect the latest calibration,
and they will deliver the new codes to the Software Development Coordinator
at ISAS. The Software Development Coordinator will make sure that the codes
from different groups are consistent and can be built
cleanly using gcc. After that, he or she will deliver
the codes to the FTOOLS team at GSFC (go back to step 1).
- The Processing team at GSFC will obtain the
Release FTOOLS with the Suzaku add-on, which will become the
base of the pipe-line processing.
The Processing FTOOLS, as well as the pipe-line processing
scripts, will be mirrored to the ISAS processing center from GSFC,
so that the data centers at GSFC and ISAS use
the identical system to produce standard Suzaku data products.
- The processing software should be built-in the software packages HEAsoft,
and the processing should be performed with the the latest release
(ver. 6.0.3 in November 2005).
- Ftools delivery should include .par files, .hlp files and unit
test scripts following the templates provided by HEASARC.
- Freeze dates of the Suzaku ftools on the
GSFC CVS are the last days of January, April, July and October.
After the freeze, only bug fixes may be committed to the CVS.
- After the freeze dates, Suzaku ftools go through the multi-platform test
at GSFC and ISAS, which takes maximum six weeks.
- After the multi-platform test,
Suzaku ftools release will be around March 15, June 15, September 15, and
December 15. These release dates may be slightly shifted being affected
by situations of other ftools development/release, because
it is desirable that
Suzaku ftools release be synchronized with other ftools release
as much as possible.
- After the release, new ftools, libraries or new functionalities
may be committed to the GSFC CVS. Going back to 1 above, the next release
Next: 4. Suzaku Function Libraries
Up: Suzaku Project Data Management
Previous: 2. Observations Types