National Aeronautics and Space Administration

The agreement between NASA and the Italian Space Agency (ASI) calls for the exchange of archived public BeppoSAX data at the Science Data Center (SDC) in Rome, Italy for selected public data sets archived at the HEASARC. To facilitate this agreement, it was initially decided that the medium for the exchange would be CDROM, with the possibility of using digital video disks (DVD) in the future when the technology had developed further.

In the fall of 1997, the HEASARC procured a JVC 600 CDROM Library with 8X read only CD drives managed by HyperROM Software from Tracer Technologies, Inc. and a CD-Studio from Young Minds, Inc. that included a Kodak PCD Writer 600, a Kodak Disc Transporter, Rimage CD printer and MakeDisc software. The CD-Studio hardware can produce up to 75 full capacity identical CDs in approximately 16 hours as the Writer 600 is a 6x speed CD recorder. Since the HEASARC will be making copies of selected files in the archive that are stored on magneto optical jukeboxes, the realized throughput is expected to be much lower than this value.

The CD-Studio is a special purpose PC with firmware that is designed to stage data as a CD image to an internal disk in the unit. It connects to a SCSI port of a PC or workstation and appears to the system as an Exabyte tape device or a CDROM device. A CD image is staged to the unit by using the MakeDisc software which refers to the tape device. After a CD image has been staged to the unit, it can be mounted as a CD device on the workstation. This allows examination of the integrity of the data before it is committed to the CD media. The Writer 600 and the Disc Transporter are connected to the SCSI port on the CD-Studio. The Rimage CD printer is connected to the parallel port of the CD-Studio. The serial port of the CD-Studio is connected to the serial port of the workstation to monitor the operational status.

The CDROM Library was delivered and installed in November 1997 on a DEC Personal Workstation 433au. It had been delivered more than 6 weeks late and arrived with some superficial cosmetic damage, but was found to be in perfect working order. The installation of the HyperROM software and the CDROM Library executed without incident.

The largest group of customers for the CD-Studio hardware has been in the financial industries. Typically, CDs are created on a daily basis to distribute to financial analysts and bankers. In this application, a GUI interface is used to burn 50 copies of the same CD. Clearly, the HEASARC's application would not require massive copies of one CD image. It is much more cost effective to send a master CD image to a contractor if a new CD publication is the goal. The HEASARC needs are to make copies of selected mission files within the archive for distribution to sites overseas and to quickly write requested data to medium that can be delivered to an investigator. Such an investigator might have a poor network connection to the HEASARC archive or their requested data set may be so large that network downloads would be a futile waste of bandwidth and time.

There are three steps that the HEASARC used to automate this process:

• Select and stage ALL the data for the desired CD set. It is best to create a basic directory structure for each CD that includes symbolic links to the actual data, otherwise the data is moved twice. The HEASARC ROSAT production run for SDC used a database dump for all of the public observations available in the archive as an input file. An HTML file was created with links back to the HEASARC archive to the directory location of each sequence that was contained on the CD. This facilitates access to products associated with the sequence that were not selected for the CD. The HTML file also contains some basic database information about the seqeunce. A pipe delimited database dump with the CD volume name is also created to aid ingest into a local SDC database. This stage also had to calculate precisely which sequences would fit on a 650 MB CDROM. The first sequence to go over the 650MB is moved to the next CD volume. The first step of data integrity is also taken here as each file selected for the CD image is checksummed and logged. For ROSAT, perl scripts were written to accomplish this task.

• Run the Makedisc against the staged directory structure. Data will be moved to the internal hard disk in the CD-Studio to create the CD image that will be burned on a CDROM. A C shell script that was used during the test phase was modifed slightly to handle the HEASARC ROSAT production run. After the CDROM image has been staged, the CD-Studio is mounted as a CDROM image. All of the data is checksummed and compared to the log file to insure data integrity. A cut command is then issued for each CD that is required. This will burn the data to a blank CDROM. Each cut command will require approximately 15 minutes; 12 minutes to burn the CD and another 3-4 minutes to move the media to the CD label printer, then to the output bin and then time to load the next CD to be burned in the recorder.

• Remove the CDs and put each set in the CDROM library to check the integrity by checksumming all files and comparing them to the checksums recorded in the logs. Additional perl scripts were written to handle this integrity check.

The HEASARC CDROM production facility is still in its infancy. Missions such as RXTE and future missions may have problems getting a single observation within the 650MB limit of a CDROM. Many of the scripting software and WEB interfaces still need to be developed, but should be portable to a DVD production facility when that technology becomes available.

Proceed to the next article

Return to the previous article

Select another article