HEASARC Staff Scientist Position - Applications are now being accepted for a Staff Scientist with significant experience and interest in the technical aspects of astrophysics research, to work in the High Energy Astrophysics Science Archive Research Center (HEASARC) at NASA Goddard Space Flight Center (GSFC) in Greenbelt, MD. Refer to the AAS Job register for full details.

Search:

[Advanced Search]



HEASoft & Docker


What Is Docker?

To learn all about Docker, please visit the Docker website, and to learn more about containers and images, please see Docker's explanation or one from AWS.

For our purposes, Docker enables packaging HEASoft as a standalone executable container that includes everything needed to run HEASoft: the code, the system tools and libraries, and the settings.

The primary benefit for HEASoft users is that it allows for an easier installation process and use of our software on any operating system supported by Docker.

Installation

Currently, we provide two approaches for installing HEASoft with Docker, both of which require that you have installed the Docker Engine (i.e. the Docker application or executable) and have it running. Also note that older versions of Docker may not be compatible with some of the syntax used below, so it's a good idea to make sure your Docker application is up-to-date.

The docker commands below require that in addition to having the Docker application running, you should have the command-line docker executable available in your Unix PATH. It typically installs as /usr/local/bin/docker, so either add /usr/local/bin to your PATH (and run "rehash"), or in the commands below replace docker with /usr/local/bin/docker.

Our two approaches are the following:

  • A Dockerfile that retrieves a complete HEASoft source code tar file and then creates an Ubuntu Linux container into which all of HEASoft is installed. To use this approach, download this tar file.

    Unpack it anywhere you choose, and then run make: 

       $ tar xf heasoft-6.34docker0.99.tar
   $ make
    Or, if necessary: 
       $ sudo make
  • For users who do not need all of HEASoft, the standard "a la carte" tar files now unpack with a Docker folder from which one can install a container that includes only the packages that were selected for inclusion.

    Once you have downloaded and unpacked your HEASoft source code, make sure that you have a GNU tar executable - for example gtar, gnutar, or perhaps just tar - in your PATH or else assigned to the TAR environment variable, and then run make inside the Docker folder: 

       $ cd heasoft-6.34/Docker
   $ make
    Or, if necessary: 
       $ sudo make
For either approach, note that you may need to run sudo make instead of just make if your docker executable requires root permission to run. In either case, make will launch the appropriate set of commands to create and set up a new Ubuntu Linux container, and then configure, build, and install HEASoft inside the container. This process may take a while depending on your hardware and which approach you selected.

At first you will see some screen output from the various build commands, and you may eventually see a message that a log limit of 1MiB has been reached, but this is normal and does not imply that an error has occurred.

Using HEASoft in Docker

When make has finished, you may open the container in Docker, using for example: 
 $ docker run --rm -it heasoft:v6.34 tcsh
This command should provide you with an interactive docker prompt for the new container in a T C-shell, and place you in the /home/heasoft directory. Alternatively you may want to start up docker adding a volume from your host machine to the container, for example: 
 $ docker run --rm -it -v /Users/user1/data:/mydata heasoft:v6.34 tcsh
This should mount the host directory "/Users/user1/data" inside the container as "/mydata". More information about this and other Docker options can be found at the Docker website.

At this point you might run fversion to confirm that the installation is working, 

 $ fversion
or fhelp to get more information about a particular task or package, for example: 
 $ fhelp heatools
Note that the HEASARC CALDB has been set up in your environment using the remote setting: 
  CALDB=https://heasarc.gsfc.nasa.gov/FTP/caldb
  CALDBCONFIG=/opt/heasoft/caldb/caldb.config
  CALDBALIAS=/opt/heasoft/caldb/alias_config.fits
Xspec users may add local models as desired in the standard way.

Using GUI Tasks or Plotting

To use GUI tasks (FV, etc.) or display plot windows from e.g. Xspec, modifications can be made to your X server connections.

For example, on Mac hosts running XQuartz, open the XQuartz Preferences menu, and in the Security tab check the "Allow connections from network clients" box. Then, quit XQuartz and restart it. From a new X terminal run these commands to start up the container with an X connection: 

 $ /opt/X11/bin/xhost + 127.0.0.1
 $ docker run --rm -it -e DISPLAY=host.docker.internal:0 heasoft:v6.34 tcsh
For Linux hosts with X11 installed, start the container with: 
 $ docker run --rm -it -e DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:ro
heasoft:v6.34 tcsh
Then, in a separate terminal: 
 $ docker ps -l -q
Use the output from this command in the next one: 
 $ xhost +local:<output from previous command>
The two steps above could alternatively be combined as: 
 $ xhost +local:`docker ps -l -q`

Python, and editing the Dockerfiles

PyXspec and the HEASP Python interface are both available, accessible by running python (which is python3 by default) and importing the relevant package.

In the future, other Python packages may be installed by default as part of our Dockerfiles, but for now users are of course free to edit these files to suit their purposes. For example, under the HEASoft prerequisites section, an Ubuntu 'apt-get' command installs a number of packages, and while you should not remove any of the existing prerequisites, the list may easily be added to, with (for example) some standard Python packages and a text editor (vim): 

        ...
        ...
        python3-astropy \
        python3-astroquery \
        python3-matplotlib \
        python3-pandas \
        python3-pip \
        python3-pyvo \
        python3-scipy \
        ipython3 \
        vim \
        ...
        ...
Caution should be taken to ensure that proper Dockerfile syntax is preserved, particularly with the trailing back slashes "\" in the context mentioned above.

An example Dockerfile that adds the extra packages listed above is available here. (Note that this example is for the complete HEASoft approach only, and would need to be renamed as "Dockerfile" to actually be used.)

Preserving Changes and Cleaning Up

If you wish to preserve changes made to an existing image (or files created in the image that aren't on a shared volume), use the "docker commit" option. For example:

Before exiting the container you want to commit a change to, run the following in a separate terminal on your host machine: 

 $ docker ps -a
 CONTAINER ID   IMAGE             COMMAND   CREATED          STATUS          PORTS     NAMES
 3faf5af4ac17   heasoft:v6.34     "tcsh"    8 days ago       Up 33 seconds             angry_bose
Using the CONTAINER_ID, create a new image with a new name: 
 $ docker commit 3faf5af4ac17 heasoft_new
Now "docker images" shows the new image: 
 $ docker images
 REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
 heasoft_new  latest    d98eeb6e562d   44 seconds ago   12.5GB
 heasoft      v6.34     274fc5d180ea   8 days ago       12.5GB
To clean up, exit the running image and use the "IMAGE ID" to remove the older image: 
 $ docker rmi 274fc5d180ea
Similarly, if you find it necessary to re-make an image for any reason, the old image will remain in the system (untagged) until you clean it up: 
 $ docker images clean
To free up disk space, you may also find it useful to occasionally "prune" various levels of the Docker system. For reference, please see this section about pruning at Docker.com.

As always, the best reference for any Docker command is the documentation at Docker.com.

Help

Please send any comments or questions to the FTOOLS Help Desk.

Last modified Thursday, 22-Aug-2024 15:36:59 EDT

HEASoft / FTOOLS Help Desk

If FTOOLS has been useful in your research, please reference this site (https://heasarc.gsfc.nasa.gov/ftools) and use the ASCL reference for HEASoft [ascl:1408.004] or the ASCL reference for the original FTOOLs paper [ascl:9912.002]:

Blackburn, J. K. 1995, in ASP Conf. Ser., Vol. 77, Astronomical Data Analysis Software and Systems IV, ed. R. A. Shaw, H. E. Payne, and J. J. E. Hayes (San Francisco: ASP), 367.

Web page maintained by: Bryan K. Irby

HEASARC Home | Observatories | Archive | Calibration | Software | Tools | Students/Teachers/Public
Last modified: Thursday, 22-Aug-2024 15:36:59 EDT

HEASARC Staff Scientist Position - Applications are now being accepted for a Staff Scientist with significant experience and interest in the technical aspects of astrophysics research, to work in the High Energy Astrophysics Science Archive Research Center (HEASARC) at NASA Goddard Space Flight Center (GSFC) in Greenbelt, MD. Refer to the AAS Job register for full details.