Analyzing NICER Data on Sciserver
SciServer is a data analysis platform hosted at Johns Hopkins University. It consists of a data hosting server which includes the entire HEASARC archive, along with an analysis environment which includes NICERDAS, HEASoft and other analysis systems. SciServer allows users the ability to access and analyze any (or all) public NICER data without having to download data or install/maintain analysis software locally. This thread shows how to access SciServer.
Read this thread if you want to: Analyze NICER data using the SciServer science platform.
Last update: 2023-07-12
It is often useful to access and analyze large amounts of NICER data. Large datasets include observations of a single bright source, or individual observations of multiple sources. It's also useful to be able to analyze NICER data without having to worry that your software environment has the latest software installed. It can be time consuming to download large amounts of data from the HEASARC archive to a local disk, and to install and maintain analysis software and calibrations.The SciServer science platform provides a user with direct, on-line access to all public NICER data along with pre-built analysis software including HEASoft and NICERDAS. SciServer enables analysis of large amounts of NICER data (along with comparison with data from other missions archived at the HEASARC) in a simple way.
This science thread describes how to create a SciServer account and how to access the HEASARC SciServer environment so users can access and analyze NICER observations.
Accessing the HEASARC environment on Sciserver
Creating a SciServer AccountUsers should first create their SciServer account. Once the account is created the user can log in at the Sciserver login page.
Creating a Virtual ContainerOnce the user has logged in, on the SciServer Dashboard page, the user should create a virtual container which includes the HEASARC environment (data archive and software). To create the virtual container:
Using the Virtual Container
To start the container, which will start the virtual environment with access to the HEASARC data archive and the HEASoft analysis software (including NICERDAS), click the name of the container you created in the container list on the "Compute" page. The container will start a jupyterlab server, and open a jupyterlab notebook.
You can analyze data entirely in a Jupyterlab notebook, or on the terminal command line (or some combination of notebook and command line):
NICER data On SciServerThe HEASARC data archive is mounted on SciServer as
NICER data is stored in subdirectories under
where the subdirectories are organized by date of observation. NICER data is stored by NICER OBSID under these directories, so (for example) the data directory for NICER OBSID 3050450119 is /FTP/nicer/data/obs/2020_07/3050450119 on SciServer.
User DirectoriesSciServer users with a username = <username> have access to the following user directories:
/home/idies/workspace/Storage/<username>/persistent. This volume is for long-term storage but has a limited storage capacity. It is best used to store jupyterlab notebooks, other analysis scripts or other relatively small files which are meant to be available "permanently"; and
/home/idies/workspace/Temporary/<username> for temporary storage of large amounts of data
It is recommended that users analysis scripts (and anything else they want to persist in newly-created containers) in the "persistent" directory.
Large amounts of data to be processed (for example, large numbers of NICER datasets to reprocess with nicerl2) can either be copied from the appropriate subdirectories of /home/idies/workspace/headata/FTP/nicer/data/obs/ to /home/idies/workspace/Temporary/<username>, or processed in place with the output directed to /home/idies/workspace/Temporary/<username>
Cookbooks & a NICER Tutorial Notebook
There are a number of jupyterlab notebooks available in subdirectories of /home/idies/workspace/headata/software/cookbooks. These notebooks provide walk throughs in python of useful analysis tasks, such as analyzing data with heasoftpy, accessing HEASARC data using VO protocols, and other useful tasks.
We suggest copying the cookbooks to your persistent area so you can modify them if desired and save the modifications. To do this, in a terminal window
The NICER tutorial is heasoftpy_notebooks/nicer-example.ipynb of the cookbooks directory. This notebook shows you how to
More information about the HEASARC environment on SciServer is available at https://heasarc.gsfc.nasa.gov/docs/sciserver/.
Information on using heasoftpy is available at https://heasarc.gsfc.nasa.gov/lheasoft/heasoftpy.