HEASoftPy is a Python 3 package that gives access to the HEASoft
tools using python. It provides python wrappers that call the
HEASoft tools, allowing for easy integration into other python
code.
HEASoftPy also provides a framework that allows for pure-python
tools to be developed and integrated within the HEASoft system
(see Writing Python Tasks).
The IXPE tools are an example of mission tools developed in python.
Although HEASoftPy is written in pure python, it does not rewrite
the functions and tools already existing in HEASoft. A working
installation of HEASoft is therefore required.
After intallation (see Installation), HEASoftPy can
be used is several ways.
1- Importing the task methods:
import heasoftpy as hsp
hsp.ftlist(infile='input.fits', option='HC', outfile='STDOUT', ...)
2- Creating a HSPTask and invoking it:
import heasoftpy as hsp
ftlist = hsp.HSPTask('ftlist')
ftlist(infile='input.fits', option='HC', outfile='STDOUT', ...)
3- For tasks written in python (e.g. ixpecalcfov.py), the tools can be used as usual from the command line, similar to the standard HEASoft tools:
ixpecalcfov[.py] ra=... dec=...
The .py extension is generally optional.
Starting with version 1.4, the mission tools are no longer imported by default when importing heasoftpy. They need to be imported explicitly.
Using the old namespace call, will give a deprecation warning, with the suggested new use. The old way will removed in the future.
# heasoftpy version >= 1.4
from heasoftpy import nicer, ixpe
nicer.nicerl2(...)
ixpe.ixpeaspcorr(...)
# heasoftpy version < 1.4
import heasoftpy
heasoftpy.nicerl2(...)
heasoftpy.ixpeaspcorr(...)
Only core tools are available under the heasoftpy namespace.
Native HEASoft tasks have the same names in HEASoftPy. So a task like nicerclean
is called by heasoftpy.nicerclean, except for tasks that have the dash symbol - in the name,
which is replaced by an underscore _. For example, the task bat-burst-advocate is available
with heasoftpy.bat_burst_advocate, etc.
Passing parameters to a task can be done in several ways. For example:
import heasoftpy as hsp
hsp.ftlist(infile='input.fits', option='HC', outfile='STDOUT', ...)
# or
params = {
'infile': 'input.fits',
'option': 'HC',
'outfile': 'STDOUT',
...
}
hsp.ftlist(params)
# or
ftlist_task = hsp.HSPTask('ftlist')
ftlist_task(infile='input2.fits', option='HC', outfile='STDOUT', ...)
hsp.ftlist(ftlist_task)
# or
ftlist_task = hsp.HSPTask('ftlist')
ftlist_task.infile = 'input2.fits'
ftlist_task.option= 'HC'
ftlist_task.outfile = 'STDOUT'
... # other parameters
ftlist_task()
# or a combination of the above
Whenever a task in called, if any of the required parameters is missing,
the user is prompted to enter a value. If the user knows that the passed
parameters are enough to run the task (e.g. pipeline tools),
they can pass noprompt=True, to distable parameter prompt.
Note that creating a task object with ftlist_task = hsp.HSPTask('ftlist')
does not actually call the task, it just initializes it.
Only by doing ftlist_task(...) is the task called and parameters are queried if necessary.
HEASoftPy Control Parameters¶There are a few parameters that are common between all tasks:
verbose: This can take several values. In all cases, the text printed by the
task is captured, and returned in HSPResult.stdout/stderr. Additionally:0 (also False or no): Just return the text, no progress prining.1 (also True or yes): In addition to capturing and returning the text,
task text will printed into the screen as the task runs.2: Similar to 1, but also prints the text to a log file.20: In addition to capturing and returning the text, log it to a file,
but not to the screen.
In both cases of 2 and 20, the default log file name is {taskname}.log.
A logfile parameter can be passed to the task to override the file name.noprompt: Typically, HSPTask would check the input parameters and
queries any missing ones. Some tasks (e.g. pipelines) can run by using
default values. Setting noprompt=True, disables checking and querying
the parameters. Default is False.stderr: If True, make stderr separate from stdout. The default
is False, so stderr is written to stdout.If the heasoftpy task being called has an input parameter with a name verbose, noprompt
or logfile, then the above control parameters can be accessed by prepending them with py_,
so verbose becomes py_verbose etc. For example: the task batsurvey_catmux has a parameter
called logfile. If the you want to use both parameters, the call would be:
heasoftpy.batsurvey_catmux(..., logfile='task.log', py_logfile='pytask.log')
with 'task.log' logging the task activity, and 'pytask.log' logging the python wrapper activity.
The help for the tasks can be accessed in the standard python way, e.g. in ipython:
hsp.ftlist?
which will print something like the following, indicating the required parameters:
Automatically generated function for Heasoft task ftlist.
Parameters
----------
infile (Req) : Input file name (default: )
option (Req) : Print options: H C K I T (default: HC)
outfile : Optional output file (default: -)
clobber : Overwrite existing output file? (default: no)
include : Include keywords list (default: *)
exclude : Exclude keywords list (default: )
section : Image section to print, eg, '2:8,1:10' (default: *)
columns : Table columns to print (default: *)
rows : Table rows or ranges to print, eg, '2,6-8' (default: -)
vector : Vector range to print, eg, '1-5' (default: -)
separator : Column separator string (default: )
rownum : Print row number? (default: yes)
colheader : Print column header? (default: yes)
mode : Mode (default: ql)
...Scrolling down further, the help message will print the standard HEASoft help text from fhelp.
--------------------------------------------------
fhelp-generated text
--------------------------------------------------
NAME
ftlist - List the contents of the input file.
USAGE
ftlist infile[ext][filters] option
DESCRIPTION
'ftlist' displays the contents of the input file. Depending on the
value of the 'option' parameter, this task can be used to list any
combination of the following:
* a one line summary of the contents of each HDU (option = H)
* the names, units, and ranges of each table column (option = C)
* all or a selected subset of the header keywords (option = K)
* the pixel values in a subset of an image (option = I)
* the values in selected rows and columns of a table (option = T)
If a specific HDU name or number is given as part of the input file
name then only that HDU will be listed. If only the root name of the
file is specified, then every HDU in the file will be listed.heasoftpy is installed automatically with HEASoft version 6.30 or newer. Make sure you have python version >3.7, and the python dependencies installed (see step 1- below) before installing HEASoft. If you have an older version of HEASoft, the following steps can be used to install or update heasoftpy manually in an existing HEASoft installation.
Assuming you have HEASoft initialized and the environment variable $HEADAS is defined:
heasoftpy is automatically installed with any new installation of HEASoft after version 6.30
Starting with HEASoft 6.32, you can update heasoftpy by running hpyupdate. First, make sure that HEASoft is initialized and that the utilites wget and tar are available in your system PATH, then simply run
hpyupdate
1- Download the latest version of heasoftpy
2- Untar the file:
tar -xvf heasoftpy-latest.tar
cd heasoftpy
3- Install python dependencies:
pip install -r requirements.txt
4- Generate the wrappers and install the package:
To install heasoftpy into the $HEADAS tree:
bash local-build.pyThis will generate the python wrappers under build/heasoftpy. Errors will be printed in the pip.log file.
5- Move the heasoftpy folder from build/heasoftpy to $HEADAS/lib/python.
rm -r $HEADAS/lib/python/heasoftpy
mv build/heasoftpy $HEADAS/lib/python
6- Move the parameter files, executables and help files (if any) to their location in the $HEADAS tree:
mv build/bin/* $HEADAS/bin
mv build/syspfiles/* $HEADAS/syspfiles
mv build/help/* $HEADAS/help
HEASoft tree¶heasoftpy does not have to be inside the HEASoft tree as long as HEASoft is initialized. Inside the newly download heasoftpy folder, do: pip install .
There may be times when a heasoftpy function needs
to be created for an installed HEASoft task
(for example, if a new HEASoft component is added to
HEASoft after heasoftpy was initially installed).
heasoftpy wrappers can be created (or re-created) using
the heasoftpy.utils.generate_py_codefunction. For example
to generate a heasoftpy function wrapper for the NICERDAS
tool nibackgen3c50, do the following in python:
from heasoftpy.utils import generate_py_code
tasks = ['nibackgen3c50']
generate_py_code(tasks=task)
You can also start a fresh heasoftpy installation as detailed in the Installation section.
As discussed in the PARALLEL BATCH PROCESSING, most heasoft (and hence heasoftpy) tasks use parameter files whose location is managed by the PFILES environment variable. Parallel calls to the same task will likely end up using the same parameter file and may cause unintended parameter changes. Users may use the suggestions in the link above, however when using python scripting, it may be convenient to use the context manager method heasoftpy.utils.local_pfiles_context. Including all parallel tasks inside a with statement, will ensure that temporary parameter files are used. The following gives an example:
from multiprocessing import Pool
import heasoftpy as hsp
from heasoftpy import nicer
def worker(args):
"""Run individual tasks"""
with hsp.utils.local_pfiles_context():
# call the tasks of interest
out = nicer.nicerl2(...)
# other tasks
# ...
return output
nproc = 5
with Pool(nproc) as p:
print(p.map(worker, [1, 2, 3, 4, 5]))
The core of HEASoftPy is the class HSPTask, which handles the parameter reading and setting (from the .par file).
It was written in a way that makes it easy for writing new codes that can be easily integrated within HEASoft. All that is needed, in addition to creating a .par file, is to create a subclass of HSPTask and implement a method exec_task that does the task function. An example is given in template. The following is short snippet:
import heasoftpy as hsp
class SampleTask(hsp.HSPTask):
"""New Task"""
name = 'sampletask'
def exec_task(self):
# model parameters as a dict
usr_params = self.params
# write your task code here #
# ...
# ...
# ------------------------- #
# finally return a HSPResult
return hsp.HSPResult(0, out, None, usr_params)
The following notebooks contain some tutorials and usage examples.
Getting Started: A quick walkthrough guide of the main features of the HEASoftPy package, and ways of calling and obtaining help for the tasks.
NuSTAR Data Analysis Example: This is a walkthough example of analyzing NuSTAR observation 60001110002 of the AGN in center of SWIFT J2127.4+5654 using HEASoftPy. It includes examples of calling the calibration pipeline, and then extracting the source light curve.
NICER Data Analysis Example: This is a walkthough example of analyzing NICER data using HEASoftPy and PyXspec.