********************************************************************************
RELEASE NOTES FOR HEASOFT 6.31
November 15, 2022
********************************************************************************
The HEASoft 6.31 release is driven by new support for the Apple
Silicon (arm64) architecture and updates to mission-specific data
analysis software (IXPE, NICER, NuSTAR, Swift), but as always
includes other enhancements and fixes.
Configuration-related changes throughout the package mean that users
who already have HEASoft installed will need to completely replace
their software installation; there is no clean way to simply overlay
this new release or to update an existing installation.
*** IMPORTANT NOTE FOR MAC USERS: ***
HEASoft 6.31 restores support for the Apple clang/clang++ compilers,
and introduces support for builds of the software on the Silicon arm64
architecture. As a result, please note the following:
* The default build architecture for HEASoft on Silicon is now
arm64. That is, the configuration is expecting to find native
arm64 compilers, NOT the Intel (x86_64) compilers you may have
installed with Rosetta and used for previous HEASoft releases.
If desired, it should be possible to still build HEASoft on
Siicon with the Rosetta Intel compilers using the configure option
"--enable-mac_intel_build", but we do not recommend this approach.
* The preferred C compilers for any Mac architecture (Silicon or
Intel) are now clang and clang++, which should by default operate
in the correct mode for the native architecture, i.e. arm64 on
Silicon and x86_64 on Intel. Similarly, the preferred Perl
interpreter is /usr/bin/perl which should again match the native
architecture.
As always, a Fortran compiler is necessary, and should be chosen
to match the architecture of the C compilers. For example, when
on Silicon using CC=/usr/bin/clang (arm64), you should choose a
Fortran compiler which is also arm64. The Homebrew and MacPorts
package managers (for example) provide Fortran compilers which
by default - i.e. when NOT using Rosetta - should match the native
architecture.
Lastly, note that your choice of Python should also match the
architecture of your other build choices. For example,
Anaconda offers Intel or Silicon versions of their software.
********************************************************************************
CFITSIO
********************************************************************************
Version 4.2.0 (Nov. 2022)
- This release includes patches to security vulnerabilities. We
strongly encourage this upgrade, particularly for those running
CFITSIO in web accessible applications.
- Fix to fits_read_key function, which was failing to properly read
keywords declared type TUINT on compilers where sizeof(int) =
sizeof(long).
- Added new functions fits_read_cols and fits_write_cols to efficiently
read or write multiple columns in a single function call
- Added new function fits_copy_selrows to copy only selected rows, such
as the selected rows returned by fits_find_rows
- Added new calculator functions ERF(X), ERFC(X) and GAMMA(X), which are
mathematical special functions of the same name
- Added new calculator function GTIFIND() which reports which GTI row
brackets a given time sample
- Added new calculator functions which operate upon vector table
values NAXIS(V), NAXES(V,n), ELEMENTNUM(V) and AXISELEM(V,n),
and ARRAY(X,d) which promotes scalar X to a vector or array
with given dimensions.
- The CFITSIO histogramming code now handles binning by any
arbitrary calculator expression rather than just a column name.
Both the binned columns as well as the optional weights may be
calculator expressions, enclosed in parentheses.
- Binning of vector columns or expressions is now supported, as long
as all binned inputs (as well as the optional weighting) have the
same vector dimensions. Binning of variable-length columns remains
unsupported.
- A bug that caused histogram weights from columns that are null values
to be tallied along with non-null values has been fixed.
- The CFITSIO calculator and histogramming functionality is now
fully reentrant and does not require multithreading interlocks.
- A long-standing segmentation fault bug in the histogramming code
related to binning any value using the "reciprocal" /XXXX notation
has been fixed.
- Added mutex locks for thread safety in ftgiou and ftfiou.
- Added several Fortran wrappers to support image read/write when
'fpixel' and 'nelements' are 8-byte integers.
- Fixed bug which was adding spaces to some cases of long string key
value output.
Perl interface (Astro::FITS::CFITSIO):
- Added functions fits_make_histd(), fits_make_histde(), fits_copy_selrows()
********************************************************************************
ATTITUDE
********************************************************************************
- atFunctions library:
* v3.5: Added atSXSBrazil(): New routine adds support for the
XRISM Resolve instrument with a particular definition of the
South Atlantic Anomaly (SAA)
* v3.4: Bug fix in atPol60ToVect()
********************************************************************************
HEASP
********************************************************************************
Important change:
heasp.h no longer includes using namespace std and CCfits and all of
heasp has been placed in its own namespace called heasp. This should
reduce the possibility of name conflicts between heasp and other
libraries. However, it does mean that any program using heasp will
require changing. The simplest solution is just to add using
namespace std, CCfits, and heasp.
arf:
- Changed default RowNumber from 1 to 0 in arfII constructor and read methods.
pha:
- Improved handling of multiple SPECTRUM extensions in the same file.
- Catch the potential confusion that the column is called COUNTS but
the HDUCLAS3 keyword should be COUNT
- Improved error trapping when writing a new spectrum file.
- Improved formatting of output from disp.
- Changed default SpectrumNumber from 1 to 0 in phaII constructor and
read methods.
- Trapped a failure to create the SPECTRUM extension in phaII::write.
rmf:
- Fixed error in vector constructor in multiplyByModel
- Added test to check method for the low energy of a channel being
less than the high energy of a channnel.
- Improved diagnostics when doing consistency check between MATRIX
and EBOUNDS extensions.
general:
- Added typedefs for RealVector, IntegerVector, and StringVector.
- Renamed the isValidXUnits, isValidYUnits, calcXfactor, calcYfactor
routines in SPutils to start with SP like all the other routines.
- Added support for the STOKESPR and STOKESWT keywords for
spectropolarimetry. These are most likely to appear in the pha and
arf files but there is the option of including them in rmf files as
well. The utility routine SPcheckStokesConsistency can be used to
check for consistency between pairs of parameter and weighting strings.
- Fixed error in SPstringTform and added all the possible pha type II
columns to the list to be ignored by SPcopyCols.
- Fixed SPreadKey and SPreadCol to work better with type II files.
The RowNumber to default to type I is changed from 1 to 0 because
otherwise there was an ambiguity when reading the first row.
C library:
- Added CheckRMF function.
- Added support for Stokes parameter and weighting in pha, arf, and
rmf objects and structures.
- Fixed errors in converting between rmf objects and structures which
were introduced when rmf objects switched to using vectors of vectors.
********************************************************************************
CALTOOLS
********************************************************************************
- quzcif: Fixed output formatting to prevent jumbled messages and to
prevent large extension numbers from overflowing the numerical field;
fixed error in reporting allowed values of codename (CAL_CNAM) under
some OS, and fixed memory allocation error
- quzcif2: New task intended to eventually replace quzcif. Includes
a new parameter "caldbver" which can be used to query files in a
specified version of the caldb.indx file. For example:
% quzcif2 nicer xti - - ALIGNMENT now now - caldbver=caldb.indx20170814
This queries the good (quality = 0) ALIGNMENT file in the caldb.indx
file version caldb.indx20170814. The default value of caldbver is "-",
which uses the current, most up-to-date caldb.indx file.
- The calibration library (callib) added/updated the following new
library routines:
* gtcalf2 - includes caldbver_str parameter for specifying the
CALDB version
* gtcnam2 - prints the unique cal_cname values as a character string
from a specified calibration index field
* calutils - includes a new subroutine add2cnamlist which creates
a dynamic character array to store allowed CAL_CNAM values
* cifsl3 - uses stricter deallocation of dynamically allocated arrays
********************************************************************************
FV
********************************************************************************
- (fitsTcl) Updated for use with new interface to CFITSIO calculator parser
********************************************************************************
HEAGEN
********************************************************************************
- barycorr: (v2.17) No longer attempts to barycenter the FPM_SEL
extension of cleaned event files, preventing an error; fixed case
of negative MJD causing infinite loop
- mkintensfilt (NEW): New tool to calculate GTIs based on a light curve
and an intensity selection.
********************************************************************************
HEASARC
********************************************************************************
- extractor: Fix for events files with no imaging coordinates,
which was causing strange behavior for HALOSAT data
- nh: Enforce chat level for all output starting from beginning,
and prevent logging if lchat=0
- searchflickpix: Reduced number of asterisks (from 3 to 2) in some
ahlog warning output to avoid misleading occurrences of "***" in
output from make.
********************************************************************************
HEASPTOOLS
********************************************************************************
- ftascii2pha: (NEW) Replacement for the older task ascii2pha.
- ftchkarf: (NEW) heasp-based version of chkarf, works for both
type I and type II files and for multiple SPECRESP extensions
in the same file.
- ftchkpha: Added support for type II files and multiple extensions in file.
- ftchkrmf: Slight change to output format.
- ftgrouppha: Added support for type II pha files including a new
parameter 'rows' which can be used to specify which spectra to group.
- Trapped the case of infile==outfile and issue warnings/errors.
********************************************************************************
HEATOOLS
********************************************************************************
- fselect (wrapper to ftselect): Updated to use an "@" input file for
the expression parameter to try to avoid quotation mark pitfalls
********************************************************************************
XIMAGE
********************************************************************************
- Updated to use new version 9.2.9 of the AST library:
https://github.com/Starlink/ast/releases
- Updates to allow compilation with Apple clang and on the arm64
architecture.
- gencolor.tcl, ra_dec_to_pixel.tcl: Updates for removing tmp
files without prompt; increased chatter setting for diagnostic
information; ra_dec_to_pixel.tcl: added save ra/dec to .par file
********************************************************************************
XRONOS
********************************************************************************
- Fixed bug in efold: with multiple time series, the longer lightcurve
does not need to be the first time series
********************************************************************************
XSELECT
********************************************************************************
- Changed filter intensity to use the new mkintensfilt tool instead of
maketime.
- Turn off the error checking code in xsl_wrtcf when running fplot
to fix problems with PGPLOT.
********************************************************************************
XSPEC
********************************************************************************
XSPEC v12.13.0:
New Models:
cglumin - convolution model to calculate luminosity with the source
distance in kpc
Updated models:
- Improved parallelism between mdefine'd and built-in model
components. Any mdefine'd models now appear in the "model ?" list
indicated by a hash (#) sign.
- Updated wrappers for change of ismabs, ismdust, and olivineabs to
double precision Fortran in previous release.
- Note that internal changes to model handling mean that initpackage
should be rerun for any local models.
Plotting:
- Added plot fitstat option. This supersedes plot chisq and works
correctly for all fit statistics.
- Added (l)edata and (l)eedata plot options.
- Fixed X-axis ranges when stacking plots vertically. Use common X-axis
if plots are not disjoint by more than an order of magnitude.
- Added setplot areascale/noareascale options which modify the data,
background, and residual plots by dividing by the AREASCAL keyword or
vector. This is mainly useful for the XMM RGS. If setplot area is
in use then this has no effect because the AREASCAL values are
included when dividing by the area.
- Added setplot errortype option to specify how the error bars should be
calculated for the plot (note - this has no effect on fitting).
This can already been done as part of the setplot rebin option.
MCMC:
- Added new chain proposal uniform option which gives a uniform
distribution between the parameter lower and upper limits. Useful for
setting initial walkers in gw method.
Miscellaneous:
- Added comp and group options to show all parameters in a given
component or group to show parm free, fixed, and linked.
- Fixed simftest.tcl so that it returns the model to the state on input.
Added reducedchisq.tcl and writesimpars.tcl scripts.
- Now stores DETNAM, FILTER, STOKESPR, STOKESWT keyword values for
spectra and will give warnings about inconsistencies with
background/correction, arfs, and rmfs.
- Improved diagnostic output when ARF files fail to load.
- Fixed case where all spectra are removed while using extended
energies, and table models are in use.
- tclout datagrp now returns 0 rather than 1 when no spectra are loaded.
PyXspec Release Notes for version 2.1.1 (Xspec 12.13.0)
New Features:
- The 'Fit.goodness()' function takes new optional argument,
'[no]fit', to provide same functionality available in standard
Xspec's 'goodness' command.
- New attributes for the 'Fit' class: 'previousGoodness' and
'previousGoodnessSims'
- New function 'AllModels.mdefine()' to provide access to standard
Xspec's 'mdefine' capability.
- The 'Xset.abund' attribute can now take an input list of 30 values
in addition to the already available selection of built-in tables.
See also:
https://heasarc.gsfc.nasa.gov/lheasoft/xanadu/xspec/issues/issues.html
********************************************************************************
XSTAR
********************************************************************************
XSTAR v2.59:
Changes since version 2.58e:
- Fixed bug introduced in version 2.58 which omitted some cross sections
for near-edge resonance structure in opacity; also fixed bug which
omitted charge transfer ionization of O I.
********************************************************************************
ASCA
********************************************************************************
- addascaspec: Passed the expression to mathpha in a temporary file to
evade the 1000 character limit for parameter values.
********************************************************************************
IXPE
********************************************************************************
- Python code v1.6, compiled tools v3.0.0
- Please note that the IXPE Python source code has been moved out
of (but still relies on) HEASoftPy and is now co-located with
the compiled IXPE tools.
- ixpeaspcorr:
The event position corrections are now handled differently.
The basic algorithm remains the same as the previous version.
* The events are split into two groups: Those events recorded
when the spacecraft was using OH1 for attitude, and those
events recorded when the spacecraft was using OH2 for attitude.
These are corrected by the previous algorithm separately, and
are corrected to the same average position (the OH1 group
average position).
* The par file interface now requires the attitude file name
argument "att_path" in order to determine the OH1 and OH2
event groups.
Changed the html file to reflect the functionality below and the
changes to the parameter behavior noted in the v1.6 release notes.
Due to changes introduced in astropy.table.Table version 4.3,
the "nan" values previously read from a file have now become
masked values, and the column has become a MaskedColumn, with
MaskedArray arrays. This led to changes in
ixpeaspcorr_lib.AttitudeFile.calc_attitude_vtis and
ixpeaspcorr_lib.Attitudefile.calc_txydz_corr to conditionally
detect this type and change the accounting for masked or nan
values to work with both the older and the newer versions of
astropy.table.Table.
A bug was discovered in ixpeaspcorr_lib.AttitudeFile.calc_txydz_corr
that was causing incorrect values of TXYDZ_CORR[1] to be written
to the output file. This was fixed.
ixpeboomdriftcorr module added. Functionality added to
ixpeaspcorr_lib.AttitudeFile.calc_txydz_corr to allow it to
use the TXYDZ_CORRBD attitude-file column output by that module
as input, if it is available.
- ixpeboomdriftcorr:
New task corrects for modeled boom drift errors using a specific
state vector and event time stamps.
The function is to remove the periodic position offsets caused
by the thermal distortions of the IXPE boom during each orbit.
The .par, .py, .py.hml, and _lib.py files were all added to
support this functionality.
- ixpeexpmap:
In some cases (faint, diffuse x-ray sources), ixpeboomdriftcorr
will be used without ixpaspcorr subsequently being applied to
the data. This module was changed to use the aspect data from
ixpeaspcorr (TXYDZ_CORR column) if present, then
ixpeboomdriftcorr (TXYDZ_CORRBD column) if present.
(in ixpeexpmap_lib.PointingDirectionMap.add_aspect_solutions).
Due to changes introduced in astropy.table.Table version 4.3,
the "nan" values previously read from a file have now become
masked values, and the column has become a MaskedColumn, with
MaskedArray arrays. This led to changes in
ixpeexpmap_lib.PointingDirectionMap.add_aspect_solutions to
conditionally test for the MaskedArray type and handle the
resulting mask or "nan" values.
- ixpegaincorrpkmap:
This module no longer uses Peak Gain maps from the CALDB, but
uses Periodic Peak Gain maps produced by the IXPE pipeline from
daily calibrations, and released with the new public data sets.
The parameter file interface has therefore changed to take
a named Periodic Peak Gain map file as input, using the new
"pkgainfile" parameter. These files are included in released
data sets as ".ppg" files, which are included in the "auxil"
subdirectory of the observation segment data directory.
- ixpegaincorrtemp:
ixpegaincorrtemp.html changed to make the temperature file name
used in the Example consistent with the DU number of the events file.
********************************************************************************
MAXI
********************************************************************************
- mxdownload_wget: Replaced retired legacy.gsfc.nasa.gov address with
heasarc.gsfc.nasa.gov
- mxproduct: Adding leading zeros to tile numbers < 100, correct use
of $num_ebands to \$num_ebands
********************************************************************************
NICER
********************************************************************************
NICERDAS 10 is a major NICER software release that contains many
new features. Please see the following link for a narrative of
changes in this version:
https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/heasoft631/
* Important note regarding the niprefilter task: When compiled on
Apple hardware with the entire GCC v12 compiler suite (e.g. from
Homebrew or MacPorts) the SAA_TIME column of the output prefilter
extension has been seen to be slightly off from expected prototype
values. We are still investigating, but in any case we recommend
on Macs (as noted abvoe) that you use clang/clang++ to build
heasoft rather than third-party (e.g. MacPorts, Homebrew) gcc/g++
v12 compilers. Using gfortran v12 paired with clang/clang++ is
acceptable and does not result in the prototype differences.
- New NICER background models included in NICER:
* 3C50: nibackgen3C50 now incorporated in NICERDAS
* Space Weather: nibkgestimator now incorporated in NICERDAS
(and HEASoft equivalents niswbkgspect, niswbkglc)
* SCORPEON: new background modeling tool (niscorpspect,
niscorpspectmod)
- New high level product extraction pipeline tools
* nicerl3-spect: complete NICER spectral extraction pipeline
* nicerl3-lc: complete NICER light curve extraction pipeline
- New lower level supporting tasks
* niextspect: extract standard NICER spectrum
* niextlc: extract standard NICER light curve
* niphasyserr: apply standard NICER systematic error vector
to spectrum
* niphaquality: apply NICER QUALITY screeing to spectral
channels
* nihaloem: calculate Halo and Local Hot Bubble emission
estimates
- New tools (nicerl2, niprefilter, niprefilter2) now accept alias NICERV4
- New tool nicalfixevt fixes invalid PI_FAST pulse heights
- New task niautoscreen for automatic per-detector screening
- New task nigeodown
- Tools nicerl2, niprefilter, niprefilter2 now accept alias NICERV4
- nicerl2 (and niprefilter, nicercal, nimpucal) can reset the
RA_OBJ / DEC_OBJ keywords of output products
- nicerl2: SOLARPHI column is now SOLAR_PHI
- nicerarf: only allow response generation on non-barycentered data
- nicerarf: user can specify R.A. / Dec. as OBJ or NOM to use the
metadata keywords embedded in the spectrum file
- nicerarf / nicerrmf: fix bugs when updatepha=YES, though
updatepha=NO is recommended
- nicerrmf, niexposum, niextract-events, nifpmsel: fix off-by-one bug
in gtifilter()
- nicerarf and nicerrmf write ANCRFILE, BACKFILE, RESPFILE with no
leading path components (i.e., filename only)
- niexposum: automatically determine which extension and column to
operate on
- nivigsum (nicerarf): now provides a warning when target is out of
field of view
- niprefilter2: now has vercheck parameter which checks if filter file
needs reprocessing
- nicercal (and nicerpi, nicertimecal, nimpucal) now have
caldbcheck / incremental keywords to prevent reprocessing if the
output product is already up to date
- niprefilter2: user can now explicitly request / reject columns with
COLNAME or -COLNAME
- nifpmsel: now accepts detlist=@filename.txt syntax to read from file
- barycorr no longer attempts to barycenter the FPM_SEL extension of
cleaned event files, preventing an error
- nifpmsel: uses more sensible tmin/tmax for empty GTIs, which prevents
barycentering failures or lockups
- nimaketime: default overonly_range=0.0-30.0 and overonly_expr=NONE
- All NICER tasks more consistently write HISTORY keywords
********************************************************************************
NUSTAR
********************************************************************************
- Updates to allow support of Mac arm64 hardware
********************************************************************************
SUZAKU
********************************************************************************
- Updates to allow compilation with Apple clang and on the arm64
architecture; removes the 'mkanlinit' task code generator from
the build in favor of static code
- hxdpinxbpi.par: Fixed missing closing quote, which was causing
problems for HEASoftPy.
********************************************************************************
SWIFT
********************************************************************************
GEN:
- swifttime: (v1.8) Revised extrapolation coefficients from 2021-2022
linear fit
- swiftxform: (v2.4) For UVOT, use DETNAM/WHEELPOS from primary HDU
if extension has UNKNOWN (2200)
BAT:
- Updates to allow support of Mac arm64 hardware and newer compilers
UVOT:
- uvotgrplot: Updates to PGPLOT Pelr module to allow support of
Mac arm64 hardware
- uvotimgrism: (v1.7) Use DETNAM/WHEELPOS from primary HDU if
specified extension has UNKNOWN (2200)
XRT:
- Updates to allow support of Mac arm64 hardware
********************************************************************************
XTE
********************************************************************************
- pcabackest (v3.12c) Improved use of CALDB output (only use unique
CALDB-returned files rather than file+ext); fixed string overflow
bug and uninitialized variable issues which cause unreliability
in some situations
********************************************************************************
HEASoftPy:
********************************************************************************
New version 1.2:
- Fixed a bug for float parameters with INDEF
- Fix for tasks with a parameter called "name"
- Fix for case of comma in parameter value
- Fix for parameter string containing new line character
- Fix to handle parameters that pass a comma-list inside quotes
- Removed heasoftpy/packages as python tasks are now handled in
the general heasoft framework
- Updated the return object to contain the parameters used in
the call not those updated by the task, if any
********************************************************************************
MISC.
********************************************************************************
- Restored full support for Apple clang/clang++ compilers (Intel
or Silicon architectures)
- Added support for compilers with Apple Silicon arm64 architecture
- Enforce requirement of Python 3.7 or newer (for HEAsoftPy) when
IXPE software is included. When installing the IXPE software,
HEASoftPy prerequisites are enforced: NumPy v1.7 or newer,
AstroPy v4.0.0 or newer, and SciPy v1.6.0 or newer.
- Attempt to prevent unintentional linking of C/Perl interface libraries
with a MacPorts or Homebrew copy of the CFITSIO library, which can
happen when using a MacPorts or Homebrew Perl, causing run-time
crashes in Perl tasks (e.g., barycorr).
- Switched default build architecture for Mac ARM hardware from Intel
to ARM: changes configure option "--enable-mac_arm_build" to
"--enable-mac_intel_build"; attempts to correctly handle compiler
architecture in either case, for example adding "-arch x86_64" to
relevant *FLAGS if using clang/clang++ for the Intel option.
- Added new installation utility "lhea-fixpython" to be run on
Python tasks (e.g., in NICER and IXPE): replaces the first line
of a script with the configured Python executable, allowing them
to be run outside of an interactive Python session.
