National Aeronautics and Space Administration

********************************************************************************
                       RELEASE NOTES FOR HEASOFT 6.31
                             November 15, 2022
********************************************************************************

The HEASoft 6.31 release is driven by new support for the Apple
Silicon (arm64) architecture and updates to mission-specific data
analysis software (IXPE, NICER, NuSTAR, Swift), but as always
includes other enhancements and fixes.

Configuration-related changes throughout the package mean that users
who already have HEASoft installed will need to completely replace
their software installation; there is no clean way to simply overlay
this new release or to update an existing installation.

                   *** IMPORTANT NOTE FOR MAC USERS: ***

HEASoft 6.31 restores support for the Apple clang/clang++ compilers,
and introduces support for builds of the software on the Silicon arm64
architecture.  As a result, please note the following:

  * The default build architecture for HEASoft on Silicon is now
    arm64.  That is, the configuration is expecting to find native
    arm64 compilers, NOT the Intel (x86_64) compilers you may have
    installed with Rosetta and used for previous HEASoft releases.

    If desired, it should be possible to still build HEASoft on
    Siicon with the Rosetta Intel compilers using the configure option
    "--enable-mac_intel_build", but we do not recommend this approach.

  * The preferred C compilers for any Mac architecture (Silicon or
    Intel) are now clang and clang++, which should by default operate
    in the correct mode for the native architecture, i.e. arm64 on
    Silicon and x86_64 on Intel.  Similarly, the preferred Perl
    interpreter is /usr/bin/perl which should again match the native
    architecture.

    As always, a Fortran compiler is necessary, and should be chosen
    to match the architecture of the C compilers.  For example, when
    on Silicon using CC=/usr/bin/clang (arm64), you should choose a
    Fortran compiler which is also arm64.  The Homebrew and MacPorts
    package managers (for example) provide Fortran compilers which
    by default - i.e. when NOT using Rosetta - should match the native
    architecture.

    Lastly, note that your choice of Python should also match the
    architecture of your other build choices.  For example,
    Anaconda offers Intel or Silicon versions of their software.

********************************************************************************
                                    CFITSIO
********************************************************************************

Version 4.2.0 (Nov. 2022)

- This release includes patches to security vulnerabilities.  We
  strongly encourage this upgrade, particularly for those running
  CFITSIO in web accessible applications.

- Fix to fits_read_key function, which was failing to properly read
  keywords declared type TUINT on compilers where sizeof(int) =
  sizeof(long).

- Added new functions fits_read_cols and fits_write_cols to efficiently
  read or write multiple columns in a single function call

- Added new function fits_copy_selrows to copy only selected rows, such
  as the selected rows returned by fits_find_rows

- Added new calculator functions ERF(X), ERFC(X) and GAMMA(X), which are
  mathematical special functions of the same name

- Added new calculator function GTIFIND() which reports which GTI row
  brackets a given time sample

- Added new calculator functions which operate upon vector table
  values NAXIS(V), NAXES(V,n), ELEMENTNUM(V) and AXISELEM(V,n),
  and ARRAY(X,d) which promotes scalar X to a vector or array
  with given dimensions.

- The CFITSIO histogramming code now handles binning by any
  arbitrary calculator expression rather than just a column name.
  Both the binned columns as well as the optional weights may be
  calculator expressions, enclosed in parentheses.

- Binning of vector columns or expressions is now supported, as long
  as all binned inputs (as well as the optional weighting) have the
  same vector dimensions.  Binning of variable-length columns remains
  unsupported.

- A bug that caused histogram weights from columns that are null values
  to be tallied along with non-null values has been fixed.

- The CFITSIO calculator and histogramming functionality is now
  fully reentrant and does not require multithreading interlocks.

- A long-standing segmentation fault bug in the histogramming code
  related to binning any value using the "reciprocal" /XXXX notation
  has been fixed.

- Added mutex locks for thread safety in ftgiou and ftfiou.

- Added several Fortran wrappers to support image read/write when
  'fpixel' and 'nelements' are 8-byte integers.

- Fixed bug which was adding spaces to some cases of long string key
  value output.

Perl interface (Astro::FITS::CFITSIO):

- Added functions fits_make_histd(), fits_make_histde(), fits_copy_selrows()

********************************************************************************
                                    ATTITUDE
********************************************************************************

- atFunctions library:

  * v3.5: Added atSXSBrazil(): New routine adds support for the
    XRISM Resolve instrument with a particular definition of the
    South Atlantic Anomaly (SAA)

  * v3.4: Bug fix in atPol60ToVect()

********************************************************************************
                                     HEASP
********************************************************************************

Important change:

heasp.h no longer includes using namespace std and CCfits and all of
heasp has been placed in its own namespace called heasp. This should
reduce the possibility of name conflicts between heasp and other
libraries. However, it does mean that any program using heasp will
require changing. The simplest solution is just to add using
namespace std, CCfits, and heasp.

arf:

- Changed default RowNumber from 1 to 0 in arfII constructor and read methods.

pha:

- Improved handling of multiple SPECTRUM extensions in the same file.
- Catch the potential confusion that the column is called COUNTS but
  the HDUCLAS3 keyword should be COUNT
- Improved error trapping when writing a new spectrum file.
- Improved formatting of output from disp.
- Changed default SpectrumNumber from 1 to 0 in phaII constructor and
  read methods.
- Trapped a failure to create the SPECTRUM extension in phaII::write.

rmf:

- Fixed error in vector constructor in multiplyByModel
- Added test to check method for the low energy of a channel being
  less than the high energy of a channnel.
- Improved diagnostics when doing consistency check between MATRIX
  and EBOUNDS extensions.

general:

- Added typedefs for RealVector, IntegerVector, and StringVector.
- Renamed the isValidXUnits, isValidYUnits, calcXfactor, calcYfactor
  routines in SPutils to start with SP like all the other routines.
- Added support for the STOKESPR and STOKESWT keywords for
  spectropolarimetry. These are most likely to appear in the pha and
  arf files but there is the option of including them in rmf files as
  well. The utility routine SPcheckStokesConsistency can be used to
  check for consistency between pairs of parameter and weighting strings.
- Fixed error in SPstringTform and added all the possible pha type II
  columns to the list to be ignored by SPcopyCols.
- Fixed SPreadKey and SPreadCol to work better with type II files.
  The RowNumber to default to type I is changed from 1 to 0 because
  otherwise there was an ambiguity when reading the first row.

C library:

- Added CheckRMF function.
- Added support for Stokes parameter and weighting in pha, arf, and
  rmf objects and structures.
- Fixed errors in converting between rmf objects and structures which
  were introduced when rmf objects switched to using vectors of vectors.

********************************************************************************
                                    CALTOOLS
********************************************************************************

- quzcif: Fixed output formatting to prevent jumbled messages and to
  prevent large extension numbers from overflowing the numerical field;
  fixed error in reporting allowed values of codename (CAL_CNAM) under
  some OS, and fixed memory allocation error

- quzcif2: New task intended to eventually replace quzcif.  Includes
  a new parameter "caldbver" which can be used to query files in a
  specified version of the caldb.indx file.  For example:

    % quzcif2 nicer xti - - ALIGNMENT now now - caldbver=caldb.indx20170814

  This queries the good (quality = 0) ALIGNMENT file in the caldb.indx
  file version caldb.indx20170814. The default value of caldbver is "-",
  which uses the current, most up-to-date caldb.indx file.

- The calibration library (callib) added/updated the following new
  library routines:

  * gtcalf2 - includes caldbver_str parameter for specifying the
    CALDB version
  * gtcnam2 - prints the unique cal_cname values as a character string
    from a specified calibration index field
  * calutils - includes a new subroutine add2cnamlist which creates
    a dynamic character array to store allowed CAL_CNAM values
  * cifsl3 - uses stricter deallocation of dynamically allocated arrays

********************************************************************************
                                      FV
********************************************************************************

- (fitsTcl) Updated for use with new interface to CFITSIO calculator parser

********************************************************************************
                                    HEAGEN
********************************************************************************

- barycorr: (v2.17) No longer attempts to barycenter the FPM_SEL
  extension of cleaned event files, preventing an error; fixed case
  of negative MJD causing infinite loop

- mkintensfilt (NEW): New tool to calculate GTIs based on a light curve
  and an intensity selection.

********************************************************************************
                                    HEASARC
********************************************************************************

- extractor: Fix for events files with no imaging coordinates,
  which was causing strange behavior for HALOSAT data

- nh: Enforce chat level for all output starting from beginning,
  and prevent logging if lchat=0

- searchflickpix: Reduced number of asterisks (from 3 to 2) in some
  ahlog warning output to avoid misleading occurrences of "***" in
  output from make.

********************************************************************************
                                   HEASPTOOLS
********************************************************************************

- ftascii2pha: (NEW) Replacement for the older task ascii2pha.

- ftchkarf: (NEW) heasp-based version of chkarf, works for both
  type I and type II files and for multiple SPECRESP extensions
  in the same file.

- ftchkpha: Added support for type II files and multiple extensions in file.

- ftchkrmf: Slight change to output format.

- ftgrouppha: Added support for type II pha files including a new
  parameter 'rows' which can be used to specify which spectra to group.

- Trapped the case of infile==outfile and issue warnings/errors.

********************************************************************************
                                    HEATOOLS
********************************************************************************

- fselect (wrapper to ftselect): Updated to use an "@" input file for
  the expression parameter to try to avoid quotation mark pitfalls

********************************************************************************
                                    XIMAGE
********************************************************************************

- Updated to use new version 9.2.9 of the AST library:
  https://github.com/Starlink/ast/releases

- Updates to allow compilation with Apple clang and on the arm64
  architecture.

- gencolor.tcl, ra_dec_to_pixel.tcl: Updates for removing tmp
  files without prompt; increased chatter setting for diagnostic
  information; ra_dec_to_pixel.tcl: added save ra/dec to .par file

********************************************************************************
                                    XRONOS
********************************************************************************

- Fixed bug in efold: with multiple time series, the longer lightcurve
  does not need to be the first time series

********************************************************************************
                                    XSELECT
********************************************************************************

- Changed filter intensity to use the new mkintensfilt tool instead of
  maketime.

- Turn off the error checking code in xsl_wrtcf when running fplot
  to fix problems with PGPLOT.

********************************************************************************
                                     XSPEC
********************************************************************************


XSPEC v12.13.0:

New Models:

cglumin - convolution model to calculate luminosity with the source
          distance in kpc

Updated models:

- Improved parallelism between mdefine'd and built-in model
  components. Any mdefine'd models now appear in the "model ?" list
  indicated by a hash (#) sign.

- Updated wrappers for change of ismabs, ismdust, and olivineabs to
  double precision Fortran in previous release.

- Note that internal changes to model handling mean that initpackage
  should be rerun for any local models.

Plotting:

- Added plot fitstat option. This supersedes plot chisq and works
  correctly for all fit statistics.

- Added (l)edata and (l)eedata plot options.

- Fixed X-axis ranges when stacking plots vertically. Use common X-axis
  if plots are not disjoint by more than an order of magnitude.

- Added setplot areascale/noareascale options which modify the data,
  background, and residual plots by dividing by the AREASCAL keyword or
  vector. This is mainly useful for the XMM RGS. If setplot area is
  in use then this has no effect because the AREASCAL values are
  included when dividing by the area.

- Added setplot errortype option to specify how the error bars should be
  calculated for the plot (note - this has no effect on fitting).
  This can already been done as part of the setplot rebin option.

MCMC:

- Added new chain proposal uniform option which gives a uniform
  distribution between the parameter lower and upper limits. Useful for
  setting initial walkers in gw method.

Miscellaneous:

- Added comp and group options to show all parameters in a given
  component or group to show parm free, fixed, and linked.

- Fixed simftest.tcl so that it returns the model to the state on input.
  Added reducedchisq.tcl and writesimpars.tcl scripts.

- Now stores DETNAM, FILTER, STOKESPR, STOKESWT keyword values for
  spectra and will give warnings about inconsistencies with
  background/correction, arfs, and rmfs.

- Improved diagnostic output when ARF files fail to load.

- Fixed case where all spectra are removed while using extended
  energies, and table models are in use.

- tclout datagrp now returns 0 rather than 1 when no spectra are loaded.


PyXspec Release Notes for version 2.1.1 (Xspec 12.13.0)

New Features:

- The 'Fit.goodness()' function takes new optional argument,
  '[no]fit', to provide same functionality available in standard
  Xspec's 'goodness' command.

- New attributes for the 'Fit' class: 'previousGoodness' and
  'previousGoodnessSims'

- New function 'AllModels.mdefine()' to provide access to standard
  Xspec's 'mdefine' capability.

- The 'Xset.abund' attribute can now take an input list of 30 values
  in addition to the already available selection of built-in tables.

See also:

 https://heasarc.gsfc.nasa.gov/lheasoft/xanadu/xspec/issues/issues.html

********************************************************************************
                                     XSTAR
********************************************************************************

XSTAR v2.59:

Changes since version 2.58e:

- Fixed bug introduced in version 2.58 which omitted some cross sections
  for near-edge resonance structure in opacity; also fixed bug which
  omitted charge transfer ionization of O I.

********************************************************************************
                                     ASCA
********************************************************************************

- addascaspec: Passed the expression to mathpha in a temporary file to
  evade the 1000 character limit for parameter values.

********************************************************************************
                                     IXPE
********************************************************************************

- Python code v1.6, compiled tools v3.0.0

- Please note that the IXPE Python source code has been moved out
  of (but still relies on) HEASoftPy and is now co-located with
  the compiled IXPE tools.

- ixpeaspcorr:

  The event position corrections are now handled differently.
  The basic algorithm remains the same as the previous version.

  * The events are split into two groups:  Those events recorded
    when the spacecraft was using OH1 for attitude, and those
    events recorded when the spacecraft was using OH2 for attitude.
    These are corrected by the previous algorithm separately, and
    are corrected to the same average position (the OH1 group
    average position).

  * The par file interface now requires the attitude file name
    argument "att_path" in order to determine the OH1 and OH2
    event groups.

  Changed the html file to reflect the functionality below and the
  changes to the parameter behavior noted in the v1.6 release notes.

  Due to changes introduced in astropy.table.Table version 4.3,
  the "nan" values previously read from a file have now become
  masked values, and the column has become a MaskedColumn, with
  MaskedArray arrays.  This led to changes in
  ixpeaspcorr_lib.AttitudeFile.calc_attitude_vtis and
  ixpeaspcorr_lib.Attitudefile.calc_txydz_corr to conditionally
  detect this type and change the accounting for masked or nan
  values to work with both the older and the newer versions of
  astropy.table.Table.

  A bug was discovered in ixpeaspcorr_lib.AttitudeFile.calc_txydz_corr
  that was causing incorrect values of TXYDZ_CORR[1] to be written
  to the output file.  This was fixed.

  ixpeboomdriftcorr module added.  Functionality added to
  ixpeaspcorr_lib.AttitudeFile.calc_txydz_corr to allow it to
  use the TXYDZ_CORRBD attitude-file column output by that module
  as input, if it is available.

- ixpeboomdriftcorr:

  New task corrects for modeled boom drift errors using a specific
  state vector and event time stamps.

  The function is to remove the periodic position offsets caused
  by the thermal distortions of the IXPE boom during each orbit.
  The .par, .py, .py.hml, and _lib.py files were all added to
  support this functionality.

- ixpeexpmap:

  In some cases (faint, diffuse x-ray sources), ixpeboomdriftcorr
  will be used without ixpaspcorr subsequently being applied to
  the data.  This module was changed to use the aspect data from
  ixpeaspcorr (TXYDZ_CORR column) if present, then
  ixpeboomdriftcorr (TXYDZ_CORRBD column) if present.
  (in ixpeexpmap_lib.PointingDirectionMap.add_aspect_solutions).

  Due to changes introduced in astropy.table.Table version 4.3,
  the "nan" values previously read from a file have now become
  masked values, and the column has become a MaskedColumn, with
  MaskedArray arrays.  This led to changes in
  ixpeexpmap_lib.PointingDirectionMap.add_aspect_solutions to
  conditionally test for the MaskedArray type and handle the
  resulting mask or "nan" values.

- ixpegaincorrpkmap:

  This module no longer uses Peak Gain maps from the CALDB, but
  uses Periodic Peak Gain maps produced by the IXPE pipeline from
  daily calibrations, and released with the new public data sets.

  The parameter file interface has therefore changed to take
  a named Periodic Peak Gain map file as input, using the new
  "pkgainfile" parameter.  These files are included in released
  data sets as ".ppg" files, which are included in the "auxil"
  subdirectory of the observation segment data directory.

- ixpegaincorrtemp:

  ixpegaincorrtemp.html changed to make the temperature file name
  used in the Example consistent with the DU number of the events file.

********************************************************************************
                                     MAXI
********************************************************************************

- mxdownload_wget: Replaced retired legacy.gsfc.nasa.gov address with
  heasarc.gsfc.nasa.gov

- mxproduct: Adding leading zeros to tile numbers < 100, correct use
  of $num_ebands to \$num_ebands

********************************************************************************
                                     NICER
********************************************************************************

NICERDAS 10 is a major NICER software release that contains many
new features.  Please see the following link for a narrative of
changes in this version:

  https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/heasoft631/

* Important note regarding the niprefilter task: When compiled on
  Apple hardware with the entire GCC v12 compiler suite (e.g. from
  Homebrew or MacPorts) the SAA_TIME column of the output prefilter
  extension has been seen to be slightly off from expected prototype
  values.  We are still investigating, but in any case we recommend
  on Macs (as noted abvoe) that you use clang/clang++ to build
  heasoft rather than third-party (e.g. MacPorts, Homebrew) gcc/g++
  v12 compilers.  Using gfortran v12 paired with clang/clang++ is
  acceptable and does not result in the prototype differences.

- New NICER background models included in NICER:

   * 3C50: nibackgen3C50 now incorporated in NICERDAS

   * Space Weather: nibkgestimator now incorporated in NICERDAS
     (and HEASoft equivalents niswbkgspect, niswbkglc)

   * SCORPEON: new background modeling tool (niscorpspect,
     niscorpspectmod)

- New high level product extraction pipeline tools

   * nicerl3-spect: complete NICER spectral extraction pipeline
   * nicerl3-lc: complete NICER light curve extraction pipeline

- New lower level supporting tasks

   * niextspect: extract standard NICER spectrum

   * niextlc: extract standard NICER light curve

   * niphasyserr: apply standard NICER systematic error vector
     to spectrum

   * niphaquality: apply NICER QUALITY screeing to spectral
     channels

   * nihaloem: calculate Halo and Local Hot Bubble emission
     estimates

- New tools (nicerl2, niprefilter, niprefilter2) now accept alias NICERV4

- New tool nicalfixevt fixes invalid PI_FAST pulse heights

- New task niautoscreen for automatic per-detector screening

- New task nigeodown

- Tools nicerl2, niprefilter, niprefilter2 now accept alias NICERV4

- nicerl2 (and niprefilter, nicercal, nimpucal) can reset the
  RA_OBJ / DEC_OBJ keywords of output products

- nicerl2: SOLARPHI column is now SOLAR_PHI

- nicerarf: only allow response generation on non-barycentered data

- nicerarf: user can specify R.A. / Dec. as OBJ or NOM to use the
  metadata keywords embedded in the spectrum file

- nicerarf / nicerrmf: fix bugs when updatepha=YES, though
  updatepha=NO is recommended

- nicerrmf, niexposum, niextract-events, nifpmsel: fix off-by-one bug
  in gtifilter()

- nicerarf and nicerrmf write ANCRFILE, BACKFILE, RESPFILE with no
  leading path components (i.e., filename only)

- niexposum: automatically determine which extension and column to
  operate on

- nivigsum (nicerarf): now provides a warning when target is out of
  field of view

- niprefilter2: now has vercheck parameter which checks if filter file
  needs reprocessing

- nicercal (and nicerpi, nicertimecal, nimpucal) now have
  caldbcheck / incremental keywords to prevent reprocessing if the
  output product is already up to date

- niprefilter2: user can now explicitly request / reject columns with
  COLNAME or -COLNAME

- nifpmsel: now accepts detlist=@filename.txt syntax to read from file

- barycorr no longer attempts to barycenter the FPM_SEL extension of
  cleaned event files, preventing an error

- nifpmsel: uses more sensible tmin/tmax for empty GTIs, which prevents
  barycentering failures or lockups

- nimaketime: default overonly_range=0.0-30.0 and overonly_expr=NONE

- All NICER tasks more consistently write HISTORY keywords

********************************************************************************
                                     NUSTAR
********************************************************************************

- Updates to allow support of Mac arm64 hardware

********************************************************************************
                                    SUZAKU
********************************************************************************

- Updates to allow compilation with Apple clang and on the arm64
  architecture; removes the 'mkanlinit' task code generator from
  the build in favor of static code

- hxdpinxbpi.par: Fixed missing closing quote, which was causing
  problems for HEASoftPy.

********************************************************************************
                                     SWIFT
********************************************************************************

GEN:

- swifttime: (v1.8) Revised extrapolation coefficients from 2021-2022
  linear fit

- swiftxform: (v2.4) For UVOT, use DETNAM/WHEELPOS from primary HDU
  if extension has UNKNOWN (2200)

BAT:

- Updates to allow support of Mac arm64 hardware and newer compilers

UVOT:

- uvotgrplot: Updates to PGPLOT Pelr module to allow support of
  Mac arm64 hardware

- uvotimgrism: (v1.7) Use DETNAM/WHEELPOS from primary HDU if
  specified extension has UNKNOWN (2200)

XRT:

- Updates to allow support of Mac arm64 hardware

********************************************************************************
                                     XTE
********************************************************************************

- pcabackest (v3.12c) Improved use of CALDB output (only use unique
  CALDB-returned files rather than file+ext); fixed string overflow
  bug and uninitialized variable issues which cause unreliability
  in some situations

********************************************************************************
                                  HEASoftPy:
********************************************************************************

New version 1.2:

- Fixed a bug for float parameters with INDEF

- Fix for tasks with a parameter called "name"

- Fix for case of comma in parameter value

- Fix for parameter string containing new line character

- Fix to handle parameters that pass a comma-list inside quotes

- Removed heasoftpy/packages as python tasks are now handled in
  the general heasoft framework

- Updated the return object to contain the parameters used in
  the call not those updated by the task, if any

********************************************************************************
                                     MISC.
********************************************************************************

- Restored full support for Apple clang/clang++ compilers (Intel
  or Silicon architectures)

- Added support for compilers with Apple Silicon arm64 architecture

- Enforce requirement of Python 3.7 or newer (for HEAsoftPy) when
  IXPE software is included.  When installing the IXPE software,
  HEASoftPy prerequisites are enforced: NumPy v1.7 or newer,
  AstroPy v4.0.0 or newer, and SciPy v1.6.0 or newer.

- Attempt to prevent unintentional linking of C/Perl interface libraries
  with a MacPorts or Homebrew copy of the CFITSIO library, which can
  happen when using a MacPorts or Homebrew Perl, causing run-time
  crashes in Perl tasks (e.g., barycorr).

- Switched default build architecture for Mac ARM hardware from Intel
  to ARM: changes configure option "--enable-mac_arm_build" to
  "--enable-mac_intel_build"; attempts to correctly handle compiler
  architecture in either case, for example adding "-arch x86_64" to
  relevant *FLAGS if using clang/clang++ for the Intel option.

- Added new installation utility "lhea-fixpython" to be run on
  Python tasks (e.g., in NICER and IXPE): replaces the first line
  of a script with the configured Python executable, allowing them
  to be run outside of an interactive Python session.

Web page maintained by: Bryan K. Irby