Xamin release notes

Release 2024-03-07

• Released the new Xamin interface, formerly known as "Beta"

Release 2021-05-13T15:52:47-04:00 (a1f3c9e)

• Updated table category descriptions
• Added debugging info (SQL query) in "find matches" results
• Fixed security issue with path parameters
• Added ROSAT All Sky Survey to Aladin lite
• Fixed XMM survey in Aladin lite
• Merged index.jsp and xamin.jsp

Release 2021-03-02

• Added tab for downloadable FTP paths in data products cart
• Changed default setting to display table links
• Updated list of master observation tables
• Fixed data products for fermilpsc
• Fixed xteindex's abstracts products
• Fixed broken chanmaster table links
• Added SQL query info for failed requests

Release 2021-01-06

• Updated batsetrigs (CGRO) data products definitions to reflect current table schema
• Added SQL query info for failed requests

Release 6.8c

• Updated to fix type mismatch error in TAP_lint tests.
• Really fixed the order of DESCRIPTION and VALUES in VOTABLE FIELDs.
• Fixed security issue with getDoc.jsp
• Fixed format issue with DataLink and __row field

Release 6.8a

• Fixed bug where time range queries on tables with set start times and null end times were not being seen

March 25, 2020

• Fixed issue with the order of DESCRIPTION and VALUES in a VOTABLE FIELD. Neither are required, but if both are there, DESCRIPTION needs to be first.

Release 6.8

• Added HDB class heasarc.exec.MultiExec to read WebSec URL list files and query Xamin.
• Added HDB class heasarc.util.StringDiff to difference arrays of strings.
• Add singleRowProd.jsp to display single row of data products.
• Fixed issue with tap queries not having the table in the DataLink accessURL

Release 6.7a

• Made fix for argument processing really work!

Release 6.7

• Fix description fields in tables for vo/tap/tables endpoint
• Fix argument processing of escaped $ and /
• Fix check now feature.

Release 6.6

• Enable inititialization to set search of external table.
• Fixed bug in setting startfv for external FITS files.
• Fixed bug in handling of Short for FITS. This briefly introduced another bug when the value was null.

Release 6.5

• Fix for NICER products.
• Updated URLs for links to ADS
• Updates to make some TAP queries run faster
• Mission updates to support Browse

Release 6.4

• Major update to installation utilities. Adds checks during installation and makes sure executable files are actually done using static list.
• Fix bug in production ingestinos.
• Add many new links to product links
• Create program to generate Xamin link products from Browse zzlink table.
• Fix integers being initialized as strings which caused problem in Edge browser

Release 6.3a

• Add Chandra to HiPS in Aladin Lite

Release 6.3

• Fix to maxrec in SxA services completed.

Release 6.2

• Fixes to handling of table uploads
• Added some code to check that the database is properly closed on exit from a query
• First elements of fix to maxrec in SxA services.

Release 6.1

• Updated to add an amount to the requested size for SIA that's based on the naxis of the image. This amount is also used as the size when size=0 is requested.
• Updated to allow for a responseFormat request in DataLink.
• Many updates to external service to use NAVO registry to dynamically request services. Services are cached for an hour.
• Use of VizieR TAP service to discover available tables with caching of results
• In external resources enable the splitting of available resources into multiple level hierarchy so that not too many tables are in each leaf.
• Allow access to Vizier and HEASARC cone and TAP services
• Support for SIAv2 remote services
• In rendered VOTable render URLs as links.

Release 6.0a

• Only a limited number of queries are allowed to run concurrently.
• There is a upper limit to the number of rows that may be specified for queries in non-streaming formats (Aligned, FITS, Excel). This is currently set at 200,000 for Aligned and FITS and 60,000 for Excel. [The Excel library used also sets a limit of 65535.]
• Updated DataLink queries to use POST is large numbers of IDs are to be retrieved.
• Fixed bug in sorting of existing query.

Release 6.0

• Many changes associated with use of DataLink to extract data products. If a TAP query (async or sync) has product results, DataLink info will also be returned. DataLink requests can include a table and row (id=table:row) or a table, row, and product(s) (id=table:row:product1,product2...). The table can also be included as part of the url, then the id is either just the row or the row and product(s).
• Data products are now searched for only when users request data. When multiple rows are added to the data cart there may be a detectable pause while products are located. Rows without data products will initially still show a product expansion but will indicate that there are no products when expanded.
• Full support for SSA and SIAv2 queries
• Support for BINARY2 data format in VOTables
• Support for short values in data returned.
• Fixes to pass TAPLint tests, notably enclosing many identifiers in quotes.

Release 5.3

• The VOTable writing functions were updated to eliminate frequent flushes of the I/O buffers that substantially slowed down performance. This could be a factor of several for large VOTable queries.
• Most non-table metadata is now cached for 10 minutes. This can substantially improve performance when a sequence of calls is made in rapid sucession (~50%)
• Old feedback is not loaded unless the user requests it which may slightly improve the time it takes for the Xamin form to start up.
• New batch regression tests added to emulate discovery queries

Release 5.2

• Updates to DataLink and TAP functionality include DataLink VOSI elements and corrections to DataLink format per validation outputs.

Release 5.1

• Reinstated Save To Hera functionality
• Fix to directory data products

Release 5.0b

• Fix to compound data products

Release 5.0a

• Fix for table uploads
• Updates to DataLink

Release 5.0

• Addition of AladinLite Mapper window and mapper functionality
• Links between table display and mapper for selected objects/catalog
• New DataLink capabilities
• SSA updates
• Removal of link to old version
• Change to default background
• Tabbed result windows.

Release 4.2

• Changes to support SSA data products for Chandra and INTEGRAL
• Updates to Hera support

Release 4.1a

• Fix parameters checking for external tables to use display table name

Release 4.1

• Restructuring of servlets to use implementation classes.
• Security fixes.
• Bibcode typo correction to security fix in 4.0a

Release 4.0b

• Security Fixes

Release 4.0a

• Patch to fix index queries with fewer than 5 tables
• Turn off memory checks which seem to be triggering frequently

Release 4.0

• Complete revision of plotting capabilities using Plotly package.
• Interactive plots
• Transition from ExtJS 5.0 to ExtJS 6.2
• Support for combined/compressed JavaScript libraries
• Update or Feedback functionality
• Use only relative URLs for cross-Xamin references enabling more code portability
• Fixes to single box interface
• Apply cookie parameters properly
• Fix XSS issue (already applied as patch)
• Limit synchronous loading of modules.
• Try to limit out of memory conditions
• Track number of requests running
• Added general capability for support of SSA data products associated with Xamin products. The structure of the product metadata files has been changed and the ProductsParser class has been updated. New virtual tables in the in database metadata have been defined including ssacolumn, ssalink and ssalinkfield. Users can specify either a known SSA product or just a table and will get the SSA products linked to that table. This initial release defines SSA products for the Swift XRT (for table swiftxrlog). Additional SSA products can be created purely by adding additional metadata.
• SSA and SIA are combined into one SxA entry point in the Xamin servlet.

Release 3.5

• Full support for product parser.
• Support for defining base and supplemental bibcode tables.
• Support for SIA macros in data product files.

Release 3.4

• Fixes for VOTable and ConeSearch format.
• Disable products for group and distinct searches
• ROSAT SIA products

Release 3.3

• Updates to support new data products in HDB code.
• New data product definitions for Chandra.

Release 3.2

• Fail quickly when illegal inputs are detected.
• Don't print out documentation to log

Release 3.1

• Include SAMP interface tool on result windows.
• Eliminate debugging outputs
• Fix some 'require:' fields in ExtJS classes.
• Use compressed ExtJS files.
• Fix bug in query notes return (? tool) (and add tests for this)
• Further updates to WebUsersGuide.

Version 3.0

       http://heasarc.gsfc.nasa.gov/xamin/xamin.jsp
        

The Application Console

Start Button

Session configuration

Output formats

Theme option removed

Options

Query start buttons

Index and table query buttons switched

Similarly, the there are now only three buttons to submit queries of tables. You use the same button to start a single table query or a correlation. You pick the button based upon whether you wish to see the results as a table, do a plot or save the results in your Xamin environment.

Bibcodes

File uploads

Virtual Observatory Tables

Keyword search results

Keyword search results queries

Selecting parameters

Printouts

Feedback

No prompt on exit

Multiple plot control windows

Single box notice

Tutorial movies

Release 2.11

• Added general capability for support of SIA (and eventually) SSA data products associated with Xamin products. The structure of the product metadata files has been changed and the ProductsParser class has been updated. New virtual tables in the in database metadata have been defined including siacolumn, sialink and sialinkfield. A new sia entry point has been added to the Xamin servlet. Users can specify either a known SIA product or just a table and will get the sia products linked to that table. Issues with multiple SIA products for a single table may still need to be addressed. This initial release defines SIA products for the Swift UVOT (for table swiftuvlog). Additional SIA products can be created purely by adding additional metadata.

Release 2.10

• Added capability to upload tables to use directly in queries.
• Transferred positional constraints to all tables in a cross-correlation to ensure that table initially queried is appropriately indexed.
• Expand support for region's in ADQL.ls -
• Quote "as" names to make GAVO happier.
• Support boolean fields.
• Fix JSON output formats to properly use double quotes
• Fix to anticorrelations
• Fix to data products with multiple roots showing up in ROSAT products

Release 2.9

• Fixed handling of large radius searches
• Fixed handling of TAP queries that have immediate errors
• Fixed definition of TAP tap_schema.columns

Release 2.8

• Added uploads to Table Explorer
• Added Inline output format for programmatic access to tables
• Added test HeasarcStarTable class for possible integration of STILTS capabilities
• Fixed bug where integer dates were not being converted to MJD.

Release 2.7

• Fix creation of indices for user tables.
• Show parameters when using user tables.

Release 2.6a

• Updated JSqlParser to version 0.9.2. This fixes a bug when we do an anticorrelation involving three or more total tables. The only modification we make to this is to allow the Posgres ILIKE operator. We not longer put in the change that allowed logical operations in the select list. E.g.,

                   select ra,dec,ra>dec,constell from messier
                  

  used to be supported. This should now be coded as

                   select ra,dec,case when ra>dec true else false end,constell from messier
                  

  Several other changes that we used to make have been incorporated into the standard JSqlParser release.
• New data products for Swift bursts and Fermi LAT were made available earlier outside the release cycle.

Release 2.6

• Added bibcode keyword for batch interface to allow users to do searches based upon bibcodes.
• Fixed issues which required Xamin code to do metadata queries multiple times when making simple queries.
• Small security changes

Release 2.5a

• Changes to add Xamin Plot capability to Browse

Release 2.5

• Adding dynamic TAP metadata (TAP_SCHEMA tables).
• Many improvements to TAP Async processing
• Fix security and XSS issues

Release 2.4c

• Allow users to log in directly to Hera from Xamin if they are already logged in.
• Adding streaming capabilities for BINARY VOTable outputs which are used in TAP queries.
• When there are multiple limits on the number of rows, use the smallest.
• Added commits in asynchronous query updates of query status. Fixed bug where query status remained at EXECUTING for TAP Async queries.

Release 2.4b

• Added needed commits for table appends and deletes.

Release 2.4a

• Fixed autocommit settings broken in 2.4 so that autocommit is set before we add metadata when saving tables.

Release 2.4

• Set fetch count in queries to limit memory usage and speed up initial data return.
• Changes to XML code to use normal IOExceptions rather than customize XMLExceptions. Allow these to propogate normally.
• Handle VizieR results that include multiple tables.
• Data product fixes.
• Fixed handling of multiple logins
• Table updates should now delete removed tables
• Process for saving logs implemented

Release 2.3e

• Fix to transfer to Hera.
• Addition of some bibliographic data products

Release 2.3d

• Software update of underlying javascript framework
• Feedback form enhancement
• Quick Search Form bug fix and enhancement

Xamin release notes

Release 2.3c

• Updated data products for the CMBMASTER - LAMBDA Master Catalog of CMB Experiments table and other tables
• Fixed bug in the grid row sorting process

Release 2.3b

• Fixed bug in the process that parses query parameters
• Feedback now requires approval before showing up in Feedback Report

Release 2.3a

• Fix issue with invalid table names
• Limit visibility of SQL errors

Release 2.3

• Rearranged Xamin tables tree in Query pane to highlight popular choices and adjusted expand/collapse behavior
• Added a bit of guidance to the Submit Options area of the Query Pane

Release 2.2b

• Change how new accounts are created. (Not visible to users)
• Make offset default sorting

Release 2.2a

• Fix bug in display of fractional seconds in UTC times

Release 2.2

• Resizable SkyView preview images
• Change no results to a non-error
• Standardize installation script
• Fix issues in data products
• Communications with Hera improved

Release 2.1

• Refactored positional constraints to allow more ways to do this, including using the Q3C library in Postgres
• Better support for differencing queries including more options when doing a query save
• Fixes to occasional concurrency bugs

Release 2.0

• Save To Hera functionality added to Products Cart
• Connect To Hera option added as menu item
• Created form for Single Box query for to be used on HEASARC Archive page
• Small changes to Products Cart user interface interaction
• Minor bug fixes to quesry results grid

Release 1.7

• The information panel has been moved into an independent pane so that users can easily view documentation from tables.

Release 1.6e

• Fix problem with SAMP requests which were being sent to 'localhost' rather than correct address

Release 1.6d

• The table monitoring capability was not working due to a conflict between the SQL security checks and the way the table creation SQL was stored.
• Occasionally the table check schedule parameters were being encoded when sent to the database. The reason for this is unknown, but a check for encoded values was added.
• Ability to set location of settings file via environment variable restored.

Release 1.6c

• Allow positions to be separated by commas if the names all contain alphabetic characters.
• Do some decoding of Servlet inputs which seemed not be be handled by system utilities.
• Add offset column to cross-correlations. Fix error that added offset columns to discovery queries (after offsets were being shown for multitable queries).

Release 1.6b

• Added Batch mode capabilities to partially emulate Browse batch mode requests.
• Included FITS library in WAR so that FITS downloads should again work.

Release 1.6

• Xamin notices added at both startup and in Help menu
• Xamin blog added to Help menu
• SAMP requests now return an error message when the connection to the server fails.
• HTTP header added to give applications a sense of the current load.
• Fixes to data products
• Changed title bars for Parameters explorer and Tables explorer. These are referenced in the documentation but were not labeled as such in the interface.
• Cone search requests now do a bit more customization of the XML output to match the cone search standard. Some changes are also made to make Xamin cone search requests look more like Browse.

Release 1.5e

• Improved cart icon processing

Release 1.5d

• Improved messages for transferring products to cart
• Fixed adding highlighted rows of a resorted table

Release 1.5c

• Improved handling of data products.
• Cross-correlation documentation improvements.

Release 1.5b

• Changes to SQL syntax used to save query tables.
• Security updates and enchancements
• Fixes to POSTs
• Better rendering of error messages with embedded new lines (including encoding newlines in XML values

Release 1.5a

• Support for session timeouts.
• Changes to query validation using function whitelist.

Release 1.5

• Users can now request that text outputs be downloaded rather than displayed on the screen.
• Some support for direct SQL queries
• When specific products were requested they could be duplicated in the results if they showed up multiple times in the products tree.
• A number of security enhancements designed to keep users from viewing system tables or functions.
• Updated state initialization of special constraints. A bug had the effect of keeping some initial queries from running to completion.

Release 1.4

• Saveas now allows an existing user table to be specified. If the new result has the same columns as the existing data the new rows will be appended. It is an error to attempt to append incompatible data.

  This allows users to upload data in chunks, or to concatenate the results from a set of queries.

• Session logins now use the login service to get the session id.
• Fixed bug where run time was not being added to query information when requested.
• Fixed formatting of various status messages that had embedded HTML.

Release 1.3h

• Precision for output coordinates increased to normalize properly with Browse
• CoordPrecision settings was being ignored.
• Additional security fix for TAR downloads.

Release 1.3g

• Additional Async fixes include issues with TAP Schema

Release 1.3f

• TAP Async fixes

Release 1.3e

• Removal of extra space in XML outputs.
• Fixes to TAP metadata queries and handling of TAP versioning.

Release 1.3d

• Fixes serious bug where Xamin queries would fail due to ingest of <base> attribute in column heading documentation.
• Fixes some problems with cone search.

Release 1.3c

• Check against XSS vulnerabilities
• Give sensible error messages after a time out.
• When a grid is re-ordered the data products were not moving with the rows. This serious bug is fixed. Noted by user.

Release 1.3b

• Updates to Web documentation for single box interface.
• Many small fixes mostly addressing issues with remote tables, e.g.,
  • Making TAP table names consistent which ensured that when correlating tables from each table were not intermingled gratuitously.
  • Filtering Vizier documentation so that on-click actions do not destroy the Xamin form.
  • Making sure that we always use the same column names in Vizier tables to get the RA and Dec.
• Making sure we use Pane/Window distinction consistently.

Release 1.3a

• Changed handling of Vizier resources to enable proper querying of resources that contain multiple tables.
• Upgrade to ExtJS 3.4
• Added Selenium testing (as separate project)
• Fixed several issues with SSA and TAP queries, including data product for SSA, reserved words for TAP queries, and handling of positional columns in TAP queries by ensuring that we use RA/Dec fields rather than vector fields for remote positional constraints.

Release 1.3

• Fixed problem with plots with literal string qualifiers. Identified in request
• Add error manager and provide more consistent handling of errors.
• Upgrades to login procedures to accommodate more rigorous checking of requests.
• Changed autoquoting to show quotes in text and allow user to explicitly delete quotes.
• Discovery queries now allow clicking anywhere on row (except arrow) to initiate query of selected table.
• Fully validate all user requests from both web and CLI environment
• Fixes to downloads in VOTable and non-browser renderable formats.
• Numerous fixes to data product definitions. Added check to ensure that all data products are found somewhere.
• Support for SAMP publication of tables.
• Add checking in some Xamin prompts (e.g., that user specifies a name for a saved session).
• Changes to formatting of table metadata query results.
• Use tabledocs table to get HEASARC and Vizier table documentation. Ensure that documentation links work, but do not exit Xamin.
• Use tabledocs and table_classes tables to defined VizieR table structure. Added new node classes to heasarc.xamin.nodes.
• Ensure saved sessions have different names.
• Allow user table selection from account page.
• Fix one/two column CSV uploads. Also ensure that user local file name is used in deciding upload format.
• Provide unit information in column metadata.
• Upgrade to ExtJS 3.4 (from 3.2). [Patch included in xamin.jsp to accommodate unmovable windows bug in 3.4].

Release 1.2

• Users can now get information on fields by mousing over grid headers.
• Users can now get all columns in table for a given row by mousing over first number field in grid.
• Fixes to CSV processing including supporting of embedded commas and newlines plus ensuring that valid column names are used.
• Use startfv.pl command to download single FITS files to ensure that proper mime-type is received by browser.
• Updates to documentation and JS documentation helpers.
• Added plot document
• Invalidate queries when a table used is no longer available due to logout or deletion.
• Stream VOTable output format.
• Make plotting more efficient. Return on a single row for the initial plot query which is used to find the available fields. The streaming VOTable output allows STILTS to read data while query is still running.
• Fix bugs in handling of data product trees.

Release 1.1

• New login framework
• Many bug fixes from release 1.0.

Release 1.0

• Initial release of full system.