These release notes apply to version 21.8 of SciDB; they cover all features and changes since version 20.10.

Supported Operating Systems

SciDB 21.8 supports the following platforms:

• RHEL 7

• CentOS 7

Support for Ubuntu 16.04 has been discontinued in this release.

SciDB Features and Changes

Transactional Support

• SciDB transactions overcome the limitations of mquery() by empowering users to:

  1. Create a transaction apart from any query

  2. Arbitrarily roll back (abort) a transaction

  3. Commit the transaction explicitly

• New transaction operators include begin(), commit(), and rollback().

• See https://paradigm4.atlassian.net/wiki/spaces/scidb/pages/2828829542 for details.

Operators

• hash() computes a distribution-independent parallelized hash signature of a SciDB array.

• secure_scan() is now part of the SciDB core engine as of 21.8.1. Refer to the Administration Guide for how to configure it. Important: Before upgrading, you must run unload_library('secure_scan') or the cluster will not start. See details under Limitations below.

Performance and Resource Usage Enhancements

• SciDB uses significantly less memory than 20.11 when processing large chunks with redimension(), redistribute(), repart(), and reshape(), or any other operator or query that causes a scatter gather of data.  These enhancements especially benefit queries involving chunk data greater than 100 MiB.

• SciDB more frequently returns reserved memory to the operating system, allowing instances to more efficiently allocate memory between each other.  This enhancement especially benefits asymmetric load and save queries without having to alter SciDB config variables, and other processes, such as R or Python applications, to use memory that SciDB is not.

Beta Features

SciDB has these new features available for review and feedback.

• subarray() has been reimplemented to select elements of an input array by specifying indices in a variety of ways, implicitly and explicitly.

• REST API - SciDB now has an experimental HTTPS interface supporting secure connections and allowing full querying and data transfer support. Contact Paradigm4 for documentation.

• dscheck.py - A program that does rudimentary integrity checks on datastore files (21.8.3 and later).

Known Issues, Limitations, Behavior Changes

• subarray() usage has completely changed - see Beta section above. That behavior was deprecated in release 20.10.

• Upgrade from 19.11 and 20.10 requires no changes to the database.

• On clusters that used the secure_scan plugin, you must run unload_library('secure_scan') prior to upgrading to 21.8.1 or later. If you fail to do so, the cluster will not start, and data directories will fill up with core files until all SciDB processes have been killed. If you get into this situation, the only remedy is:

  • Kill all SciDB instances manually.

  • Remove core files from data directories.

  • Run the psql Postgres client as a privileged user and execute the following SQL command: delete from libraries where name = 'secure_scan'; .

  • You should now be able to start the cluster. Best to remove or rename the lib/scidb/plugins/libsecure_scan.so file from the installation directory.

Paradigm4 Labs Features and Changes

•  Paradigm4 provides a host of external tools and add-ons in the form of GitHub repositories at http://github.com/paradigm4. Please contact us for guidance on using these.