23.10 Release Notes
- 1 Known Issues, Limitations, Behavior Changes
- 2 Patch History
- 3 Release Notes for SciDB 23.10.0
- 4 Supported Operating Systems
- 5 SciDB Features and Changes
- 5.1 HTTP API
- 5.2 Log file rotation behavior and new log file location
- 5.3 The builtin_equi_join operator
- 5.4 The builtin_grouped_aggregate operator
- 5.5 Support for subarray operator
- 5.6 The subdelete operator
- 5.7 Deprecation of mquery() operator
- 5.8 Added dimension keyword to some operators
- 5.9 Improvement to the remove_versions() operator
- 5.10 Low disk space warnings
- 5.11 Discontinued the de_rle plugin
- 6 Performance enhancements and bug fixes
Known Issues, Limitations, Behavior Changes
Disable pushdown optimization
Versions affected: SciDB 23.10.0–4
Pushdown optimization should be manually disabled in versions 23.10.0 through 23.10.4. If left on, it can lead to various issues including server crashes and high memory use.
To disable it manually, edit the SciDB config file to include the line:
enable-optimize-pushdown=0
It is disabled by default in SciDB 23.10.5.
Avoid curl 8.6 in HTTP API clients
Versions affected: SciDB 21.x–
There is a bug in version 8.6 of the “curl” library that causes HTTP clients to fail with this error when connecting to SciDB’s Client API over HTTPS:
curl: (56) OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 0
(See SDB-8459 for details.)
A fix for this bug was merged into the curl source code on GitHub within days of the ship date of curl 8.6; however, it hasn’t been included in a curl release so far (as of 2023-03-21). The fix should be included in curl 8.7 whenever that gets released.
If you have a client using curl 8.6, you should upgrade to curl 8.7 if available; if not, downgrade to curl 8.5.
You should instruct package managers and dependency frameworks (aptitude, conda, dnf, yum, etc.) to exclude version 8.6 when installing curl/libcurl.
Patch History
Patch notes for SciDB 23.10.8
Release date: July 18, 2024
SciDB SHA: 11b236b
Changes include:
Fixed an issue where the SciDB Client API could hang when too many parallel uploads were attempted.
https://paradigm4.atlassian.net/browse/SDB-8488
Please note that there is a companion fix in SciDBR version 4.2.0 which detects the “Too Many Requests” response issued by this fix and retries the upload.Fixed an issue where the SciDB Client API could sometimes encounter an assertion when canceling an upload query.
https://paradigm4.atlassian.net/browse/SDB-8519Adjusted the tokens used for the SciDB Client API’s auth cookies so that they now use a standard name for the signing algorithm.
https://paradigm4.atlassian.net/browse/SDB-8520Starting SciDB as a service will no longer erroneously treat config file backups as usable config files.
https://paradigm4.atlassian.net/browse/SDB-8495
Patch notes for SciDB 23.10.7
Release date: June 12, 2024
SciDB SHA: 0769f38
Changes include:
In the SciDB Client API, the setting
http-threads
now defaults to the number of query execution threads.
https://paradigm4.atlassian.net/browse/SDB-8504Enable SciDB to use more than 1024 file descriptors when permitted by the rlimit max.
https://paradigm4.atlassian.net/browse/SDB-8447
Patch notes for SciDB 23.10.6
Release date: April 29, 2024
SciDB SHA: cc615d4
Changes include:
Fixes for the
op_count()
operator in which it can return different results from equivalent calls usingsummarize()
andaggregate()
.
https://paradigm4.atlassian.net/browse/SDB-8463Tweaks to
subarray()
: https://paradigm4.atlassian.net/browse/SDB-8469Empty pick arrays when using
subarray()
will now yield an empty result instead of an error.Also fixed a problem with cases in which
subarray()
is called with an empty pick array whileinverse:true
is set.Please note that
subdelete()
will still maintain the same behavior as before.
Eliminated an unnecessary use of a file descriptor.
https://paradigm4.atlassian.net/browse/SDB-8447Fixed an issue when using
filter(, is_null(attr))
with empty chunks.
https://paradigm4.atlassian.net/browse/SDB-8474Fixed a race condition in the SciDB Client API which could lead to a crash when canceling queries under high query load.
https://paradigm4.atlassian.net/browse/SDB-8435The default log rotation frequency has been changed from hourly to daily. This will only affect new SciDB deployments.
https://paradigm4.atlassian.net/browse/SDB-8416
Patch notes for SciDB 23.10.5
Release date: March 27, 2024
SciDB SHA: 56b2ad5
Changes include:
Mitigation to avoid potential crashes in RocksDB when initially storing data into an array, and the addition of diagnostic tools for RocksDB, specifically tailored to its use in SciDB.
https://paradigm4.atlassian.net/browse/SDB-8451Addition of a new config option
http-max-json-bytes
to the Client API for the maximum JSON payload length that the server will accept from a client.
https://paradigm4.atlassian.net/browse/SDB-8457Pushdown optimization now defaults to OFF in new installations. If the SciDB config on an existing installation has
enable-pushdown-optimization=1
, this will not change the setting that is currently in use.
https://paradigm4.atlassian.net/browse/SDB-8453Delivering new scripts for help in diagnosing and debugging “too many open files” errors.
https://paradigm4.atlassian.net/browse/SDB-8447
Patch notes for SciDB 23.10.4
Release date: March 6, 2024
SciDB SHA: bf30ffc
Fixes include:
Additional fixes for memory and index management:
Support for excluding arrays matching a given regex when running the
scidb_backup.py
utility
https://paradigm4.atlassian.net/browse/SDB-8425Additional logging for the Client API when a client and server get out of sync while resuming an interrupted query
https://paradigm4.atlassian.net/browse/SDB-8436
Patch notes for SciDB 23.10.3
Release date: February 14, 2024
SciDB SHA: 1a41e7e
Major changes:
https://paradigm4.atlassian.net/browse/SDB-8360
The
namespaces
library is always loaded by default.load_library(