Memory Usage

(SDB-7510 hypotheses. Need validation.)

• values from /proc/<pid>/status (or top)

• variables from SciDB config.ini:

  • mem-array-threshold

  • merge-sort-buffer

  • sg-send-queue-size

  • sg-recv-queue-size

scidb 19.x

• global_arena_holdover = excessive (lea arena behavior)

• transaction_arena_holdover = excessive (lea arena behavior)

• receive_merge_factor = 3

scidb 20.x

• global_arena_holdover = small (limited arena behavior)

• transaction_arena_holdover = excessive (lea arena behavior)

• receive_merge_factor = 3

scidb 21.x

• global_arena_holdover = small (limited arena behavior)

• transaction_arena_holdover = small (limited arena behavior)

• receive_merge_factor = 2

all versions

• rocksdb_overhead = unknown

• sg_send_buffer = SUM of (sg-arrays * sg-send-queue-size) chunk bytes at head of send queues

• sg_recv_buffer = SUM of (sg-arrays * sg-recv-queue-size) chunks bytes at head of receive queue

scidb_data_bytes = SUM OF

• mem-array-threshold

• global_arena_holdover

• transaction_arena_holdover

• rocksdb_overhead

• merge-sort-buffer * num queries doing {redimension, sort}

• sg_send_buffer

• sg_recv_buffer

• recv_merge_factor * sg_recv_buffer