The stats_query() operator provides per-instance per-query resource statistics. Available only in the Enterprise Edition.

Synopsis

stats_query()

Library

The stats_query() operator resides in the monitor library. Run the following query to load this library:

  iquery  -aq "load_library('monitor');"

The stats_query() operator lets you gather per-query SciDB resource statistics for diagnostic purposes.

Summary

Inputs

Attributes

The following attributes reveal resources utilized on behalf of queries.

Attribute	Description
query_id	The query identifier
time_since_start_ms	Amount of time that has elapsed, in milliseconds, since this query started
time_since_reset_ms	Amount of time that has elapsed, in milliseconds, since this query was reset
time_active_since_start_ms	Amount of time that has elapsed, in milliseconds, since this query started in which the query was active
time_active_since_reset_ms	Amount of time that has elapsed, in milliseconds, since this query was reset in which the query was active
bytes_net_send	Size of chunks sent across the network on behalf of this query in bytes
bytes_net_recv	Size of chunks received on behalf of this query from the network in bytes
bytes_mem_available	Total size of all RAM currently available to this query in bytes
bytes_mem_allocated	Total size of all RAM currently allocated by this query in bytes
bytes_mem_peak_usage	Maximum amount of RAM allocated by this query in bytes
mem_allocations	Number of allocations made by this query
cache_used_mem_size	Total size of all mem chunks currently in cache on behalf of this query in bytes
cache_swap_bytes	Size of mem chunks written to disk on behalf of this query (cache swap out) in bytes
cache_swap_num	Number of mem chuncks written to disk on behalf of this query
cache_load_bytes	Size of mem chunks read off disk (due to read cache miss) on behalf of this query in bytes
cache_load_num	Number of mem chunks read off disk on behalf of this query
cache_drop_bytes	Size of mem chunks dropped from cache on behalf of this query in bytes
cache_drop_num	Number of mem chunks dropped from cache on behalf of this query
time_cache_write_ms	Time that this query spent waiting on Mem chunk writes in milliseconds
time_cache_read_ms	Time that this query spent waiting on Mem chunk reads in milliseconds
num_db_wr	Number of DB Chunks written to disk on behalf of this query
bytes_compr_db_wr	Total size of all compressed DB Chunks written to disk on behalf of this query in bytes
bytes_decomp_db_wr	Total size of all decompressed DB Chunks written to disk on behalf of this query in bytes
num_db_rd	Number of DB Chunks read from disk on behalf of this query
bytes_compr_db_rd	Total size of all compressed DB Chunks read from disk on behalf of this query in bytes
bytes_decomp_db_rd	Total size of all decompressed DB Chunks read from disk on behalf of this query in bytes
time_catalog_access_ms	Time spent waiting on the execution of system catalogue queries on behalf of this query in milliseconds
query_status	The completion status of this query. One of the following: "0 - pending", "1 - active", "2 - completed", "3 - errors"
query_str	The string that instantiated this query.

Some attributes (counters) are disabled if the resource-monitoring variable set to False in the config.ini. The default value for this variable is False if not specified in the config.ini

Query history is disabled if the stats-query-history-size variable is set to zero in the config.ini. The default value for this variable is 32 if not specified in the config.ini

Example

Ensure that the resource-monitoring variable is set to True in the config.ini

Ensure that the stats-query-history-size variable is set to 32 in the config.ini

The following example assumes that the three iqueries were executed just after SciDB was started and just prior to the stats_query() being invoked:

iquery -naq "list('arrays');"

iquery -naq "list('operators');"

iquery -naq "list('queries');"

This is not likely to be the case on your system and is used here just to show that the queries executed prior to stats_query() show up as part of the query history as displayed by the query_str attribute.

Full stats_query() statistics

$ iquery -aq "stats_query();"
{inst,n} query_id,time_since_start_ms,time_since_reset_ms,time_active_since_start_ms,time_active_since_reset_ms,bytes_net_send,bytes_net_recv,bytes_mem_available,bytes_mem_allocated,bytes_mem_peak_usage,mem_allocations,cache_used_mem_size,cache_swap_bytes,cache_swap_num,cache_load_bytes,cache_load_num,cache_drop_bytes,cache_drop_num,time_cache_write_ms,time_cache_read_ms,num_db_wr,bytes_compr_db_wr,bytes_decomp_db_wr,num_db_rd,bytes_compr_db_rd,bytes_decomp_db_rd,time_catalog_access_ms,query_status,query_str
{0,0} '0.1466691475073606160',2,2,0,0,364,402,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'1 - active','stats_query()'
{0,1} '0.1466691457010605345',18065,18065,2,2,326,619,1152921504606846975,0,0,0,714,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','list(\'queries\')'
{0,2} '0.1466691465436203221',9639,9639,2,2,326,597,1152921504606846975,0,0,0,384,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,'2 - completed','list(\'arrays\')'
{0,3} '0.1466691465461545544',9614,9614,2,2,326,603,1152921504606846975,67117080,67117080,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','list(\'operators\')'
{0,4} '0.1466691466324962764',8750,8750,2,2,326,599,1152921504606846975,0,0,0,714,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','list(\'queries\')'
{1,0} '0.1466691475073606160',1,1,0,0,48,86,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'1 - active',''
{1,1} '0.1466691457010605345',18064,18064,1,1,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{1,2} '0.1466691465436203221',9638,9638,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{1,3} '0.1466691465461545544',9613,9613,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{1,4} '0.1466691466324962764',8750,8750,0,0,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{2,0} '0.1466691475073606160',1,1,0,0,48,86,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'1 - active',''
{2,1} '0.1466691457010605345',18064,18064,1,1,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{2,2} '0.1466691465436203221',9638,9638,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{2,3} '0.1466691465461545544',9613,9613,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{2,4} '0.1466691466324962764',8750,8750,0,0,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{3,0} '0.1466691475073606160',1,1,0,0,48,86,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'1 - active',''
{3,1} '0.1466691457010605345',18064,18064,1,1,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{3,2} '0.1466691465436203221',9638,9638,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{3,3} '0.1466691465461545544',9613,9613,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''
{3,4} '0.1466691466324962764',8750,8750,1,1,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''

A sub-set of stats_instance() statistics

$ iquery -aq "project(stats_query(), query_id,bytes_mem_available,bytes_mem_allocated,bytes_mem_peak_usage,mem_allocations,query_status,query_str);"
{inst,n} query_id,bytes_mem_available,bytes_mem_allocated,bytes_mem_peak_usage,mem_allocations,query_status,query_str
{0,0} '0.1466691804111660403',1152921504606846975,0,0,0,'1 - active','project(stats_query(), query_id,bytes_mem_available,bytes_mem_allocated,bytes_mem_peak_usage,mem_allocations,query_status,query_str)'
{0,1} '0.1466691747803255791',1152921504606846975,0,0,0,'2 - completed','list(\'queries\')'
{0,2} '0.1466691750113435610',1152921504606846975,0,0,0,'2 - completed','list(\'arrays\')'
{0,3} '0.1466691752234748447',1152921504606846975,67117080,67117080,1,'2 - completed','list(\'operators\')'
{0,4} '0.1466691759880452211',1152921504606846975,0,0,0,'2 - completed','list(\'queries\')'
{1,0} '0.1466691804111660403',1152921504606846975,0,0,0,'1 - active',''
{1,1} '0.1466691747803255791',1152921504606846975,0,0,0,'2 - completed',''
{1,2} '0.1466691750113435610',1152921504606846975,0,0,0,'2 - completed',''
{1,3} '0.1466691752234748447',1152921504606846975,0,0,0,'2 - completed',''
{1,4} '0.1466691759880452211',1152921504606846975,0,0,0,'2 - completed',''
{2,0} '0.1466691804111660403',1152921504606846975,0,0,0,'1 - active',''
{2,1} '0.1466691747803255791',1152921504606846975,0,0,0,'2 - completed',''
{2,2} '0.1466691750113435610',1152921504606846975,0,0,0,'2 - completed',''
{2,3} '0.1466691752234748447',1152921504606846975,0,0,0,'2 - completed',''
{2,4} '0.1466691759880452211',1152921504606846975,0,0,0,'2 - completed',''
{3,0} '0.1466691804111660403',1152921504606846975,0,0,0,'1 - active',''
{3,1} '0.1466691747803255791',1152921504606846975,0,0,0,'2 - completed',''
{3,2} '0.1466691750113435610',1152921504606846975,0,0,0,'2 - completed',''
{3,3} '0.1466691752234748447',1152921504606846975,0,0,0,'2 - completed',''
{3,4} '0.1466691759880452211',1152921504606846975,0,0,0,'2 - completed',''

Individual stats_query() Statistic

$ iquery -aq "project(stats_query(), query_str);"
{inst,n} query_str
{0,0} 'project(stats_query(), query_str)'
{0,1} 'list(\'queries\')'
{0,2} 'list(\'arrays\')'
{0,3} 'list(\'operators\')'
{0,4} 'list(\'queries\')'
{1,0} ''
{1,1} ''
{1,2} ''
{1,3} ''
{1,4} ''
{2,0} ''
{2,1} ''
{2,2} ''
{2,3} ''
{2,4} ''
{3,0} ''
{3,1} ''
{3,2} ''
{3,3} ''
{3,4} ''