/
stats_query()

stats_query()

Feb 16, 2021

The stats_query() operator provides per-instance per-query resource statistics. Available only in the Enterprise Edition.

Synopsis

stats_query()

Library

The stats_query() operator resides in the monitor library. Run the following query to load this library:

 

iquery -aq "load_library('monitor');"  

The stats_query() operator lets you gather per-query SciDB resource statistics for diagnostic purposes.

Summary


Inputs

Attributes

The following attributes reveal resources utilized on behalf of queries.

Attribute

Description

Attribute

Description

query_id

The query identifier

time_since_start_ms

Amount of time that has elapsed, in milliseconds, since this query started

time_since_reset_ms

Amount of time that has elapsed, in milliseconds, since this query was reset

time_active_since_start_ms

Amount of time that has elapsed, in milliseconds, since this query started in which the query was active

time_active_since_reset_ms

Amount of time that has elapsed, in milliseconds, since this query was reset in which the query was active

bytes_net_send

Size of chunks sent across the network on behalf of this query in bytes

bytes_net_recv

Size of chunks received on behalf of this query from the network in bytes

bytes_mem_available

Total size of all RAM currently available to this query in bytes

bytes_mem_allocated

Total size of all RAM currently allocated by this query in bytes

bytes_mem_peak_usage

Maximum amount of RAM allocated by this query in bytes

mem_allocations

Number of allocations made by this query

cache_used_mem_size

Total size of all mem chunks currently in cache on behalf of this query in bytes

cache_swap_bytes

Size of mem chunks written to disk on behalf of this query (cache swap out) in bytes

cache_swap_num

Number of mem chuncks written to disk on behalf of this query

cache_load_bytes

Size of mem chunks read off disk (due to read cache miss) on behalf of this query in bytes

cache_load_num

Number of mem chunks read off disk on behalf of this query

cache_drop_bytes

Size of mem chunks dropped from cache on behalf of this query in bytes

cache_drop_num

Number of mem chunks dropped from cache on behalf of this query

time_cache_write_ms

Time that this query spent waiting on Mem chunk writes in milliseconds

time_cache_read_ms

Time that this query spent waiting on Mem chunk reads in milliseconds

num_db_wr

Number of DB Chunks written to disk on behalf of this query

bytes_compr_db_wr

Total size of all compressed DB Chunks written to disk on behalf of this query in bytes

bytes_decomp_db_wr

Total size of all decompressed DB Chunks written to disk on behalf of this query in bytes

num_db_rd

Number of DB Chunks read from disk on behalf of this query

bytes_compr_db_rd

Total size of all compressed DB Chunks read from disk on behalf of this query in bytes

bytes_decomp_db_rd

Total size of all decompressed DB Chunks read from disk on behalf of this query in bytes

time_catalog_access_ms

Time spent waiting on the execution of system catalogue queries on behalf of this query in milliseconds

query_status

The completion status of this query.  One of the following:

  "0 - pending", "1 - active", "2 - completed", "3 - errors"

query_str

The string that instantiated this query.

Some attributes (counters) are disabled if the resource-monitoring variable set to False in the config.ini. The default value for this variable is False if not specified in the config.ini

Query history is disabled if the stats-query-history-size variable is set to zero in the config.ini. The default value for this variable is 32 if not specified in the config.ini

 

Example

Ensure that the resource-monitoring variable is set to True in the config.ini

Ensure that the stats-query-history-size variable is set to 32 in the config.ini

 

The following example assumes that the three iqueries were executed just after SciDB was started and just prior to the stats_query() being invoked:

 

iquery -naq "list('arrays');"

iquery -naq "list('operators');"

iquery -naq "list('queries');"

 

This is not likely to be the case on your system and is used here just to show that the queries executed prior to stats_query() show up as part of the query history as displayed by the query_str attribute.

 

 

Full stats_query() statistics

$ iquery -aq "stats_query();" {inst,n} query_id,time_since_start_ms,time_since_reset_ms,time_active_since_start_ms,time_active_since_reset_ms,bytes_net_send,bytes_net_recv,bytes_mem_available,bytes_mem_allocated,bytes_mem_peak_usage,mem_allocations,cache_used_mem_size,cache_swap_bytes,cache_swap_num,cache_load_bytes,cache_load_num,cache_drop_bytes,cache_drop_num,time_cache_write_ms,time_cache_read_ms,num_db_wr,bytes_compr_db_wr,bytes_decomp_db_wr,num_db_rd,bytes_compr_db_rd,bytes_decomp_db_rd,time_catalog_access_ms,query_status,query_str {0,0} '0.1466691475073606160',2,2,0,0,364,402,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'1 - active','stats_query()' {0,1} '0.1466691457010605345',18065,18065,2,2,326,619,1152921504606846975,0,0,0,714,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','list(\'queries\')' {0,2} '0.1466691465436203221',9639,9639,2,2,326,597,1152921504606846975,0,0,0,384,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,'2 - completed','list(\'arrays\')' {0,3} '0.1466691465461545544',9614,9614,2,2,326,603,1152921504606846975,67117080,67117080,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','list(\'operators\')' {0,4} '0.1466691466324962764',8750,8750,2,2,326,599,1152921504606846975,0,0,0,714,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','list(\'queries\')' {1,0} '0.1466691475073606160',1,1,0,0,48,86,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'1 - active','' {1,1} '0.1466691457010605345',18064,18064,1,1,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {1,2} '0.1466691465436203221',9638,9638,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {1,3} '0.1466691465461545544',9613,9613,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {1,4} '0.1466691466324962764',8750,8750,0,0,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {2,0} '0.1466691475073606160',1,1,0,0,48,86,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'1 - active','' {2,1} '0.1466691457010605345',18064,18064,1,1,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {2,2} '0.1466691465436203221',9638,9638,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {2,3} '0.1466691465461545544',9613,9613,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {2,4} '0.1466691466324962764',8750,8750,0,0,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {3,0} '0.1466691475073606160',1,1,0,0,48,86,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'1 - active','' {3,1} '0.1466691457010605345',18064,18064,1,1,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {3,2} '0.1466691465436203221',9638,9638,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {3,3} '0.1466691465461545544',9613,9613,0,0,96,172,1152921504606846975,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed','' {3,4} '0.1466691466324962764',8750,8750,1,1,96,172,1152921504606846975,0,0,0,699,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,'2 - completed',''


A sub-set of stats_instance() statistics

$ iquery -aq "project(stats_query(), query_id,bytes_mem_available,bytes_mem_allocated,bytes_mem_peak_usage,mem_allocations,query_status,query_str);" {inst,n} query_id,bytes_mem_available,bytes_mem_allocated,bytes_mem_peak_usage,mem_allocations,query_status,query_str {0,0} '0.1466691804111660403',1152921504606846975,0,0,0,'1 - active','project(stats_query(), query_id,bytes_mem_available,bytes_mem_allocated,bytes_mem_peak_usage,mem_allocations,query_status,query_str)' {0,1} '0.1466691747803255791',1152921504606846975,0,0,0,'2 - completed','list(\'queries\')' {0,2} '0.1466691750113435610',1152921504606846975,0,0,0,'2 - completed','list(\'arrays\')' {0,3} '0.1466691752234748447',1152921504606846975,67117080,67117080,1,'2 - completed','list(\'operators\')' {0,4} '0.1466691759880452211',1152921504606846975,0,0,0,'2 - completed','list(\'queries\')' {1,0} '0.1466691804111660403',1152921504606846975,0,0,0,'1 - active','' {1,1} '0.1466691747803255791',1152921504606846975,0,0,0,'2 - completed','' {1,2} '0.1466691750113435610',1152921504606846975,0,0,0,'2 - completed','' {1,3} '0.1466691752234748447',1152921504606846975,0,0,0,'2 - completed','' {1,4} '0.1466691759880452211',1152921504606846975,0,0,0,'2 - completed','' {2,0} '0.1466691804111660403',1152921504606846975,0,0,0,'1 - active','' {2,1} '0.1466691747803255791',1152921504606846975,0,0,0,'2 - completed','' {2,2} '0.1466691750113435610',1152921504606846975,0,0,0,'2 - completed','' {2,3} '0.1466691752234748447',1152921504606846975,0,0,0,'2 - completed','' {2,4} '0.1466691759880452211',1152921504606846975,0,0,0,'2 - completed','' {3,0} '0.1466691804111660403',1152921504606846975,0,0,0,'1 - active','' {3,1} '0.1466691747803255791',1152921504606846975,0,0,0,'2 - completed','' {3,2} '0.1466691750113435610',1152921504606846975,0,0,0,'2 - completed','' {3,3} '0.1466691752234748447',1152921504606846975,0,0,0,'2 - completed','' {3,4} '0.1466691759880452211',1152921504606846975,0,0,0,'2 - completed',''

 

Individual stats_query() Statistic

 

$ iquery -aq "project(stats_query(), query_str);" {inst,n} query_str {0,0} 'project(stats_query(), query_str)' {0,1} 'list(\'queries\')' {0,2} 'list(\'arrays\')' {0,3} 'list(\'operators\')' {0,4} 'list(\'queries\')' {1,0} '' {1,1} '' {1,2} '' {1,3} '' {1,4} '' {2,0} '' {2,1} '' {2,2} '' {2,3} '' {2,4} '' {3,0} '' {3,1} '' {3,2} '' {3,3} '' {3,4} ''