I am running scidb under heavy load and once in a while i am checking if my query still running as there is no entry for 24h in scidb.log
however when i run iquery -a -f “list(‘queries’)” i get this
SystemException in file: src/query/QueryProcessor.cpp function: validateQueryWithTimeout line: 318
Error id: scidb::SCIDB_SE_EXECUTION::SCIDB_LE_RESOURCE_BUSY
Error description: Error during query execution. Not enough resources: remote query processor not ready to execute, try again later. Try again…

my disks are under heavy load but scidb itself is running on the nodes which have plenty of ram.
why is this error comes up?

SciDB creates a number of threads to run queries. When a new query comes in, we wait for a thread to become available. If a thread is not available in deadlock-timeout seconds, then we throw this error.

Supposing you want to run N queries concurrently, for most applications, we recommend these settings in config.ini:

Note: these are not the defaults right now. So we recommend stating these in the config.ini explicitly. More threads use more memory and some malloc allocations “stick” until the thread is reused (thanks, malloc!). So we don’t recommend going crazy - set N to 4 or 6 unless you have very high concurrency.

Also, you can set deadlock-timeout in the config file to wait longer; specified in seconds. Default is

deadlock-timeout=30

In 15.12 there could be a deadlock situation if you are sending queries to multiple different instances (i.e. multiple coordinator / load balancing scenario). If there’s one thread left, then this could occur:

Well… obviously everyone should talk to us and get a Commercial or Academic EE license. How else could I respond to that question?

Depending on your situation, you might find a way to use Linux mechanisms to detect when they stop or restart… It might be challenging on a multinode cluster. Naturally, whenever we need to solve this problem, we use EE, so I honestly haven’t given much thought to alternatives…