I'm coding a Merb application which uses a combination of SimpleDB and Tokyo Tyrant for storage. For both of these data stores I'm implementing IN (list) type-functionality by spinning up a thread for each value in list and then merging the result sets. Bearing in mind that this is a web application, is there a limit to the number of threads I should be creating? Ruby 1.8.7, so they're not kernel threads.

Sorry, to clarify, these threads are just making service calls to the SimpleDB or Tokyo Tyrant servers, and not doing very much processing themselves. I was under the impression that MRI threads were OK for that?
–
Mark RendleDec 14 '09 at 17:32

3 Answers
3

Threads seems like a bad approach for what you're trying to do here, and if you can't use JRuby, I'd just drop the threads altogether. However, you could create a ruby file loading the database and use the benchmark library to do some benchmarking on which number is the fastest. You probably want to look at the memory used too.

If you are using MRI then using threads in such cases won't be of a big help as MRI uses green threads that are not helpful when it comes to computational operations. I believe using JRuby(native threads) will be helpful then.
I keep hearing that for native threads it's best to use (number of cores + 1) to make use of the available cores.