Activity

Notes and Questions:
1) Running my test single threaded does not recreate this problem.
2) Running my test multi threaded yields this problem reliably (100% of the time so far).
3) To date I've only used/tested Couchbase via the Ruby client library.
4) How many other developers use the Ruby client library, and of those that use it, are their apps multi-threaded?
5) I noted that the tests for the Ruby client library didn't appear to have any that are multi-threaded.
6) Do any of your other client tests using threading, and attempt to perform set operations simultaneously?
7) Perhaps the 'Component" for this issue be changed to Ruby Client Library; please do so if you believe that is more correct.

Charles Leu
added a comment - 17/Jan/13 1:01 PM Notes and Questions:
1) Running my test single threaded does not recreate this problem.
2) Running my test multi threaded yields this problem reliably (100% of the time so far).
3) To date I've only used/tested Couchbase via the Ruby client library.
4) How many other developers use the Ruby client library, and of those that use it, are their apps multi-threaded?
5) I noted that the tests for the Ruby client library didn't appear to have any that are multi-threaded.
6) Do any of your other client tests using threading, and attempt to perform set operations simultaneously?
7) Perhaps the 'Component" for this issue be changed to Ruby Client Library; please do so if you believe that is more correct.

Matt Ingenthron
added a comment - 17/Jan/13 4:36 PM Valid questions Charles. Let me pass this to our Ruby client lead and have him look it over. We can move it to the RCBC project if it's in fact different.

I've learned that this particular issue is/was due to the sharing of the Couchbase connection amongst threads. When the test app uses thread-local storage to house/use a connection per thread, things work without ado.

Perhaps you want to put logic in place (e.g. a monitor per connection) to ensure connection sharing doesn't lead to a SEGV or other problem (e.g. deadlock). I guess the client could also implement such logic, however using thread-local storage with multiple connections is clearly a better option.

Note that I'm fine with things as they are. However if the client lib had the aforementioned logic in place, it would have saved us some time (i.e. on both my end and yours).

Charles Leu
added a comment - 17/Jan/13 9:52 PM I've learned that this particular issue is/was due to the sharing of the Couchbase connection amongst threads. When the test app uses thread-local storage to house/use a connection per thread, things work without ado.
Perhaps you want to put logic in place (e.g. a monitor per connection) to ensure connection sharing doesn't lead to a SEGV or other problem (e.g. deadlock). I guess the client could also implement such logic, however using thread-local storage with multiple connections is clearly a better option.
Note that I'm fine with things as they are. However if the client lib had the aforementioned logic in place, it would have saved us some time (i.e. on both my end and yours).
Thank you for your consideration and support.

As you pointed out above, the libcouchbase handle itself isn't supposed to be shared between threads, and currently the caller should protect the connection instance either by copying it with Bucket#dup for each thread, which will lead to establishing new connection, or implementing thread-safe wrapper which will make sure that the handle won't be accessed from several threads.

Sergey Avseyev
added a comment - 22/Jan/13 5:53 AM Hi, Charles.
As you pointed out above, the libcouchbase handle itself isn't supposed to be shared between threads, and currently the caller should protect the connection instance either by copying it with Bucket#dup for each thread, which will lead to establishing new connection, or implementing thread-safe wrapper which will make sure that the handle won't be accessed from several threads.

Interesting, just ran into this segfault in the Ruby client 1.2.2. I also have a multi-threaded app where each thread needs to do get and set operations against CB. I get that we shouldn't be sharing the handle between threads, and the Thread.current storage in 1.2.2 does fix this for connections initiated from the Couchbase module.

The packaged CouchbaseStore ActiveSupport::Cache implementation however is not threadsafe in this regard--it creates one connection to the bucket and shares that across any threads, potentially resulting in the segfault.

Mike Evans
added a comment - 21/Mar/13 1:40 PM Interesting, just ran into this segfault in the Ruby client 1.2.2. I also have a multi-threaded app where each thread needs to do get and set operations against CB. I get that we shouldn't be sharing the handle between threads, and the Thread.current storage in 1.2.2 does fix this for connections initiated from the Couchbase module.
The packaged CouchbaseStore ActiveSupport::Cache implementation however is not threadsafe in this regard--it creates one connection to the bucket and shares that across any threads, potentially resulting in the segfault.
The fix is rudimentary, so look for a pull request shortly.

I'm wondering if it wouldn't be helpful in some circumstances to have the option of an imposed mutex/monitor instead of a thread local variable in the Couchbase module itself. Thinking about my own app-dozens of Couchbase Models, each with their own Bucket connection-then potentially dozens of threads. Seems like that would be spinning up a lot of connections to the server per-process.

Mike Evans
added a comment - 21/Mar/13 2:35 PM Just pushed:
http://review.couchbase.org/#/c/25294/
I'm wondering if it wouldn't be helpful in some circumstances to have the option of an imposed mutex/monitor instead of a thread local variable in the Couchbase module itself. Thinking about my own app- dozens of Couchbase Models, each with their own Bucket connection -then potentially dozens of threads. Seems like that would be spinning up a lot of connections to the server per-process.

Yeah, I'm planning to implement pooling there, to allow to re-use and share connections if it is possible, but once you will store docs in different buckets, it will require separate connection per bucket

Sergey Avseyev
added a comment - 21/Mar/13 3:08 PM Yeah, I'm planning to implement pooling there, to allow to re-use and share connections if it is possible, but once you will store docs in different buckets, it will require separate connection per bucket

Mike Evans
added a comment - 02/May/13 12:42 PM Pushed a Connection Pool branch for review:
http://review.couchbase.org/#/c/26028/
which relies on:
https://github.com/mperham/connection_pool
It's mainly targeted at ActiveSupport::Cache::CouchbaseStore, however could be used more generically for any Couchbase bucket connections:
@pool = Couchbase::ConnectionPool.new # => default connection args and 5 connections in the pool
@pool.set('foo', 'bar') # => checks our a connection to run set on--will block until a connection is available
@pool.get('foo') # => 'bar'
I think I implemented all operations but might have missed some.