I'm trying to run a single worker that could perform a periodic task
for a given user.
>From a controller, I imagine something like:
def start_job
MiddleMan.ask_work(:worker => :foo_worker, :worker_method => :perform_task,
:data => { :user_id = current_user.id })
end
def check_job
@status = MiddleMan.ask_status(:worker => :foo_worker)[current_user.id]
end
My worker is something like:
class FooWorker < BackgrounDRb::MetaWorker
set_worker_name :foo_worker
def create(args=nil)
@mutex = Mutex.new
@mutex.synchronize do
@statuses = {}
register_status(@statuses)
end
end
def do_task(some_user_id)
thread_pool.defer(some_user_id) do |user_id|
user = User.find user_id
save_status user_id, :progress, "Starting Task"
user.do_some_database_stuff
save_status user_id, :progress, "Task Stage 2"
user.do_some_other_database_stuff
save_status user_id, :progress, "Task Stage 3"
save_status user_id, :completed, true
end
end
def save_status(user_id, key, data,)
@mutex.synchronize do
if @statuses[user_id].nil?
@statuses[user_id] = {}
@statuses[user_id][:completed] = false
end
@statuses[user_id][key] = data
register_status(@statuses)
logger.info "statuses synced for #{user_id}, #{key}, #{data}"
end
end
end
Problem is, when I use thread_pool, it gets as far as the first part
of the task and then just dies, and the backgroundrb script just
outputs "going to sleep for a while" over and over until I kill it. I
tried taking out the mutex and the register_status bits and the same
problem happens. When I use "Thread.new" in place of the thread_pool
line, everything works. However, I get some "Some read error" messages
during the "do_task" work. Everything happens like it should - I even
ran two sessions concurrently and initiated the task with two
different users at the same time. The work gets done; although a bit
slowly and with those strange errors. Oddly enough, when I run the
jobs from the rails irb console, they go without any error - even if I
start a couple jobs for multiple users and spam "MiddleMan.ask_status"
while the jobs are still running. (of course, when I start the server
using the 'Thread.new' method, I get a "no marshal_dump is defined for
class Thread" exception in the backgroundrb_server log. I'm guessing
this has something to do with why I'm supposed to be using thread_pool
instead of Thread).
When I get those read errors, the backgroundrb_debug.log shows "Client
disconected" (sic). I don't know if this is normal for rails talking
to bgrb or if it's an error. I don't see it on every line, but it
seems to coincide with the "some read error" messages in the script.
Am I doing something wrong? Anyone have a working example using
thread_pool? Also, I've never used Mutex before, so if my usage is
off, please let me know.
Thanks very much,
- Jason L.
--
My Rails and Linux Blog: http://offtheline.net