Re: Building a youtube-like web app using activemessaging and amazon s3

Great screencast Zhang Xue, thanks!

DFischer wrote:

Question though, why use activemessaging when all it is, is essentially a callback to convert the video through ffmpeg? Can't you do an "after_create" callback in the model?

I'm not the author of this screencast but I think that reason is very simple. Suppose we use "after_create" callback. After uploading, file is coverted (ffmpeg) and send to s3 service. It could take a few minutes so in this case browser will be still waiting. Good solution should be asynchronous and activemessaging will give you it. Another reason will be possibility to reduce number of simultaneous threads converting files (but I don't know if activemessaging can this, but I think it should). Imagine that 10-20 people upload file simultaneous. It could kill the server .

Re: Building a youtube-like web app using activemessaging and amazon s3

Re: Building a youtube-like web app using activemessaging and amazon s3

@Radarek - sorry for the late reply, just found your post.

"Good solution should be asynchronous and activemessaging will give you it. Another reason will be possibility to reduce number of simultaneous threads converting files (but I don't know if activemessaging can this, but I think it should). Imagine that 10-20 people upload file simultaneous. It could kill the server."

Activemessaging won't help with simultaneous upload - that depends on the http serving set-up, but it will help with the conversion. Messages to convert the uploaded files will be queued, and will be processed only by the number of processes you choose to start. Messages are received by a ruby process running the 'poller' script - each is single process, and only use multi-threading for message broker polling, not for processing.

So, for example, if you start up 4 poller processes, that is the most that will run simultaneously - the other 16 messages wait in the queue until a processor finishes and picks up the next message for processing. You control the number of poller processes much as you would the number of mongrel instances in your cluster - it depends on how many you spin up. Wo do exactly this set-up for audio processing - sometimes we end up with a few files waiting in the queue, but it never hoses the server.