Bug Description

Currently, the daemon is run via a shell script that performs some basic actions and then executes some Twisted (Python) code. The reactor.run() method is called directoy, instead of using either a Twisted plugin or, alternatively, a Twisted application.

We need to (at the very least) set up the daemon code so that a Twisted application is instantiated and then run as part of a Twisted service. This will provide greater stability, improved logging support, easier command-line control, and graceful exiting.

The deamon code is very monolithic right now, and won't lend itself to this very easily, as things are. I have some plans about how to accomplish this, and will create a blueprint for it, once that's turned on for the project.

Thanks for fleshing out a blueprint on this Duncan; this is definitely something that we will want to have a solid plan for first. I would like if we can scope this work out to specifically getting the existing daemon API running as a twistd service although in future a complete evaluation of the current daemon API would be useful.

When the blueprint is set up if you could associate it with this bug that would be great. Thanks!

Hrm, so... I've started working on this in a branch for utilizing twisted.application (for use with twistd), but... there's so much that's not actually done in such a way that we can use Twisted in the way it's meant to be used. I'm fixing only what is strictly necessary for this ticket, but there are some pretty extensive changes necessary. For the stuff I'm not changing right now, I'm leaving TODO breadcrumbs throughout the code... and as of now, there are several loaves' worth...

On 18 March 2012 21:15, Duncan McGreggor <email address hidden> wrote:
> Hrm, so... I've started working on this in a branch for utilizing
> twisted.application (for use with twistd), but... there's so much that's
> not actually done in such a way that we can use Twisted in the way it's
> meant to be used. I'm fixing only what is strictly necessary for this
> ticket, but there are some pretty extensive changes necessary. For the
> stuff I'm not changing right now, I'm leaving TODO breadcrumbs
> throughout the code... and as of now, there are several loaves' worth...
>
> In short, I may have to put this work in a separate branch.

Can you elaborate a little more on how the current code could not be
used with twistd?

On Mon, Mar 19, 2012 at 12:22 AM, Jono Bacon <email address hidden> wrote:
> On 18 March 2012 21:15, Duncan McGreggor <email address hidden> wrote:
>> Hrm, so... I've started working on this in a branch for utilizing
>> twisted.application (for use with twistd), but... there's so much that's
>> not actually done in such a way that we can use Twisted in the way it's
>> meant to be used. I'm fixing only what is strictly necessary for this
>> ticket, but there are some pretty extensive changes necessary. For the
>> stuff I'm not changing right now, I'm leaving TODO breadcrumbs
>> throughout the code... and as of now, there are several loaves' worth...
>>
>> In short, I may have to put this work in a separate branch.
>
> Can you elaborate a little more on how the current code could not be
> used with twistd?

Sure.

The way it's currently written, it's very difficult to tell what is
deferred and what is not. In fact, there's actually quite a lot of
blocking code occurring in methods that should be returning deferreds.
Even more, there are methods that *should* be returning deferreds (or
using inlineCallbacks) when they're not. The (non-)functional result
is a daemon that only appears to be async.

One of the benefits of using twistd is not only that you get a daemon
out of the deal, you also get appropriate process management and
signal handling. It's not clear to me that this application will
behave as expected, handle errors appropriately, and take advantage of
the benefits of twistd without first fixing the code.

There may be more problems; I won't be able to tell until I've
finished trying to put this stuff in a twisted app...

That being said, I'm actually *trying* to get it to run with twistd.
For more details, you should check out the branch... there's a LOT of
comments in there now ;-)