Thursday, May 26, 2011

Fedora 15 and offlineimap users beware

This is just a word of warning: if you have upgraded to F15 and you use offlineimap, observe very carefully what it does and ensure you have a backup of your email.

This post has updates. See at the end of the post

I noticed yesterday that I couldn't search for some email that I knew existed. On closer inspection I found a huge chunk of my archive inbox to be missing. Several months worth of emails, they stopped in March and then continued with the emails from yesterday that I had copied in from my Inbox.

The same happened to my fedora-devel and xorg-devel folders, possibly others. Re-running offlineimap to sync these pulled down hundreds of emails, so I let it run overnight. This morning I discovered that sometimes offlineimap simply states something like "Deleted 1018 messages" and they're gone again. This is cumulative, it seems to delete only newer ones (I was at September 2010, but after one more run I'm now back to July 2010 being the newest messages in in my xorg-devel folder).

The good news is that the emails are still on the server. Also, so far only mailboxes with huge numbers of emails (>10000) seem to be affected, smaller mailboxes appear to work fine.

I have not filed a bug yet because I don't know what the actual issue is. Could be offlineimap, could be something else. But in the meantime, beware.Update May 30: Filed as Bug 708898

Update May 30I downgraded to offlineimap 6.2.0 which is in F14 (offlineimap-6.2.0-2.fc14.noarch) and managed to eventually resync my mailboxes. Then I went back to F15's offlineimap-6.3.3-1.fc15.noarch. Of course it didn't trigger on my test mailbox, but it did then do the same thing again on my xorg-devel inbox.

F15's offlineimap-6.2.0-3.fc15.noarch does not show this behaviour and seems to resync the locally deleted emails. Of course that'll take a while, but I'll try to get logs this time (my first couple of tries to get logs from offlineimap crashed it).

I haven't filed a bug yet because I haven't narrowed it down yet. As pointed out in the article, I suspect it is offlineimap but I'm not sure. I've updated the whole system, who knows what could trigger this behavior.

Hi there, offlineimap hacker here. Unfortunately without a debug log of what has been happening, this is extremly tough to reproduce. (I expect you would say the same about graphics issues with x.org :-)). Unfortunately, loss of emails is much more annoying that a temporarily corrupted screen... Sigh.

If you can log it with a -d imap,maildir -l debug.log and send it to us (or to one dev privately due to privacy issues), we could try to find out what went wrong.

Spaetian "spaetz" Spaeth

P.S. With respect to speed compared to isync, I would say that the python speed is the least of our worries :).