Projects

Recent blog entries by jasper

Oops, Jerzy Balamut, the solid pop3 author pointed out
that the pop3 protocol dosn't count bytes stuffed line
beginings in the size returned by the LIST and STAT
commands. I checked the RFC's, and he's right.

It's not that bad actually, cos i've been thinking about
writing a script that takes a mail on stdin and writes it
into a maildir with the right line endings etc.

It's needed for MTA's that arn't as flexable as exim, and
i can now encode the file size without byte stuffed
line beginings in the filename (a bit of a hack, but not a
problem).

I'm setting up a high capacity pop server to replace one
thats thrashing it's self to death at the moment, so I've
been trying to find the most efficent pop server for what I
want to do.

At the moment we are running qpopper,
but if a mailbox gets above 50Mb or so the server really
slows down as qpopper copies the entire mailbox
before and after a user transfers their mail.

We need a system that works well with:

Large (50Mb+) mailboxes

Many users accessing large mailboxes at the same
time

To get around the problem of large mailboxes I want a pop
server that uses maildir, and for vairous reasons I
want one that runs standalone, i.e. not through
inetd.

After some searching through vairous mailing list
archives and freshmeat
I decided upon Solid
POP3.

The problem now is that the wire-format for POP3 has
all line endings as \r\n, where as the normal unix line
endings are just \n. This means you have to scan through
each message to get a count of the lines so you can account
for transmitting all the line endings as \r\n - you need to
know the size of each message so you can give the right
responce to the LIST command.

With the test mailbox I was using (an excessive 1Gb in
size and 329 messages) this takes 2 minutes!, which is
WAY to long as far as user interactivity goes!

Now, since the only processes that are going to be
touching these mailboxes are the MTA i'm using (exim), and the pop server I
could do whatever I wanted with the mailbox format.

I set up exim to deliver all the mails with \r\n line
endings, and then created a new mailbox type in the solid
pop3 source, crlf_maildir.

Since we don't need to munge the line endings any more
all we have to do for tha vairous pop commands is:

Getting the data for LIST is just a question of looping
through readdir and stat'ing each file
for it's size.

RETR is just a mmap and write.

TOP is just mmap, loop through counting line
endings, and then write. (fetchmail uses TOP
<msg> 99999999, and so might other pop clients, so if
we check to see if no. of lines >= size of message we can
just chuck out the message as a whole...)

... and everything else is as before

I've implemented most of this, and it now takes less than
a second to authenticate a client, rather than 2 minute,
which is a nice imporvement ;-)