POP3 Email Client with full MIME Support (.NET 2.0)

C# class reading ASCII emails from a POP3 server and converting them using MIME to aSystem.Net.Mail.MailMessage derived class for further processing. The complete code is provided (pure C# 2.0, only .NET framework DLLs used). If possible, it matches MIME multiparts to body, attachment, etc. of MailM

Introduction

This is part 2 of my articles about email receiving with POP3 and MIME processing. My first article POP3 Email Client (.NET 2.0) covered the reliable downloading of emails from POP3 servers, which left us with a pure ASCII representation of the email body. This was the easier part.

In this article I provide the code to split the raw ASCII email into body, attachment, alternate views, etc. This was much harder to do, because while the POP3 specification is simple and specified straight forward in one RFC, there are several MIME related RFCs, which provide a multitude of possibilities how simple stuff like an email's actual text can be sent. The MIME specification allows for great flexibility, but Microsoft, being Microsoft, of course supports only a subset (for example no recursion of MIME parts within MIME parts). The provided code supports both worlds completely and gives the programmer the flexibility to access information about the received email as needed.

If you wonder why I wrote this article despite the fact that there are various articles on CodeProject for MIME support, here are some of the shortcomings encountered:

Structure of a simple email

The first 4 lines are called the header of the email and they are separated from the body by an empty line. The end of the email is marked with a line containing just one "." (a period sign). There will be many more header lines when you look at a real email, some RFC standard ones and others, like this one from GMail:

X-Gmail-Received: f105c784e77f8b689759558db72ccd07f60387ba

Introduction of MIME

In the beginning there were just plain ASCII emails as defined in RFC 2822. Plain ASCII was soon not sufficient, though, and the Multipurpose Internet Mail Extensions specification MIME was created to support non US-ASCII texts, multi-part message bodies, rich text (HTML), images, sounds and attachments. The specification tried to offer great flexibility and to cater to all kind of possibilities. The result was numerous RFCs (2045, 2046, 2047, 2049, 2231, 2387, 4288, 4289, ... ). As it often happens in big groups, the whole thing became rather complicated and, even worse, left it to the implementer how precisely body text, etc. are implemented.

In order to help you with the extraction of information from MIME based emails, I'm going to explain to you the basic MIME principles. First let's have a look at a complete MIME email. It might be a bit confusing, but it gives a good overview of the various MIME elements which I will explain one by one. This email has one email header, followed by the email body text and a .GIF picture. Notice the "--0-494165446-1157210079=:74253" line, which separates the various parts of the email, called MIME entities.

The Content-Type field is used to specify the nature of the data in the body of a MIME entity, by indicating media type and subtype identifiers, and by providing auxiliary information that may be required for certain media types. Some of the media types are:

text

image

message

audio

application

multipart

Each of the media type defines its own set of subtypes, which might be followed by a set of parameters, each specified in an attribute=value pair. For example:

Content-Type: text/plain; charset=ISO-8859-1; format=flowed

The media type is "text", the subtype is "plain", the attribute is "charset" and the attribute value is "ISO-8859-1". There could be more attribute=value pairs like "format=flowed".

Content-Type Multipart

The media type "multipart" provides the flexibility to split an email into several parts, like plain text, HTML text and attached files. There are versions of multipart (subtypes), but all have the same attribute "boundary". Its value is a string which is unique in the whole email and is used for marking the boundary delimiter lines of the various parts. Let's look at the previous example again, this time only with Content-Type information lines:

The first 3 lines are part of the email header. The end of the header is marked by an empty line. All other lines are part of the email body, which ends with the line having only a "." (period). The boundary delimiter line breaks the body itself into the email text and the file attachment. This line always starts with "--" followed by the boundary string. The last boundary delimiter line is followed by trailing "--".

Each MIME entity has a entity-header and a entity-body separated by an empty line. Since emails and MIME entities use the same structure and the same kind of header lines, it is possible that whole emails can become a MIME entity, which is useful for mail systems (Content-Type: message). But of course having an email in another email in another email leads to many complications, and so it is no wonder that most mail program use a different solution for forwarding an email, they just merge it with the email text body. This has the advantage that even mail clients not supporting MIME can handle forwarding. Similarly, even the MIME specification is recursive, Microsoft's System.Net.Mail.MailMessage is not ! More about this later.

Content-Type: multipart/mixed

Often the top most multipart subtype is "mixed". It indicates that the email consists of several MIME entities, without specifying anything more about the kind of entities. "multipart/mixed" is used as default, if the actual subtype is not recognised by the email client.

Content-Type: multipart/alternative

The subtype "alternative" is used, if the same email is sent in plain text and HTML. Both have the same content, but in alternative coding. The email client is supposed to display to the user the last alternative part understood by the client. If an email consist of a plain text entity followed by an HTML entity, the email client is supposed to display the HTML text, even if it also knows how to display plain text, because the HTML version came last. An email with plain text and HTML can look like this:

Notice that the picture is part of multipart/mixed, not multipart/alternative.

Content-Type: multipart/related

Multipart-related can be used to send HTML text and graphics or other related material in the same email. It is beyond the scope of this article to explain the details of any other of the multipart mediatypes.

Content-Transfer-Encoding

POP3 defines that the body of an email is 7bit US ASCII code. Since the text displayed to the user can be any Unicode and file attachments are usually array of bytes, the email sender must encode this content to ASCII and we, the receiver of the email, need to decode it. If the value is "7bit", no encoding was used. "8bit", or "binary" has the same meaning, but is not supported by the .NET framework. I treat "8bit" like "7bit", i.e. take the content as it is, whereas "binary" is illegal in POP3, because some character sequences like CRLF "." CRLF have a special meaning in POP3, but might occur in random binary.

Content-Transfer-Encoding: quoted-printable

If a MIME entity consists mostly of US ASCII characters, it is enough to encode just some special characters and all bytes not covered by the US ASCII characterset. "quoted-printable" does this by sending a "=" and the hex value of the byte as ASCII characters. A carriage return (hex: 0D) becomes: "=0D". There are a number of rules dealing with special circumstances. I couldn't find a decoder for quoted-printable in .NET, so I copied the QuotedPrintable Class, by Bill Gearhart source code from ASP emporium.

Content-Transfer-Encoding: base64

Base64 uses a limited set of characters ("A"-"Z", "a"-"z", "0"-"9", "+", "/") to express a 6 bit value. Any 3 bytes can be expressed with 4 encoding characters. As an example, let's take the first 4 ASCII characters "R0lG" of the graphic file in our example email:

Using the code

The best way to get an understanding of a library is to use it. The Main function in the downloadable code does just that. It connects to an POP3 server (don't forget to provide the proper server name, user name and password) and downloads at most 5 emails. The code will not delete the emails from the server, but the server might delete them anyway, depending on its settings. The structure of the 5 emails will be displayed on the console. "Program.cs" also contains the method SendTestmail() to generate some sample emails.

Emails are received by Pop3MimeClient derived from Pop3MailClient, which is described in POP3 Email Client (.NET 2.0), by Peter Huber and offers all the functionality to interact with the POP3 server. Pop3MimeClient adds the method GetEmail which fetches one particular email from the POP3 server and returns it decoded as RxMailMessage.

Mapping MIME to System.Net.Mail.MailMessage

The System.Net.Mail.MailMessage class is used by System.Net.Mail.SmtpClient for sending emails with SMTP. MailMessage includes only the information needed to send an email. Receiving an email creates some additional information. Therefore, a new class RxMailMessage is inherited deriving from MailMessage and adding properties like DeliveryDate or DeliveredTo.

The SmtpClient converts MailMessage to a MIME conformant email, but MailMessage provides hardly access to any MIME related functionality. When receiving an email, we would like to store the complete information. Pop3MimeClient receives the first MIME entity by MIME entity and stores them as a MIME entity tree in the new Entities collection property of RxMailMessage. If possible, the info is also copied to the properties inherited from MailMessage. This gives the user the freedom to choose if the complete email in MIME form is used for further processing or just the simpler, but possibly incomplete Body, AlternateViews or Attachments as defined by MailMessage. The method decodeEntity can be used as an example how to loop through all MIME entities of an email.

8.10.2006 Improvements Attachment Handling

Detecting content-disposition header field and creating an attachment if it looks like: "C-Disp: attachment"

Only MIME entities from multipart/alternative parents become alternative views

Added end markers for multiparts in RxMailMessage.MailStructure()

17.9.2006 Original Post

I was not too sure how to map the various multipart entities to email body, etc. I analysed probably thousands of emails I received and populated the RxMailMessage properties as appropriate, but it is very likely that you might receive a different formatted email. Please provide some feedback here, if you do so or find any bugs.

But recently I also needed to internationalize From field... so for anyone who needs it, here is the code (you need to replace private void ProcessHeaderField(RxMailMessage message, string headerField)):

The 8-bit hexadecimal value 20 (for example, ISO-8859-1 SPACE) may be represented as _ (underscore, ASCII 95). (This character may not pass through some internetwork mail gateways, but its use will greatly enhance readability of Q encoded data with mail readers that do not support this encoding.) Note that the _ always represents hexadecimal 20, even if the SPACE character occupies a different code position in the character set in use.

In order to download text file email attachments, I am writing the ContentStream of the attachments to a FileStream, but the resulting file has equals signs inserted in the text. The equals signs are inserted at fairly regular intervals, normally between 68 and 73 characters. It probably has to do with the fact that the Transfer Encoding of the attachments is Quoted Printable.

Does anyone know how I can prevent the equals signs from being inserted?

Hello,I am new in this area of development. I am writing an application to read all Email body on Windows Mobile 6.So I realize that the most email on windows mobile are in Mime Format. I will like to know if with your application it possible to parse WM6 MIME format to get the email Body? Thanks you for your Help

Is there are any way to filter all junk mail from POP3 Email ?Because POP3 Email Client is reading all email from inbox including junk mail.Can guide me how to filter all junk mail from pop3 email based on email status?

Hi, First of all, thanks to Peter Huber SG for such a nice API. I have been using this in many projects now.and then thanks to pmcmurray and BillJam11 for the fix. I was facing the same problem.

I encountered a problem recently when i tried using the API with POP3 SSL connecting to MS Exchange 2007. The exception comes likeItem has already been added. Key in dictionary: 'size' Key being added: 'size'

Upon investigation I came to know that on line 707 in Pop3MimeClient.cs, the parameter collection of AttachmentContentDisposition is being cleared.AttachmentContentDisposition.Parameters.Clear();

But the lines after it adds one parameter in the collection.AttachmentContentDisposition.Size = messageContentDisposition.Size;

So now when we try to add parameters in the collection using foreach loop, exceptions is being raised for 'size' parameter.

So either you can move the clear parameter line to after the size assignment i.e.

First of all 5 out of 5 from me This has saved me hours and hours of work. A great piece of work, thank you. (4 years on and still going strong)

Now my question !Could someone please tell me how to get more than 5 mails from a pop server? I have set maxDownloadEmails to 20 yet it hangs at 5 (attempting download of 6) and seems to sit there with memory being swallowed up like no tomorrow.