I have a longstanding situation with a client who has a tremendous amount of email. To make matters worse, the bulk of the message content and attachments are text files. So, we're talking about a lot of headers, small files and a whole bunch of content to index.

The platform the data is currently on is Hosted Exchange 2010. I chose that under the assumption that it was the most robust email solution out there. The hosted platform I am on has some really nice features, such as being able to grab a weekly PST snapshot of every mailbox for backup purposes, which is difficult feat with an IMAP provider.

The worst user is one whose inbox receives about 5000 messages per month and most of them cannot be removed because they contain important legal information about contracts and deals, so the integrity of the threads must be preserved.

To compound the issue further, there are 5 mailboxes that are all subscribed to with Apple Mail via Exchange protocol on 5 computers. They insist on using Apple mail, and on having all those mailboxes available to all the machines, which is legitimate as they need to be able to search correspondence related to deals and contracts across all the accounts at the company to make sure no exchanges get missed when responding to a client.
The only viable alternative would be to suggest that they start a different database to hold those threads, which would be a lot of extra work for them and would probably require hiring extra workforce to process.

I have exhausted my knowledge of how to deal with this and the volume of email coming in continues to increase. The email incoming rate in the primary mailbox has doubled since last year.

What I have done to address the problem so far is:

"chunked out" all server side mailbox folders to a size of less than 5000 headers

moved non-primary accounts for each user into Outlook 2011 in headers only mode

removed all non essential communications out of the mailbox (Facebook alerts, promotions, etc.)

moved content before 2012 to a separate exchange mailbox for the primary user to bring the total mailbox size down to under 7GB for the primary and its archive, all other mailboxes are smaller

had the hosted Exchange provider move all mailboxes to a new server-side database

exported all local data to a Mail archive, deleted all Mail data and recreated all accounts

No matter what I do, I am having consistency issues with Apple Mail maintaining sync with the accounts reliably. Symptoms:

server side folders randomly disappear or lose their content (all is well server side, Mail just fails to display a folder or its content)

inbox stops updating, loses all its content or reverts to a date in the past

The users insist on continuing to use Apple Mail. Although I am fully aware that there are several issues with this application, that it does not properly implement IMAP or Exchange, has issues maintaining the integrity of its local caches and many other issues, I am still doing my best to try and allow the users to continue using it as they prefer to do. Part of the reason is that I am not certain Outlook 2011, or any other email client, can handle this gracefully. And if I were to insist that the only way this will work well is to switch email clients, I would have to be absolutely sure that the new solution would indeed maintain integrity.

Archiving content locally is not an option as that content would have to be propagated to at least 5 machines and kept in sync with every archive operation, which must happen frequently with the amount of content coming in.

If anyone knows of a sure fire solution for this, something that would resolve this situation once and for all, I am all ears.

I have tried to add the Exchange accounts as IMAP in Mail, which does yield better sync results but aggravates a different problem that is a deal breaker - with that many connections to the same mailbox (5 Mail + 2 iOS devices), the Exchange server runs into too many concurrent IMAP connections issue. This happens regardless of which client I use, I was able to replicate the problem using MS Outlook 2010 and 2007, Thunderbird on Win and OS X, Mail and Outlook 2011. So, at best I can maintain a single IMAP connection without risking connection rejections because of the IMAP concurrency limit. I have also verified that other hosted Exchange implementations suffer from the same limitation by testing against Office 365.

I could move the situation back to an IMAP only provider, but again, I am not sure which provider I would choose who could guarantee better results.

1 Answer
1

Leverage Outlook Web Access... Consider real Outlook 2011 for Mac as well, as it can handle the volume and concurrency issues. Make note that the Outlook for Mac uses the same Spotlight search solution and is just as adept at trawling through indexed mail.

In my experience, Apple Mail tends to fall over in these situations. You've hit the limits of the application. If the users insist on doing this and working this way, they need to be on more robust software.

Outlook Web Access has a nasty problem that can't sustain a message draft for very long. If you take more than a few minutes to compose a message, OWA comes back with "message modified form another location" and you have to copy and paste the content into a new message in order to send it. So that's a drag if you have to use it all the time. It also cannot maintain a connection to more than a single mailbox at a time. You'd have to have 5 different browsers open to connect to 5 different mailboxes. And can't search accross all of them. Outlook 2011 will be what I recommend, hope if does better
–
zentechMay 9 '13 at 5:52