Thoughts on “VMware Zimbra vs. Microsoft Exchange”

In a blog post by Christopher Wells, alias vSamurai, the author positions VMware Zimbra Collaboration Server (ZCS 7.x) as an enterprise-ready drop-in replacement for Microsoft Exchange Server 2010 environments of all sizes. He also suggests Zimbra is a better multi-tenant solution for ISPs. The author does this by comparing both products in a feature comparison.

These reviews are helpful in order for companies to make an informed decision. After all, there’s nothing wrong with a bit of competition. However, we wanted to create a response because some statements are flawed or just plain wrong. In the process, we will be following the structure of the referenced blog:

Backup and Restore
The author starts off by claiming that “the ease with which backup and restore can be performed in Zimbra outweighs the capabilities of Exchange”. While it’s interesting to note the author implicitly admits Exchange is more capable, he misses the point. The product should follow a well-designed backup and recovery strategy, based on customer demands and compliance regulations. Where Exchange has server, database, mailbox and single item recovery options, Zimbra is built on top of MySQL, meaning recovery requires brick level restore or (partially) restoring information from MySQL dumps. Also, in Zimbra the databases only contains meta information; the actual messages and attachments are stored on the file system. While this makes sense for Zimbra, as many SQL people consider storing binary data in databases a bad practice, it increases the complexity of backup and restore, because meta information and file system needs to be in sync. Note that Exchange’s Extensible Storage Engine (ESE) is purpose-built for storing mailbox information, including attachments.

ScalabilityThen, the author claims that Zimbra has better scaling capabilities than Exchange. First, let’s start by looking at the definition of scaling. A system is said to scale well if:

it can handle increased load without (serious) performance penalties, or

the system is able to accommodate growth by adding resources (scale up) or additional systems (scale out).

Ideally, scaling up should show a linear pattern. Scaling out most of the time doesn’t, which makes sense when looking at how computers are designed using shared resources like buses for example.

Now, scaling isn’t solely a matter of hardware; a system also requires software built to scale. The role-based model of Exchange, with its specific roles for serving mailboxes and handling replication, routing e-mail and servicing clients, is a good example of a thought-out scalability supporting concept. Of course, you can install all roles on a single server, which is currently the recommended practice by Microsoft, but you’re still able to design fit-for-purpose farms and clusters.

In their analysis, VMware primarily focuses on the CPU utilization figure. That figure implies that Zimbra has more headroom than Exchange using the same configuration. However, Exchange also has several background processes which perform tasks in the background, like optimizing the database to reduce the number of IOPS. Yes this takes up a certain % of CPU cycles, but optimizing storage for sequential access could explain the significant 240% decrease in IOPS for Exchange. Lower IOPS reduces storage requirements – and costs – for Exchange. The over 60% lower latency figure for Exchange is also an indication overall processing of messages is faster in Exchange.

CostsAs often in these Open Source Software (OSS) discussions, the cost card is played. The author claims that on average, Zimbra is 50% cheaper than Exchange. However, this claim is made without any supporting references or figures, making it difficult to verify this statement. However, from our experiences, those claims are often primarily based on retail prices and licensing costs. What is often overlooked (or ignored) in comparisons with OSS, are training costs or hidden costs like support or maintenance.

Functionality is also a potential cost saver, as companies can work more efficiently due to added or enhanced functionality. These savings depend on customer needs, although some are widely used and immediately contribute to lower costs, like for example AutoDiscover (automatic configuration of Outlook 2007 and later clients or ActiveSync devices).

Exchange natively supports Outlook, common browsers and mobile devices; Zimbra requires an Outlook plug-In, Zimbra Connector for Microsoft Outlook, increasing support and maintenance costs. Note that this connector is only available for Zimbra Collaboration Server Network Edition Professional users.

Storage BenefitsA full paragraph is dedicated to the benefits of using Zimbra with NetApp storage. However, the NetApp products and technologies mentioned are not Zimbra specific, and therefor in our opinion do not add anything to the discussion.

Feature ComparisonThe author then continues with a “direct” feature comparison between Zimbra and Exchange. Let’s have a look:

Platform Architecture

First, author claims ESE is over 20 years old, the .EDB file is non-modular and the ESE engine is non-tunable. Yes, ESE exists for over 20 years, but that’s also 20 years of experience in building a fit-for-purpose database engine. With each new Exchange version, ESE was redesigned to meet evolving requirements and expectations in a changing world. When looking at the VMware IOPS comparison in the Scalability section, it’s Zimbra that should worry about storage.

Second, author claims Database Availability Groups (DAGs), based on Fail-over Clustering, isn’t a proven technology for large deployments. Exchange 2010 is on the market since October 2009. Like many Exchange fellows, we have designed or seen large Exchange deployments (i.e. thousands of mailboxes). Also, if millions of Office 365 users aren’t proof of a successful large scale multi-tenant ISP-like deployment based using multiple data center DAGs, what is?

To be honest, is it really that important which exact technology is used and how old it is? In the end functionality and performance are more important, as they are relevant in any business case for Exchange. What would a decision maker most likely ask, “Does it use Microsoft SQL Server?” or “What can we do with it and how much will it cost?”. We think and know out of experience it will probably be the latter.

Reliability & Robustness

The author claims Microsoft is considering (moving Exchange storage) to SQL and needs to prove robustness of the new architecture. While Microsoft has considered the SQL storage engine several times, it decided to stick with the optimized ESE engine. This was also true for Exchange 2010 back in 2009, like you can read in this blog. Main reason for deciding to stick with ESE is performance.

When pleading for ZCS, the author states “Linux has better uptime”. While this may have been true in the Windows 98 era, from experience, managed Exchange systems can reach similar uptime figures. On the contrary, I’ve seen Linux systems crashing every few days. The only conclusion you can draw here is that reliability not only depends on hardware and software components and their quality, it also depends a lot on if and how systems are managed. Also, don’t confuse uptime with availability, as planned downtime will reset my uptime statistic, but that’s all it is: a statistic.

Tiered Storage (was Platform Scalability)

Tiered storage, or Hierarchical Storage Management, is about classifying data in terms of things like security, performance or pricing. Exchange itself partly supports this concept, using elements like DAGs, databases, mailboxes, personal archives and retention policies. For example, you can home your mailbox on multiple lean and mean servers using fast SAS storage while personal archives, used to automatically store e-mail older than 1 year using retention policies, are served by a fat server using inexpensive SATA disks on JBOD storage.

Because Zimbra uses file base storage, ZCS can utilize existing HSM solutions using stub-based mechanisms to replace items with links to files relocated to slower or cheaper storage. But that doesn’t mean that Exchange doesn’t support it; When an Exchange customer wants to use this level of storage tiering, they can opt for 3rd party solutions like Symantec Enterprise Vault (item-level stubbing) or storage solutions.

Note however, there are some important factors to take into consideration with stubbing:

Data stored on a different tier, e.g. tape, isn’t always available online;

Tiered storage adds complexity, introducing the need to compare reduced costs for storage against additional costs due to increased complexity;

Author claims DAGs do not provide Exchange infrastructure protection and have a learning curve. The first part of that claim is absolutely true: DAGs are designed to increase the availability of Exchange databases served by Exchange servers holding the Mailbox role, while providing a fail-over mechanism. Covering for the other tasks are the other Exchange roles. Mail flow within an Exchange Environment is automatically redundant when you have multiple Hub Transport servers, as they monitor connectivity and possible routes for delivery. For client access, multiple Client Access servers can be made redundant using load balancing technology. Exchange has these built-in features that work independent of where Exchange is running, i.e. they also work in a non-virtualized system and no additional high priced product is required to make the underlying services highly available.

Regarding the learning curve claim, every new technology has a learning curve. DAG is built on top of fail-over clustering (nothing new) and easier to manage than its predecessors, CCR and SCR. Then again, we’d prefer Exchange admins who know what they’re doing, rather than somebody who learned an SRM trick.

Speaking of which, the whole argument that “ZCS with VMware’s Site Recovery Manager (SRM) is proven, scalable and effective” is apparently nothing more than a plug for VMware’s SRM product in conjunction with VMware licenses (vSphere required), as we see no credible arguments.

Platform Extensibility

The author states that Microsoft recommends using its proprietary shell. We assume he means PowerShell, which is here to stay. Other vendors, like Cisco or Quest, are adopting it and offer modules to manage their products using PowerShell. Heck, even Zimbra offers PowerShell scripts to manage Zimbra through encapsulated SOAP requests. For the record, we both don’t know of any Exchange admin complaining about some Linux product requiring bash (Bourne-Again shell) or perl for scripting, turning this in a non-argument.

The author continues by apparently mixing a few things up. The argument given for ZCS is that “SOAP API allows server access using web services framework for client access and Zimlets for integration with 3rd-party services” while Exchange offers “limited SOAP access” and “Outlook add-ins require developer effort”. This is apples versus oranges; Outlook is a fat client and Zimlets are like web parts. If you want to make a nice dashboard, we’d suggest you use something like SharePoint instead of bloating your e-mail web client. Finally, SOAP and Exchange Web Services (EWS) are targeted at developers, PowerShell at automation. If you’re curious about the power of EWS, we’d suggest you check out the excellent blog by Glen Scales.

Platform Openness

While Exchange is mostly closed source, a lot has changed since the 90’s. Exchange has a developer center nowadays, where SDK and APIs are published on how to interact with certain parts of the Exchange ecosystem, e.g.:

It’s true that the current Outlook version doesn’t support all available standards for exchanging calendaring or contact information. However, for most companies that isn’t an issue. When required, solutions and workarounds are available.

Yes, there are browsers out there that don’t have the full featured Premium OWA (like Opera), but “limited browser support for OWA” is a bit over-simplified, especially if you take into consideration the combined market shares of the fully supported browsers (without Safari, between 81-91% since December 2011).

Mac Support

Outlook team and Mac Outlook are produced by two different teams, which might be one of the reasons of the feature disparity between Outlook 2010 and Outlook for Mac 2011. Apart from differences caused by the underlying operating system, we agree features should be as on par as possible for all available platforms.

Note that the mentioned Zimbra desktop client doesn’t support Exchange’s native MAPI protocol, adding the requirement to enable the IMAP or POP protocol on the Exchange server.

Linux

The author proceeds by arguing there’s no Outlook client or Exchange Server for Linux. That is a moot point; there’s also no Zimbra server for Windows. Also, when somebody’s trying to convince you using arguments like, “ZCS server components love the Linux platforms”, that’s not very convincing now, and is often seen with discussions when emotions prevail over rational thinking.

Mobile Support

More and more (mobile) clients are adopting the Exchange ActiveSync (EAS) protocol for exchanging e-mail, calendar, contact and task information with Exchange. In fact, even Blackberry announced they will adopt EAS in their upcoming Blackberry 10 OS product. This is probably driven by Microsoft releasing EAS protocol as part of their Open Specifications Promise, turning EAS more or less into the de-facto standard for (corporate) e-mail synchronization for mobile clients.

Multi-tenancy
The author doesn’t show how Zimbra is a better multi-tenancy solution for ISPs when compared to Exchange 2010. But since Exchange 2010 Service Pack 2, there is no need for third party hosting software as it is now fully incorporated in Exchange without extra costs.
However; the intent was possibly to prove this implicitly via the costs argument of on-premises deployments. One other way is to look at actual hosted Zimbra and Exchange solutions available commercially.

Let’s compare costs from random Zimbra providers (picked from Zimbra’s Partners list), Exchange hosting providers and Office 365 subscriptions. It is not an extensive comparison, but it should give us an indication. Some (not all) are shown here:

(Click the image for a larger view)

This table shows that the Exchange subscriptions are comparable or provide more functionality for lower costs. We do not see the 50% cost benefit argument at all and in our opinion shows that Exchange 2010 is a very viable multi-tenancy solution for ISPs. One very important difference we want to point out is the available storage per mailbox. This tended to be a lot (several factors) more with Exchange than with Zimbra, without heavily impacting the price. This fact alone suggests that Exchange can be a very viable groupware solution to ISPs. Below is a list with direct links:

Final wordsThis concludes the author’s feature comparison, but there are still some important elements missing, like product support, directory integration, IPv6 readiness, traffic management (e.g. ethical walls) or IRM. Also, what about integration or support of Unified Communications technologies, like single inbox – including voicemail – or voice access to mailbox?

Now don’t get the impression we want to condemn Christopher for trying to compare both products, even though by reading just the header and counting the numerous VMware-related logos on the site we were a bit hesitant regarding what the “conclusion” would be (we have a saying here, We from WC Eend recommend WC Eend).

We do appreciate good comparisons, because it can shake up our opinions of what is and what should be with Exchange and start interesting discussions. It‘s also an opportunity to learn about similar products. We believe competition is healthy and comparisons can be educational; It can help companies make a better fit for their needs and budget, or at least provide a starting point.

It is however crucial for a fair comparison that the facts, conclusions and opinions stated are correct and sound. Unfortunately, this is not the case with this article. There are numerous factual errors and most opinions stated are poorly argumented. To add to that, the author uses a feature list which can be found on the internet in several places, like here. This may be an indication authors are copying content, without knowledge or cross-checking facts.

Therefore, with the information provided in Christopher’s blog post, one can’t conclude that Zimbra is an adequate replacement for all environments, Enterprise or SMB. Also, we do not see any indication that Zimbra is better suited for multi-tenancy by ISPs. If anything, we think we have shown that Exchange is a more than capable, competitive and well-though product.

You’re invited to comment or share your opinions in the comments below.

Personal note by Dave Stork

I have been working very actively with Exchange since Exchange 2003 and seen it grow into what can be defined as the market leader in Enterprise groupware software (see for some older sources this PDF here). I also some user experience with Zarafa and Kerio Connect, with the latter even some administrator experience.

For full disclosure, I work for a company that is Microsoft Partner and Zarafa partner. I myself am specialized in Exchange deployments and only have end-user experience with Zarafa, however that doesn’t prevent me being openly critical about the product or articles when it is compared with Exchange. I have blogged about a new Zarafa web interface and commented on two Dutch pro Zarafa articles. See those Dutch articles here and here, and for the (Microsoft Translator translated) English pages see here and here.

My motto is that my criticism, although possibly harsh, should be rationally sound and unbiased as possible. I hope that it is apparent in this blog post I’ve co-authored and for that matter in all the things I do.