How to Get Unbelievable Deduplication Results with Windows Server 2012 and Veeam Backup & Replication!

Veeam builds innovative software that helps a business to realize the promises of virtualization. Veeam’s products support virtual machines (vm’s) running in multiple hypervisor environments including VMware and Hyper-V. The changes and improvements to Windows Server 2012, particularly as they relate to Hyper-V, and to storage, are incredibly exciting! Let me give you a great example!

Veeam Backup & Replication Deduplication

In the virtual environment, just like the physical environment, backup is essential! Veeam Backup & Replication provides simple, fast, reliable backups of your running virtual machines. It uses API sets and the Microsoft VSS process to ensure application quiescence, and total reliability. The ever growing demands on storage seem to be a constant concern to the business. There is no doubt that backups of virtual machines could impose a significant load to a storage environment. Veeam has implemented both compression and deduplication features to help recover what would otherwise be a significant amount of data storage space used to store backups of vm’s. Veeam Backup & Replication uses what is known as a “per job” deduplication, meaning that the replication is implemented on a job by job basis. This deduplication alone can yield very significant data recovery percentages depending on the contents of the virtual machines.

Windows Server 2012 Deduplication

Last month Microsoft introduced a new deduplication feature with the launch of the Windows Server 2012 operating system! Microsoft Research has created a new method of deduplication that is achieving significant data recovery rates on a “per volume” basis. Microsoft’s storage team indicates that a volume that has deduplication enabled and is storing virtual machines, or their backups, can achieve savings rates in the high 70% range. This new deduplication feature uses data broken into chunks and eliminates duplicates while adding pointers and using advanced mechanisms to provide for and prevent potential data loss. The deduplication feature also uses a data aging system to ensure that only data that is resident on the volume for greater than 4 days (4 is the default and is adjustable based on your own situation) is deduplicated to prevent the deduplication of data that is constantly changing. This is one of the most significant improvements to the Windows Server 2012 operating system and will certainly provide significant benefits to the business in the realm of storage. You can find out more about the details of this new deduplication functionality at the Windows Server Teams Storage blog.

The Test

Imagine what would happen if we put Veeam Backup & Replication and its “per job” based deduplication feature together with Windows Server 2012 and its “per volume” based deduplication feature. The results can only be called Jaw Dropping! These two deduplication features complement one another incredibly well and return results that have to be seen to be believed. The process takes an initial file size of 250 GB , and when both deduplication methods are applied, the ending size on disk is an astounding 7GB! The first time I saw it I thought there must be some mistake! Not so! It is the most compelling disk deduplication demonstration I have ever seen!

A good friend from Australia, Charles Clark, recorded the following detailed demonstration of exactly how to implement Veeam Backup & Replication along with Windows Server 2012 volume based deduplication.

Windows Server 2012 is unquestionably the best server operating system ever created! When it is used in conjunction with Veeam Backup & Replication it provides not only the best backup solution for your virtual machines, but also the most effective method of deduplicating and storing those backups on physical disks.

Veeam and Microsoft, working together to provide amazing innovations that will help your business realize the promises of virtualization and the cloud!

Sorry I know this is an old post, but W2012 does not accurately display the true “Size on Disk” when using dedupe. You can confirm this by subtracting your disks free space from Capacity. “Size on Disk” seems to only report files that are not in any way reduced through the dedupe process.

GD Star Ratingloading...

http://www.markora.net Luke

Can this software used in older version of Unix and Windows OS, whats the limitations

GD Star Ratingloading...

Aurimas N.

As I understand this post talks about storing backups on server 2012 with deduplication enabled.
Do the recommended backup job settings also apply when backuping file server with deduplication enabled (and storing on non deduplicated volumes)?

GD Star Ratingloading...

Joaquin

Since this feature is only recommended by MS (see webinar) for volumes up to 1.5TB and a dedup rate of 100GB/hour I do not see the benefit of Server 2012 R2 dedup. Our fileserver (2008 R2) has ~16 TB of data in 2 volumes and veeam produces a full backup if 14TB. I know that we have many duplicate files on that server so dedup would have been great, but it’s working only for real small volumes (in comparison to what a file server can handle).

GD Star Ratingloading...

Author: Chris Henley

Chris Henley is a Microsoft IT Professional Evangelist at Veeam focused on Technologies related to Windows Server and Hyper V. He is a published author and a regular speaker and presenter at user groups and major technology conferences around the world. He is fun and... More