Hybrid View

Hot backup question on open source?

Hot backup question on open source?
I have been playing with Zimbra for a couple of weeks now and am enjoying it very much. I have a question about hot backups on the opt/zimbra directory.
Tim Brennan mentioned that bad things could happen if the database changes due to an email floating in while a backup is in process. Did he mean it could destroy your mail server or just give you a bad copy to restore from?
I use webmin file system backup for timed daily backups to a USB external just to take off site if needed and I have a RAID mirror for real disaster recovery. The USB drive is just a last resort. Is the hot backup a threat to my stable system? Some input is greatly appreciated.

Hi,
Currently Zimbra OSS does not support Hot Backup, according to Zimbra. Zimbra says the database state will not be consistent if you use Hot backup. You can though get some scripts that will automatically shutdown all Zimbra processes and perform the backup. One of such script is on http://swedcore.net. Or rather type backup or script in the forum search.
There a vote going on for Hot backups in Zimbra OSS. Please support.

HOT Backups in the community edition

I had the problem of backing up (daily) a group of 50 users' mailboxes and taking the server down by doing a cold backup is not really an option. I am using the community version. I am not a Linux guru by any means, however was able to fashion the following scripts that function just fine for my needs, and perhaps can help someone else looking for a similar functionality. I am also up for any "professionalization" that can be done by the real Linux guys seemingly abundant in the Zimbra cosmos.

I am using several steps here. First, a script to export a single user account using zmmailbox. This is used by the next script.

#!/bin/bash
# get a list of all the accounts and create a script to
# execute the baccount.sh script with the email address
# of the user as a command line variable
#use zmaccts and extract only the email address from the result
#then concatenate the "/backup/./baccount.sh <emailaddress>" string for each account and write it to the file
/opt/zimbra/bin/./zmaccts | grep "@" | awk '{print "/backup/./baccount.sh " $1}' > /backup/accountbackup.sh
# change permissions on the newly created script
chmod 755 /backup/accountbackup.sh
# Execute the newly created script - this creates a tgz file for each account
/backup/./accountbackup.sh
# this mounts an SMB drive of a neighboring win machine and copies the backups over - could be anything
mount -t smbfs -o username=username,password=password //windowsMachineName/windowsShare /data
cp /backup/accounts/*.tgz /data

The above script queries Zimbra for a list of users and generates a shell script called /backup/accountbackup.sh,
changes permissions on the newly created shell script and executes it, calling the baccount.sh script for each email address in the system and subsequently generating a complete mailbox export for each account.

In my example, it then mounts a remote smb volume and copies the backups off-server.

I am sure this can all be made more generic and probably simplified, but I have found the flexibility of having these individual mailbox backups to be very very handy when coupled with a weekly full system backup. This procedure is run nightly.
The restore functions best when restored to an empty, newly created account.

Just a quick note,
This technique is only possible on 5.0.9+ but there are several reasons to only use it on 5.0.10+:
We found that minor mail store corruptions like a missing blob, size mismatch, or 0 byte db entries can cause a tar operation to halt prematurely/you’ll notice missing data.

You can determine if that is the problem by running “tar tfz account.tgz” and verifying that the listing completes without a corrupt stream error message.

This is similar to bug 31937 (another error handling one was bug 31016) - a fix for truncating or padding mismatched blobs will be included in 5.0.10 NE & we currently plan to release an updated 5.0.10 FOSS as well.

What would actually break

Hello!

I am adding to this thread as we are discussing this same issue and it's a go/no-go issue for us. Doing cold backup is not really an option, and as it stands now, neither is the commercial version*. So, i have read a little and eveyone basicly says that scary things might happend if i backup a running Zimbra install Scary things is something i genreally wan't to avoid, but it would be nice to now in mor detail what these scary things are.

Say a do an rsync on the entire /opt/zimbra while it's running, and the later on just stop zimbra, rm /opt/zimbra and moving the backup-copy into it's place. I understand that there would be som mails in mysql that is not on disk, and vice versa, but would anything more severe happend (a few mails that "cannot be opened" is pretty acceptable given that the entire RAID would have failed to get me there)? Is it possible that, say, the database would totally corrupted in the backup and the backup totally unusable?

I have tried this operation on a test system under moderate load a few times and it works acceptable every time ....

And please exuse my english

//Joakim

- - -
*(we would only use the backup-feature and paying yearly for 103 accounts (not users, mind you, accounts) just to get a backup feature seems a little har for managament to accept)

Wait I re-read that, you're saying you're not going to take a 2nd rsync with Zimbra shutdown; that really is just ripe for a multitude of issues - from PID's to database, index, and store changes. Yes you may get it up and running with slight finesse in a pinch, but I would still take that cold copy occasionally. By doing the 1st rsync you can minimize both your downtime (& risk) considerably.

Thanks for that (really quick) reply, active forums are nice.. Yeah, i actually read the script you are referring to (and a few other) and i think it's a really slick solution, two syncs to minimize downtime and let the occational mail just wait on the sending MTA for a few minutes. That downtime is not accepable according to my spec though (say we have a 400k mails, just to STAT those files would take a while no matter how i do it).