Backing Up to External USB from FreeNAS

Newbie

From reading all the Googled threads on this topic, I feel like I'm going to get a pile of abuse for this post... but stick with me.

Backing up to an external USB drive directly from the FreeNAS host hardware is a sought after feature and a real use-case for home users. It has two advantages:
1) it is a backup, and if the sh*t hits the fan in your main Zpool any backup is better than no backup
2) it is easily man-portable and can be swapped in and out to store off-site or quickly grabbed in the case of a disaster.

The typical community response at this point is that plugging USB drives into FreeNAS should be avoided and you should do the backup from a client PC on the network. Fine, but some people don't have PCs always-on in their house, or perhaps their only PC is the laptop that they occasionally need to take out of the house yet still want backups to continue. For whatever reason, lets keep the faith that as many threads suggest, backing up to a locally attached USB drive is actually a sought after feature for home users no matter how unsuitable it might be for enterprise level deployments.

So the next, often touted, issue is that supposedly ZFS over USB is baaaaaaaaad. Though even the experts don't seem to agree on exactly why this is. However the argument that ZFS is not particularly compatible with any subsequent data recovery use-case (reading the files on your Windows laptop for example) is compelling enough to rule out ZFS as the file-system of choice for this purpose. In any case, it probably makes more sense in a home-brew one-shot backup solution to have the backup on a different file-system to avoid a fs level corruption ruining everything.

So, the feature request is now getting narrowed down. Can we backup to a non-ZFS, locally attached USB device?

Most people now seem to turn to ext3/4, the Linux-Lover's fs of choice. There is a lot of commentary out there from FreeNAS ultras about why ext is baaaaaaaaad. However the simple fact that modern ext4 is not supported on the FreeNAS FreeBSD current kernel is enough to rule this out too.

What's left? Is there a file system that FreeNAS/BSD can mount, is generally considered to be 'okay' for use on USB drives and readable by most consumer OSes?

Ruling out NTFS for Linux/BSD write compatibility and FAT32 for the file-size limits, it would appear to leave one contender... exFAT.
Supported natively in Windows and MacOS, supported in Linux with exfat-fuse and supporting in FreeBSD with fusefs-exfat.

Newbie

Installing fusefs-exfat in a jail or backing up via a VM is not an option as the USB passthrough does not work.

It would seem to need fusefs-exfat installed on the FreeNAS base system. Which is going to need either a custom compile of FreeNAS, or for fusefs-exfat to be included in future releases. With my limited experience of compiling embedded BusyBox systems I think I can probably get this up and working, however it's not ideal as it will require constant maintenance to keep it aligned with the main FreeNAS releases.

How can I get this added as a feature request for future FreeNAS releases?

The reason people frown on USB is because it dosent include the whole sata bus. ZFS is an awesome filesystem when it is allowed to control the disks, take that away and it is crippled.

Having said that, I still believe it’s far superior to anything else and there is a resource page here about having a USB drive backup pool. There are some considerations to account for such as scrubs and how to safely detach it.

I expect the response would be that FreeNAS is an appliance that doesn't officially support this. And this would be correct.

That being said, I agree that many home users running smaller systems need to back up to usb disks. I'm one of them. Having spent a fair amount of time looking at options to achieve this, there are only 2 possibilities I've come up with to back up to non-zfs drives:

1. Pull the data off your FreeNAS using another device. This could be a simple device, like a Raspberry Pi. This gives you a lot of flexibility (use whatever drive/filesystem you want), but may be slow because you may be limited by your network transfer speed and small devices like RPi's are typically further constrained in terms of network and usb performance.

2. You can actually mount NTFS-formatted drives on a FreeNAS host. This uses the built-in ntfs-3g and fuse. Performance probably won't be all that great (haven't tested this). If you think of FreeNAS as an appliance, this is a hack that may stop working at any point.

Nevertheless, to do this, add the following to /boot/loader.conf to automatically load the fuse ko:fuse_load="YES"

Reboot.

Then, you can create a mountpoint (note: the mountpoint probably won't persist across reboots):mkdir /mnt/<your_mountpoint>

Newbie

To be clear, my use-case for this is as follows. I have about 10TB of data, this roughly breaks down as follows:
A) 5TB of stuff that would be annoying to lose but relatively easy to replace (rips of media, software library, things that can be re-ripped or re-downloaded pretty easily).
B) 4TB of stuff that would be really hard to replace but possible with time and effort (rips of my home-video tapes, old digitized photos, the seemingly bottomless pool of 'unsorted' digital photographs from the last 10 years that are also still on SD/CF cards somewhere)
C) 1TB of stuff that is truly irreplaceable (documents, the 'family album' digital photographs, code projects etc).

A - I don't backup, I have a mirrored pool to mitigate the pain-in-the-ass that losing this data would be, but I don't want to spend any money on it.
C - Is on a mirrored pool and I am remotely syncing this with S3 in two AWS regions. This needs a proper off-site and I'm willing to pay for it.

It's the data in B that I look to the USB solution for. I don't care enough to pay for offsite storage, but I care enough that I want to be able to add one more layer of protection from a NAS corruption, and rescue this easily in the event of a fire/disaster.

My old Netgear NAS, despite all it's other failings, had a good solution to this. You could schedule backups jobs to NTFS/FAT USBs from the UI. It was robust enough that it would fail gracefully with an alert if you'd forgotten to attach the USB drive for the backup window.

This guide doesn't mention USB, rather e-SATA. I presume the principles would be the same.

I think the main issue with ZFS for USB backup is that it invisibly performs a lot of maintenance on the FS which is great in a high-availability system, but not so much when you might pull the drive at any moment. If you're not actively writing to an exFAT mount then it is, for all intents and purposes, fairly safe to hot-pull the drive at any moment. With ZFS you never know whether the drive is in the middle of a scrub or another maintenance task and could be corrupted by a hot-pull at that very moment. Or am I misunderstanding ZFS? Is it safe to hot-pull drives like this?

Newbie

Not safe but safer. If your USB attached exFAT drive is just doing a short nightly rsync task, then it's likely that for the rest of the time it's gone into power saving mode, retracted the drive heads and spun down. You're much better off pulling this drive in an emergency than one that might be doing a maintenance task.

Newbie

Would it be possible to have a pair of ZFS formatted USB drives that have the same gptid/zpool config (perhaps initially cloned via DD?) so to the OS, look like they are the same Drive then write some scripts that on a cron:

mounts the USB drive
imports the zpool from the USB drive
rsyncs the required files
exports the zpool
unmounts the drive

That way, you could be sure if you pull your USB outside of a backup window, it's pretty safe. You can swap the drives around on a schedule and offsite one at a time. You could even work into the script a weekly scrub.

Newbie

There shouldn't be anything to do to mount the usb drive, you just need to import the pool.

The script in the resource linked to is quite a fitting example. The author has two different disks with different pool names (you could probably create 2 pools with the same name since you wouldn't have them both imported at once; I wouldn't clone the disks though). You can see how the mountpoints are derived etc.

There's no reason you can't automate the entire process. Depending on your schedule, maybe add in some notifications (email) to let you know once the backup starts and finishes so you know when it's safe to pull the disk.