Backing Up FreeNAS to an external hard drive
I love this little device: it's an iXsystems MiniNAS running FreeNAS 9.2, with tons of disk space, RAM, fast network connections, all on a low-profile device that uses precious little energy (30W). Nice! And having all my important stuff on one box not only gives me the freedom to screw around with my desktops but simplifies and centralizes the work that goes into backing up my information.
It's tempting to be lulled into security by a hefty NAS running the ZFS file system on RAID-Z. Yes you've got some redundancy and a resistant file system. But RAID is not backup. And here I ran into some trouble. FreeNAS gives you tons of options for transferring zpool datasets around, and since it's networked you can rsync your heart's content to other systems, but what if you just want to back the stuff up to a hard drive locally? Like an external, USB hard drive? Turns out, there's a way, but it requires a bit of Unix-foo.
Fortunately, this is FreeBSD, so lots of things are possible.
My first attempt was to write a little script that would work from the C shell, to be run by root. Rsync was clearly the right tool for this job, probably something like:
rsync -av /directory/to/sync /mnt/destination/directory
And there would have to be a mount command there too. Plugging a USB drive into the slot and watching the contents of dmesg revealed the machine was recognizing this FAT-formatted drive as /dev/da1. So as root, I tried:
mount_msdosfs -o /dev/da1 /mnt/external
The -o flag is necessary so the file system is prepped for a large drive. That seemed to work at first, but I wanted the errors to go to a log file to be sure, so I used instead:
rsync -av /directory/to/sync /mnt/destination/directory &2>errorlog.txt
That was important because I spotted a huge number of errors. First of all, the FAT filesystem doesn't recognize owners or groups of files, so I had to add the -o and -g flags to rsync to tell it to ignore ownership. But more seriously, rsync choked on any file with foreign characters, and in my music collection alone there were hundreds of files not being synced because of this issue. I went to the rsync developer's for help and didn't find any easy solutions. How could so many people be using this software and having no trouble?
The answer was in the filesystem. The problem was some combination of the MSDOS filesystem (ie, FAT) being mounted with inappropriate options, or the FAT system simply not liking the foreign characters. I tried an experiment: I opened up an rsync account at rsync.net and tried syncing the same troublesome files. No problem whatsoever; not a single error message. So that confirmed for me the problem was the file system on those external drives, not rsync itself. After some grunting and complaining, I decided to reformat the external drives to UFS and make them essentially "Unix disks"
This is no easy decision to make: hardly any system other than FreeBSD can read UFS. Linux supports it poorly and most other OSes don't support it at all. But hell, if it backs up my stuff and gives me some protection from disaster, who cares? And I'm never too far away from a FreeBSD box anyway, happily.
This blog gave me the way forward in formatting an external drive to the UFS filesystem. Catch: you need a FreeBSD box to do it, and if you don't have one, you've got to do it from the NAS itself. Dangerous, because if you format the wrong disk you hose your system. Fun times! Again, stick in the drive and monitor dmesg to see how it is recognized. In my case, it was (again) /dev/da1 (meaning /dev/da0 was probably part of my NAS: need to remember to avoid touching that one). OK, so:
fdisk -BI /dev/da1 # Format the external drive: one huge partition, bootable bsdlabel -wB /dev/da1s1 # Write standard (bootable) disk label to the 1st partition newfs -O2 -U /dev/da1s1a # formatthe partition with UFS2 and soft updates
That worked without a hitch, and was as methodical and precise as I would have expected from FreeBSD. And now, to mount it. I rewrote my little script as follows:
#!/bin/sh mount /dev/da1s1a /mnt/external rsync -av /mnt/sharkstuff/randymon /mnt/external 2> rmonbkup-error.log umount /mnt/externalIt's not perfect, but it works. So now on a weekly basis, I plug in an external drive, ssh into the NAS and run:
sudo mkdir /mnt/external sudo name-of-my-scriptIt copies everything over to the external drive and logs the errors to the error.log (but these days there are no errors, happily). Then it unmounts the drive. I have two drives and two separate data collections to back up so I have two copies of the script, one for each. Then I file the hard drives away somewhere physically separate from the NAS in case fire or water ever take out my box. And now I sleep easily! It would be no sweat to automate the script to run at a given time every Sunday but I'd have to be concious and organized to make sure the box were turned on and the right hard drive attached at the indicated time. This non-automated version works well enough.
Adventures in Unix!
Post script: I was told on the FreeNAS forums that support for the UFS file system would disappear in a future release of FreeNAS, Not sure what kind of uber-geek dick-swinging that was, but as of November 2020 I am running an up-to-date version of FreeNAS and this script still works perfectly. (If they do drop support for UFS drives, I'll simply format the external drives with ZFS and use "zfs pool send" or equivalent - cross that bridge when the time comes).
The author does not allow comments to this entry
Display comments as Linear | Threaded