The ZFS NAS Box Thread

For some time now, I have been facing the fragmentation of my "Bulk Storage" among 3 PCs due to a number of factors - using different PCs at different times to run downloads, lack of space forcing me to split things up among the PCs, and lack of space forcing me to burn things to DVDs, where it is less accessible.

I have also recently moved to California, where electricity is expensive, and even my underclocked X2 box with the gfx card slowed wayyy down for 2D pulls a good 150W at idle.

So, my thinking has recently turned to building a NAS box - something in the 50-60W range, probably headless, that can sit in my closet next to my router and hold all my files and run all the larger downloads, etc. Size around 1.5-2TB for now.

I intend to use ZFS as the primary file system for this box. with RAIDZ providing the raid support.

I see a lot of questions around here about NAS in general, and a few about ZFS in particular, so I was hoping we could have a good discussion.

Then whatever box I end up building, I will provide photos of the build, a cheat sheet for getting the software up and running, and whatever benchmarks (within reason...I have to go to work after all) the Hive Mind desires.

So... thoughts on ZFS, while I dig out the hardware config I was thinking of?

How many physical drives do you want to start out with, how quickly will you need to expand, and what kind of IO are you expecting?

Your main decisions up front will be RAIDZ/RAIDZ2 vs mirrored stripes. They have very different performance characteristics, storage overhead, and expansion characteristics. For instance, you currently cannot expand a RAIDZ device by simply adding another drive - you can either add another RAIDZ vdev to a pool, or you can replace all of the drives in the RAIDZ vdev with larger ones. Mirrored configurations have fewer limitations, plus they are potentially bootable.

Once you have an idea of your pool configuration, you will know the number of drives you'll need right away and the number of drives you'll need in the future. The next trick is figuring out how to fit them all in a single case and/or connecting external storage with a controller / enclosure combo supported by Solaris / OpenSolaris / FreeBSD (eventually). This can be quite difficult, especially if you are on a budget.

Finally, you need to work out a strategy to replicate / back up all of this data.

ZFS Best Practices Guide is an excellent resource, I expect what would be valuable is digging through it and picking out the information that is relevant to smaller / budget configurations.

Well, initially I was thinking of 4x500GB in RAIDZ, which would let me stay within the onboard SATA controller capability of most modern motherboards, as long as I use an IDE boot drive (already have one to re-use for that).

At some point I might want to add another 4x(whatever) (4x1TB? ) when the price of drives comes down, so I was looking at cases with enough 3.5" + 5.25" bays to handle about 9-10 drives, perhaps with the addition of some 5-in-3 racks and a controller card in the future.

At this point I am learning towards the enterprise Solaris 10 distribution for stability reasons. I've done a practice install in VMWare and was able to get it up and running pretty quickly.

Max load might be a few downloads and a single read operation over the network from one of my other PCs. I was planning on using Samba.

I was looking at this bad boy, not for its form factor, but for the fact that it's cheap, has lots of on-board SATA connectors, and has on-board video. (There are other motherboards with more SATA connectors, but they lack on-board video, which is a requirement for me given the case I've got for it.) I'll slap in the cheapest C2D I can find and 2GB of memory (ZFS is greedy that way). For disks, I'm thinking 5x500GB, plus a 250G boot drive (already got it, otherwise I'd go smaller), and possibly a SATA DVD burner. Like F16PJ, I'm going to run it headless, but eventually I'll hook it up to a KVM in case it needs console attention.

[ One concern I have here is that on-board SATA probably won't do sequential spin-up, but I'm hoping that won't be too much of an issue. Thoughts? ]

I'm leaning towards FreeBSD because I'm more familiar with it, and I think it's more mature for running random applications like BT clients and whatnot.

I plan to serve home directories via both NFS and Samba, each from their own partition, thus using ZFS nifty quota and reservation magic. I'll have other, general shares for things like music, movies, books, uploads, etc, but I haven't yet decided whether those will get separate partitions or not. I'm leaning towards probably; that way I can segment administration privs more conveniently. (Not because it's practical, of course, but because it's fun.)

Mostly I'm waiting for some retail giant to have a sale on hard drives, I think. That, and for FreeBSD-7.0 to be released.

I went with Linux and the /dev/md device in RAID6, formatted using ext3, on Fedora Core 6 for my new NAS. I can grow the array, add spare devices, and hotswap in new disks with relatively little effort or downtime. What benefit would ZFS/RAIDZ provide over ext3/RAID6?

Originally posted by IceStorm:Why ZFS over ext3? Why RAIDZ over RAID5 or RAID6?

I went with Linux and the /dev/md device in RAID6, formatted using ext3, on Fedora Core 6 for my new NAS. I can grow the array, add spare devices, and hotswap in new disks with relatively little effort or downtime. What benefit would ZFS/RAIDZ provide over ext3/RAID6?

The layering involved in a normal RAID setup scares me. You have a hardware or software raid solution, LVM for large drives on top of that, and file systems on top of that.

ZFS collapses all of that down into one system. You essential tell it "hey, use these drives, in this mirroing/striping/raid setup, and give me filesystems with these names" and that's it. Literally 6 or 7 commands from blank drive to working system.

It's also transactional, and the checksum for each piece of data is stored in the parent node, so it can detect bit-rot where a normal file system wouldnt.

If energy usage is really a concern, wouldn't you be better off NOT RAID-ing your drives together? I mean - I only run a lowly FreeNAS box (2 drives) and a SimpleShare (1 drive) device, but when I access such networked files I'm only going to spin up a single drive, whereas a RAID-5 configuration is going to spin up at least three drives.

It's only a concern to where I know from experience that an nForce430/GF6100/Brisbane @ 1.9 combination will idle at 50-80W with a single drive, which is less then 1/2 what any of my other desktops idle at.

Even with 4 drives at idle it should still be less.

Besides, having 4x500GB drives storing a bunch of hard-to-replace (not irreplacable, that stuff gets backed up) media files with no redundency is just asking for trouble.

Originally posted by F16PilotJumper:The layering involved in a normal RAID setup scares me. You have a hardware or software raid solution, LVM for large drives on top of that, and file systems on top of that.

That's only if you want to. Linux supports large volumes in the kernel now, so you don't need to go with an LVM middleman. On my system it's just the /dev/md0 device and the ext3 filesystem.

quote:

ZFS collapses all of that down into one system. You essential tell it "hey, use these drives, in this mirroing/striping/raid setup, and give me filesystems with these names" and that's it. Literally 6 or 7 commands from blank drive to working system.

Creating a RAID volume with Linux is two commands. The first is to have mdadm put a bunch of disk together into a RAID device, the second is to format the array.

quote:

It's also transactional, and the checksum for each piece of data is stored in the parent node, so it can detect bit-rot where a normal file system wouldnt.

and wrt growing the array, i'm pretty sure this is in development. but even if it never happened, building a second zfs array for greater capacity is a _small_ price to pay for end-to-end data integrity. i personally tend to completely rebuild arrays far more often than i expand them due to the ongoing rapid price drop in higher capacity hard drives. it almost always makes more sense to replace older hard drives due to the benefits gained by increasing storage density.

Originally posted by F16PilotJumper:It's only a concern to where I know from experience that an nForce430/GF6100/Brisbane @ 1.9 combination will idle at 50-80W with a single drive, which is less then 1/2 what any of my other desktops idle at.

Elecy is not my prime concern, here in the North West of England, but is becoming more of an issue these days.Check out the Abit board, it's pretty new. Based on the new Nvidia 7050PV+630a chipset. Has built-in Geforce 7 video with support for an analog *and* a digital display at same time. It largely supercedes those gf6150 boards. It's micro ATX and should thus consume less elecy.One thing I noticed was that you have no boot drive protection? Lose that and you lose access to your data until the drive is replaced and you have restored the data to it. that's not a good solution You don't have to use whole drives for ZFS, you can use slices i.e. a small slice 1 on every drive can contain the small ufs /boot needed to have root on ZFS under FreeBSD and also block device swap. In your config you could have a 1GB slice 1 on each drive and 499GB as slice 2. Make your ZFS pool from the 4x499GB slice 2 i.e. "zpool create tank raidz ad0s2 ad1s2 ad2s2 ad3s2".Make 2 gmirrors (that's GEOM mirrors not zfs) from the 4x 1GB slice 1; i.e. gm0=ad0s1+ad1s1, gm1=ad2s1+ad3s1. On 1st mirror put a ufs /boot filesystem just to enable booting into the root which is on zfs. On 2nd mirror put swap. Thus you can still boot after single drive failure and you save the power consumption of your boot drive. Cheers.

Originally posted by besonen:and wrt growing the array, i'm pretty sure this is in development. but even if it never happened, building a second zfs array for greater capacity is a _small_ price to pay for end-to-end data integrity. i personally tend to completely rebuild arrays far more often than i expand them due to the ongoing rapid price drop in higher capacity hard drives. it almost always makes more sense to replace older hard drives due to the benefits gained by increasing storage density.

You can grow a zfs zpool. What you cannot do is increase the number of drives in a zfs raidz. What you can do is increase the number of vdevs in a pool. Also you can increase the drive size in a raidz.i.e. you have a raidz made of 4x500gb which is the only vdev in your zpool giving effective storage of 1500GB. You could increase this zpool size by:

1) buying more drives, creating another raidz or mirror and adding that new vdev to the zpool. The new space is automatically used by all filesystems in the zpool. i.e. buy 2x500GB drives, make a zfs mirror, giving 500GB effective storage, add this mirror to the zpool, giving the zpool a new total effective stoarge of 2000GB.

2) replace all the drives in the raidz, 1 by 1 (waiting for zfs to reconstruct the data on the new drive), and when you have replaced the final drive the zpool will "magically" increase in size. i.e replace the 500GB drives, 1 by 1, with 750GB drives, and when you finish the zpool effective storage will jump to 2250GB.

I just saw that Infrant was bought by Netgear. Oh well. I was going to recommend he just get an Infrant unit, but with Netgear now owning the company, I don't know. Also, no ZFS, but it does support ext 2 and 3

You can grow a zfs zpool. What you cannot do is increase the number of drives in a zfs raidz.

This led to me to think how many people actually add drives to their RAID array as opposed to those who just build a new array, which further led me to think about the potential difficulty of finding matching drives after the initial purchase, which led me still further (and back to the this thread): I know hardware RAID generally likes all drives to be as similar as possible (size, manufacturer, firmware, etc), but does ZFS care? I would venture that it doesn't so much, but I haven't found anything definitive either way.

Oh, did I ever post the story about the client that 'cooked' two PC's (consecutively) because they put them in the closet? It was a pantry closet and the boxen were used in the kitchen. The client wanted the computer 'out of the way'. (This was so only an LCD, keyboard, and mouse would be visible on the cute little receipe desk.) The heat from the PC was making the chocolate chips in the cookies soft.

I currently run ZFS on a Xen domU and pass it five 180GB drives which it runs in RAIDZ. My only complaint is that RAIDZ is not expandable right now. I can't add a 6th drive without backing up all the data and re-creating the pool.

Beyond that, I absolutely LOVE ZFS and find it a joy to work with since all the commands are simple and powerful.

Originally posted by tam:which further led me to think about the potential difficulty of finding matching drives after the initial purchase, which led me still further (and back to the this thread): I know hardware RAID generally likes all drives to be as similar as possible (size, manufacturer, firmware, etc),

That limitation is entirely arbitrary. There is no problem from a software point of view, so long as the replacement drive has at least the same number of sectors as the one it replaces. Any more just get ignored. It really is poorly programmed RAID software which insists on the drives to perfectly match.

quote:

but does ZFS care? I would venture that it doesn't so much, but I haven't found anything definitive either way.

Indeed ZFS does not care. It does not function at the drive level per se, unlike hardware RAID controllers. Software RAID (at least that in FreeBSD) does also not care about drives. They are all GEOM based and as such are just given some device which to store data on. That may be a complete disk e.g. ad0, a slice of a disk e.g. ad0s1, a partition within a slice e.g. ad0s1a, a GEOM stripe of disks e.g. /dev/stripe/gs0, a GEOM raid5 array /dev/raid5/data, etc., etc.. The interface with the data is entirely abstracted.

e.g. if a device within a zfs pool fails, and that device was previously a GEOM stripe, say /dev/stripe/data on two small old disks, you are not required to replace that device with another of the same type. You could buy a large single unit and use that to replace the old device:

zpool replace tank /dev/stripe/data /dev/ad6

This is great for recycling old drives when initially constructing your zfs raidz, but later replacing them with large modern drives.Cheers.

Originally posted by tam: but does ZFS care? I would venture that it doesn't so much, but I haven't found anything definitive either way.

No. Like Linux software RAID, it will simply use the lowest common denomenator. So if you have one 80GB drive, and two 120GB drives, your RAIDZ will have 160GB of storage space. (n-1)*80 = 160.

I buy your answer, but not so much the justification. I was under the impression that some hardware RAID controllers could get their panties in a bunch if you fed them drives of the same size but with different eg, firmware.

I buy your answer, but not so much the justification. I was under the impression that some hardware RAID controllers could get their panties in a bunch if you fed them drives of the same size but with different eg, firmware.

One scenario that I've heard goes like this:

Buy N identical drives with the same firmware. Build a RAID array using every sector of each drive.

Months or years later, a drive in the array fails. Buy another drive with a later version of the firmware. Attempt to replace failed drive with new drive, but find out that the new drive is slightly smaller than the old drives by a few sectors. RAID array refuses to rebuild using your new drive.

Of course you can probably rebuild with a larger drive, if you have one around. Or you can avoid the problem by telling the RAID controller to leave a little bit of free space when doing the initial setup.

I sure hope that closet is air conditioned. But other than that I think the concept is a good one. This is essentially what is being done more and more at the enterprise level, architecture wise.

- AndrewZ

Door is open w/ceiling fan running in main room.

This is pretty cool:

quote:

“I built a test zpool spanning 7 drives (raidz) on S10U2. The 7 disks were split between 3 controllers. I then started replacing the 18GB drives with 36GB drives, one at a time, and watched it rebuild the zpool, growing as it did.Finally, when I had all the drives replaced, I took the system down, moved the drives around on the controllers, reloaded the OS onto the internal drives on my Sun Blade 1000 workstation. I then forced a write of random data to one of the drives in the 7 disk raidz array. Then I did the zpool import - it reported the array, with errors. I did the zpool import , and it came right in, and mounted up. From there I did a scrub (and yes, it did essentially chew up all available system resources, while it scrubbed the 200+GB pool on this small and not so mighty system), and it corrected all the issues. All in all it was a good test, and I was rather impressed that it was able to juggle re-ordering of the drives, re-enumeration of the controllers they were attached to, erasure of one of the drives, all while importing the pool on a rebuilt system.”

I'd say you're crazy to spend that much on a mobo and CPU for a "cheap" solution. You don't need a lot of CPU power for this project. Buy yourself an MSI socket AM2 nForce motherboard and the slowest Sempron available. The money you save on hardware will go a long way towards the very slightly higher power bill.

On second though, I don't know about the Sempron drawing more power. At idle, the Sempron might draw less power being single core. Maybe someone who knows more than me might chime in.

Originally posted by 1966Ford:I'd say you're crazy to spend that much on a mobo and CPU for a "cheap" solution. You don't need a lot of CPU power for this project. Buy yourself an MSI socket AM2 nForce motherboard and the slowest Sempron available. The money you save on hardware will go a long way towards the very slightly higher power bill.

On second though, I don't know about the Sempron drawing more power. At idle, the Sempron might draw less power being single core. Maybe someone who knows more than me might chime in.

ZFS runs best on a 64-bit CPUs and 64-bit Solaris due to address space issues. I may go with 2GB of RAM to let it cache more too. I suppose I could go with a Sempron64, or at the very least the 'standard' 65W TDP X2. I kind of wanted to play around with the 45W part a little bit though, considering my luck undervolting my family's Brisbane core 3600 (which has been happily humming along at 50W idle, 80W full CPU + IGP load for the whole system for 3 months now with no issues). No in-OS undervolting on Solaris, so I was hoping that spending the extra coin would get me the same power envelope.

I thought of re-using my Barton 2500 box (currently on ghetto-MediaPC-duty and awaitng a new 1:1 pixel mapping LCD HDTV to drive over DVI) and adding a SATA controller card, but when I ran the numbers it only came out to $200 cheaper and that old box runs hot, hungry, and loud.

Really 'cheap' is not the primary constraint here. This will be my file server for the lifecycle of the machine, probably about 3-4 years. I may delay the build for 2-3 months so I can save up the cash to do it right, get the Seasonic PSU and 2GB of RAM, instead of skimping.

I find myself gaming less and less so I am probably going to go with a just-videocard upgrade of my 2-year-old primary gaming machine, and spend what would be spent on a new gaming machine on the fileserver instead, since having a central repository of media will improve my experience with *all* of my machines.

ZFS runs best on a 64-bit CPUs and 64-bit Solaris due to address space issues. I may go with 2GB of RAM to let it cache more too.I thought of re-using my Barton 2500 box

It seems we are planning the same kind of project.I've been doing some testing recently. The Barton 2500 is not up to the ZFS job. It becomes a bottleneck. I'd also recommend a minimum of 2GB RAM. ZFS is pretty memory hungry and it's design depends upon plenty of RAM to get reasonable performance.

quote:

Really 'cheap' is not the primary constraint here. This will be my file server for the lifecycle of the machine, probably about 3-4 years. I may delay the build for 2-3 months so I can save up the cash to do it right, get the Seasonic PSU and 2GB of RAM, instead of skimping.

Again same here. There's cheap quality stuff and there's just plain cheap. The latter is just false economy. I consider MSI to be one of the poorer manufacturers which pretty crappy support.From what I've researched the Abit board seems to be the better mATX available at the mo for this kind of project:

It looks like it is going to be the board of choice for media centre PCs, but lends itself perfectly well to our needs too. Not got much in the room for expansion, but there are a couple of pci express sockets. You can get PCI express x1 SATA cards with 2 ports for next to nothing if you need more SATA ports.Cheers.

ZFS runs best on a 64-bit CPUs and 64-bit Solaris due to address space issues. I may go with 2GB of RAM to let it cache more too.I thought of re-using my Barton 2500 box

It seems we are planning the same kind of project.I've been doing some testing recently. The Barton 2500 is not up to the ZFS job. It becomes a bottleneck. I'd also recommend a minimum of 2GB RAM. ZFS is pretty memory hungry and it's design depends upon plenty of RAM to get reasonable performance.I'd stick with the Athlon 64 x2 brisbane too, for low power consumption and probably the best bang for buck 64bit computing.

quote:

Really 'cheap' is not the primary constraint here. This will be my file server for the lifecycle of the machine, probably about 3-4 years. I may delay the build for 2-3 months so I can save up the cash to do it right, get the Seasonic PSU and 2GB of RAM, instead of skimping.

Again same here. There's cheap quality stuff and there's just plain cheap. The latter is just false economy. I consider MSI to be one of the poorer manufacturers which pretty crappy support.From what I've researched the Abit board seems to be the better mATX available at the mo for this kind of project:

It looks like it is going to be the board of choice for media centre PCs, but lends itself perfectly well to our needs too. Not got much in the room for expansion, but there are a couple of pci express sockets. You can get PCI express x1 SATA cards with 2 ports for next to nothing if you need more SATA ports.Cheers.

Originally posted by tam:I buy your answer, but not so much the justification. I was under the impression that some hardware RAID controllers could get their panties in a bunch if you fed them drives of the same size but with different eg, firmware.

Some hardware RAID controllers might. I'm really unsure and it likely varies greatly from chipset to chipset. I've only done hardware RAID once with a pair of matched drives. I know from experience exactly what Linux software RAID and ZFS RAIDZ do however, and that was our topic.

quote:

Good to see those SCSI drives are still chugging along for ya.

They are large, and I was lucky to find a good server to put them in that can handle their height, but they chug along unnoticed and uncomplaining year after year. They are one of the few hardware components I almost never worry about.

It seems we are planning the same kind of project.I've been doing some testing recently. The Barton 2500 is not up to the ZFS job. It becomes a bottleneck. I'd also recommend a minimum of 2GB RAM. ZFS is pretty memory hungry and it's design depends upon plenty of RAM to get reasonable performance.I'd stick with the Athlon 64 x2 brisbane too, for low power consumption and probably the best bang for buck 64bit computing.

What kind of performance do you really need? Your bottleneck is likely to be your network anyway. Mine is even moreso because my OpenSolaris install is running as a domU under Xen an apparently there are issues with the software NIC emulator for any domU that isn't Linux based. So my network connection is even slower than a normal 100Mbps. Even so, I never find myself getting impatient for files to move faster.

My server is a 6-way P3-Xeon 700Mhz system with 3GB ECC PC100, so Opensolaris/Xen does get its own CPU. But its only 700Mhz and it runs fine IMO.

What kind of performance do you really need? Your bottleneck is likely to be your network anyway.

I have Intel gigabit NICs using 16KByte jumbo frames and the disk subsystem is definitely the bottleneck.It has no problem maxing out a 100Mbit/s connection. However, it averages around 18MByte/s with an Athlon XP 2500 Barton and ZFS. Before experimenting with ZFS I had a GEOM RAID5 software array on the same hardware which could get about 40MByte/s burst over samba onto an XP box, but more like 25MByte/s normally.So I have some idea what the same hardware can do. It should do more too. The current system shares the PCI bus between the gigabit adapters and hard drives. With the Abit AN-M2, all the drives will be on the PCI Express bus and just the NIC will be on PCI. Hopefully it should manage 60-70MByte/s.Cheers.

So the network is not your bottleneck. However, you didn't answer why you NEED that performance. Are you really going to notice on a day to day basis? I'm only asking because I know my performance is closer to 10MBytes/s or less and I never notice myself waiting for files to move unless I'm copying 13GB DV files, and I do that rarely and the transfer is still the fast part since after I transfer I'm going to be transcoding which takes hours.

Huge throughput makes sense in a business situation where you have tons of users. On a home network it is often of negligible benefit and secure storage is much more important.

Not sure I like the looks of the early reviews of that abit board on the 'egg, and Solaris support for the NF6xx/GF7xxx chipsets is probably minimal at best...

I have to apologize to anyone watching this thread, but my NAS build is going to have to be shelved for the foreseeable future, primarily due to unforseen issues with vehicular transportation and CA smog checks.

Also, I re Kill-a-Watted my current X2 system and it only draws 122W at idle w/downloads running. I swear it was 190W before, but it is possible I didn't re-measure after setting up RMClock CPU scaling and using RiveTuner to underclock my 6800GT to the minimum possible speed in 2D mode. At any rate, I currently see little power cost savings in building a new box.

Also, the 7200.11's in sizes below 1TB should be coming out soon, so I see little sense in buying 4x500GB of 7200.10s when there are newer drives with higher densities just over the horizon.

Give me two or three months and I may revisit this, and of course feel free to continue the discussion. I still don't think that adding more and more non-redundant storage to my main box is a good answer. Someday a HDD failure will get me and I will have to replace a bunch of hard to find music or movies. While these items are technically non-critical and replaceable, with tools like ZFS around I see no sense in taking a chance.

Originally posted by Alton:So the network is not your bottleneck. However, you didn't answer why you NEED that performance. Are you really going to notice on a day to day basis? I'm only asking because I know my performance is closer to 10MBytes/s or less and I never notice myself waiting for files to move unless I'm copying 13GB DV files, and I do that rarely and the transfer is still the fast part since after I transfer I'm going to be transcoding which takes hours.

I'd like the server's storage to be just as fast as disks connected to the local machine. Burning a 16x DVD requires ~22MByte/s. I do that pretty much every day. So yes, I would notice it day to day if it couldn't provide at least 22Mbytes/s.But I'm not saying I need the even higher speeds I mentioned. I'm was talking about max speed. Tweak to get the max as high as possible and you help keep up the average, which is generally what you'll be seeing.Gigabit cards are cheap now, so I don't see why they shouldn't be used.

quote:

Huge throughput makes sense in a business situation where you have tons of users. On a home network it is often of negligible benefit and secure storage is much more important.

How does having fast storage make it less secure, in my case? Would you think my storage more secure if I replaced my gigabit cards with 100MBit/s? Afterall we are talking about moving storage onto ZFS here, so data security *is* paramount

With a gigabit switch $70 or less, Cat6 cable the standard, and high quality Intel Pro/1000 NICs $25, there is no reason to have the network be a bottleneck unless you can't physically run cable (like at the last place I lived).

Originally posted by gildenman:How does having fast storage make it less secure, in my case? Would you think my storage more secure if I replaced my gigabit cards with 100MBit/s? Afterall we are talking about moving storage onto ZFS here, so data security *is* paramount

Actually, I was confusing your posts with someone elses (or merging them in my head anyway) and though you were recommending against ZFS for performance reasons. ZFS IS slower than ext3, but worth the extra slowness.

I've never burnt a DVD over the network, and you are right, with my setup I wouldn't want to try. It would be coaster city. I do all that locally.

Actually, I was confusing your posts with someone elses (or merging them in my head anyway) and though you were recommending against ZFS for performance reasons.

No worries

quote:

ZFS IS slower than ext3, but worth the extra slowness.

The checksumming of data is something I'd not really thought much about before ZFS. I've now been playing with a test FreeBSD ZFS setup for a couple of weeks now. I tried turning off checksumming to provide some extra speed on an Athlon XP, but it made little difference and of course that's a bad idea. I thought I'd try a zpool scrub yesterday to recheck all data on the drives. It turned up 2 checksum errors! Both automatically fixed by ZFS, of course.That's after only a couple of weeks. All drives are reporting no SMART errors using the wonderful smartd (which monitors all drive SMART attributes and can configurably perform a SMART short test everyday and a SMART long test every week). (Smartd can email you when any attribute changes , giving excellent pre-warning of impending drive problems. That extra level of protection is a really great feature.)So it's not the drives. This error came from somewhere else. When I think about it, drives that have failed in the past always end up with bad sectors on them where those files are effectively lost. Using ZFS for media storage really is the only way to do it.

quote:

I've never burnt a DVD over the network, and you are right, with my setup I wouldn't want to try. It would be coaster city. I do all that locally.

With BURNproof, coasters are not a problem. It's just that the burn would take much longer. They take long enough as it is A proper media storage server should just appear to be as good as local storage i.e. you should just forget it's there. Previously that was not possible, but now with decent gigabit it is becoming a reality. I initially tried it out with the Realtek gigabit NICs built into two motherboards. However, those things are awful. They just don't work as they are supposed to with any real load. I spent days changing this and that trying to get them to simply work properly. I ended up buying two Intel gigabit cards, simply slotted them in and all problems just vanished. Great hardware, great drivers.

Previously that was not possible, but now with decent gigabit it is becoming a reality. I initially tried it out with the Realtek gigabit NICs built into two motherboards. However, those things are awful. They just don't work as they are supposed to with any real load.

I have my whole house wired to a 24-port managed switch and most of my machines have Intel Pro-100 cards. I just can't afford to upgrade to gigabit. Though at times it would be nice.

Just one point - for all those arguing that RAID expansion isn't critical. For my needs, unfortunately it is... I'm in the process of migrating my "file server" from an OS X server box, running a raid 0+1 config (4x160gb drives), over to an ubuntu box with raid 5.

I'm doing this "on the cheap" as I'm closing on a house in 3 weeks, so here is my strategy:Break the raid 0+1, remove one of the stripes. Data is intact on the other, albeit only in a raid0 config.

Purchase 1 additional 160gb drive, create a 3 disk raid 5 array on the new box. (usable size will match the raid 0 on the old machine).

Copy data from the stripe to the new raid 5 array.Verify data.

Break the stripe on the old machine, add the two drives one by one to the raid 5, expanding it.

I'll end up with a 5 disk raid 5, with 2x the capacity I had on the old machine.

Now... having said all of that... for 640gb, I'd almost be better off spending the $220 or so and buying 2 500gb drives and just mirroring them. If cash were not so tight right now, I'd just give in and do that.