Who was the doofus who decided that the default layout for a Linux software RAID array puts the RAID superblock at the very end of the device or partition? The upshot of this is that the system can have a hard time telling the difference between an array that is built on raw devices, versus an array that is built on partitions that extend to the end of the devices. If you rely on the system's automatic array detection logic, it may get it wrong; this can have some rather unpleasant (and very puzzling) results.

I managed to get the MD subsystem on my work desktop into a very confused state because of this, and ended up accidentally corrupting (and needing to rebuild from scratch) the RAID-1 array I was setting up. No real harm done (no data lost), but man was that a head-scratcher!

When I finally figured out WTF was going on, I forced it to use "version 1.2" superblocks when I rebuilt the array. This puts the superblocks at a fixed offset from the *start* of the device or partition. Seems a lot more sensible to me!

Moral of the story: Always use the "--metadata=1.2" option if you're building a RAID array on partitioned devices!

The years just pass like trains. I wave, but they don't slow down.-- Steven Wilson

axeman wrote:Good to know, dude. I'll still take it over fakeraid any day.

Oh yeah, me too. Not the least because fakeraid typically doesn't work in Linux anyway, because the drivers are (usually) Windows-only. Now that GRUB 2 supports booting from software RAID arrays, you can even use software RAID for the boot volume...

The years just pass like trains. I wave, but they don't slow down.-- Steven Wilson

mentaldrano wrote:JBI, why would you RAID partitions instead of devices? Surely you have two drives in this machine that are in RAID-1? Or are you RAIDing two drives that have one partition on each?

Yes, RAIDing two partitioned drives.

The reasoning behind doing it this way was to insulate myself somewhat from differences in drive sizes. If the array is built on raw devices, and a drive needs to be replaced, you can get tripped up by the fact that different brands/models of drive may have slightly different capacities. If a replacement drive is smaller than the other drives in the array, it won't work. By using partitions under the RAID, I can make the RAID array a percent or so smaller than the physical drives, thereby sidestepping that potential landmine.

The "gotcha" mentioned in the OP hit when I got the bright idea of "hey, I could partition the pad space too, and make a small scratch array out of that". Shortly thereafter, all hell broke loose.

I retrospect, maybe I should've just used the --size option when creating the array, instead of trying to trick it using partitions.

notfred wrote:Agree with mentaldrano. When I've done RAID on Linux I do it on the bare devices and then use LVM to produce the equivalent of partitions within the RAID device.

My home system is actually booting from a RAID-1 which is built on partitioned drives, with LVM on top of *that*. I don't recall if I was even given a choice of whether to use partitions or raw devices when I set that one up; in that case the disk setup was done via Ubuntu's "Alternate Install" CD.

I'm not sure why the home system *doesn't* get confused, since the partitions extend to the end of the disks, and the array *isn't* using the 1.2 metadata format. As a guess, maybe it is because on that array the type of the underlying partitions is set to "Linux RAID autodetect". I intentionally *didn't* set the partition type on the drives in the new array because the mdadm documentation claims that this is deprecated.

And yes, I agree that if I'd used raw devices I would've never seen this.

The years just pass like trains. I wave, but they don't slow down.-- Steven Wilson

just brew it! wrote:The reasoning behind doing it this way was to insulate myself somewhat from differences in drive sizes. If the array is built on raw devices, and a drive needs to be replaced, you can get tripped up by the fact that different brands/models of drive may have slightly different capacities. If a replacement drive is smaller than the other drives in the array, it won't work. By using partitions under the RAID, I can make the RAID array a percent or so smaller than the physical drives, thereby sidestepping that potential landmine.

Ugh, I JUST had to deal with this. I had a Seagate Barracuda LP 2 TB drive die on me and the replacement (WITH THE SAME MODEL NUMBER) is 100 MB smaller than the old one. I had to shrink my array to get it to sync up with the new drive.

For the record software raid (Windows or Linux) and host raid(fake raid) are basically the same. i just avoid them both because they both suck. the exceptions I've found are these: Linux software raid with ZFS is ok. Or if you're using an Intel Server board and can use ESRT2 host raid. ESRT2 is based off of LSI Raid Controllers and works very well with most Linux distros. But if RAID is a major concern, then spend the money on a server motherboard and a decent raid card. (highpoint rocket raid is not a decent raid card, on board raid is better and more reliable then rocket raid)

EV42TMAN wrote:For the record software raid (Windows or Linux) and host raid(fake raid) are basically the same. i just avoid them both because they both suck.

The generally accepted usage is that "fake RAID" means you need to load a proprietary driver. And yes, that does indeed suck. It is non-portable, and quality/reliability of the driver and RAID BIOS varies widely.

OTOH pure software RAID on Linux has gotten quite good in the past couple of years (modulo little hiccups like the one which spawned this thread). Bootable software RAID arrays don't require serious voodoo any more (they only require entry-level voodoo... ), and performance of RAID-5 is even pretty reasonable on a modern multi-core CPU.

The years just pass like trains. I wave, but they don't slow down.-- Steven Wilson