HOWTO: Linux Software Raid using mdadm

1) Introduction:

Recently I went out and bought myself a second hard drive with the purpose of setting myself up a performance raid (raid0). It took me days and a lot of messing about to get sorted, but once I figured out what I was doing I realised that it's actually relatively simple, so I've written this guide to share my experiences I went for raid0, because I'm not too worried about loosing data, but if you wanted to set up a raid 1, raid 5 or any other raid type then a lot of the information here would still apply.

2) 'Fake' raid vs Software raid:

When I bought my motherboard, (The ASRock ConRoeXFire-eSATA2), one of the big selling points was an on board raid, however some research revealed that rather than being a true hardware raid controller, this was in fact more than likely what is know as 'fake' raid. I think wikipedia explains it quite well: http://en.wikipedia.org/wiki/Redunda...ependent_disks

Hybrid RAID implementations have become very popular with the introduction of inexpensive RAID controllers, implemented using a standard disk controller and BIOS (software) extensions to provide the RAID functionality. The operating system requires specialized RAID device drivers that present the array as a single block based logical disk. Since these controllers actually do all calculations in software, not hardware, they are often called "fakeraids", and have almost all the disadvantages of both hardware and software RAID.

After realising this, I spent some time trying to get this fake raid to work - the problem is that although the motherboard came with drivers that let windows see my two 250 GB drives as one large 500 GB raid array, Ubuntu just saw the two separate drives and ignored the 'fake' raid completely. There are ways to get this fake raid working under linux, however if you are presented with this situation then my advice to you is to abandon the onboard raid controller and go for software raid instead. I've seen arguments as to why software raid is faster and more flexible, but I think the best reason is that software raid is far easier to set up!

3) The Basics of Linux Software Raid:

For the basics of raids try looking on Wikipedia again: http://en.wikipedia.org/wiki/Redunda...ependent_disks. I don't want to discuss it myself because its been explained many times before by people who are far more qualified to explain it than I am. I will however go over a few things about software raids:

Linux software raid is more flexible than hardware raid or true raid because rather than forming a raid arrays between identical disks, the raid arrays are created between identical partitions. As far as I understand, if you are using hardware raid between (for example) two disks, then you can either create a raid 1 array between those disks, or a raid 0 array. Using software raid however, you could create two sets of identical partitions on the disks, and for a raid 0 array between two of those partitions, and a raid 1 array between the other two. If you wanted to you could probably even create a raid array between two partitions on the same disk! (not that you would want to!)

The process of setting up the a raid array is simple:

Create two identical partitions

Tell the software what the name of the new raid array is going to be, what partitions we are going to use, and what type of array we are creating (raid 0, raid 1 etc...)

Once we have created this array, we then format and mount it in a similar way to the way we would format a partition on a physical disk.

4) Which Live CD to use:

You want to download and burn the alternate install Ubuntu cd of your choosing, for example, I used:

Code:

ubuntu-6.10-alternate-amd64.iso

If you boot up the ubuntu desktop live CD and need to access your raid, then you will need to install mdadm if you want to access any software raid arrays:

Code:

sudo apt-get update
sudo apt-get install mdadm

Don't worry too much about this for now - you will only need this if you ever use the Ubuntu desktop cd to fix your installation, the alternate install cd has the mdadm tools installed already.

5) Finally, lets get on with it!

Boot up the installer
Boot up the alternate install CD and run through the text based installation until you reach the partitioner, and select "Partition Manually".

Create the partitions you need for each raid array
You now need to create the partitions which you will (in the next step) turn into software raid arrays. I recommend using the space at the start, or if your disks are identical, the end of your disks. That way once you've set one disk up, you can just enter exactly the same details for the second disk. The partitioner should be straightforward enough to use - when you create a partition which you intend to use in a raid, you need to change the type to "Linux RAID Autodetect".

How you partition your installation is upto you, however there are a few things to bear in mind:

If (like me) you are going for a performance raid, then you will need to create a separate /boot partition, otherwise grub wont be able to boot - it doesn't have the drivers needed to access raid 0 arrays. It sounds simple, but it took me so long to figure out.

If, on the other hand, you are doing a server installation (for example) using raid 1 / 5 and the goal is reliability, then you probably want the computer to be able to boot up even if one of the disks is down. In this situation you need to do something different with the /boot partition again. I'm not sure how it works myself, as I've never used raid 1, but you can find some more information in the links at the end of this guide. Perhaps I'll have a play around and add this to the guide later on, for completeness sake.

If you are looking for performance, then there isn't a whole load of point creating a raid array for swap space. The kernel can manage multiple swap spaces by itself (we will come onto that later).

Again, if you are looking for reliability however, then you may want to build a raid partition for your swap space, to prevent crashes should one of your drives fail. Again, look for more information in the links at the end.

On my two identical 250 GB drives, I created two 1 GB swap partitions, two +150 GB partitions (to become a raid0 array fro my /home space), and two +40 GB partitions (to become a raid 0 array for my root space), all inside an extended partition at the end of my drives. I then also created a small 500 MB partition on the first drive, which would become my /boot space. I left the rest of the space on my drives for ntfs partitions.

Assemble the partitions as raid devices
Once you've created your partitions, select the "Configure software raid" option. The changes to the partition table will be written to the disk, and you will be allowed to create and delete raid devices - to create a raid device, simply select "create", select the type of raid array you want to create, and select the partitions you want to use. Remember to check which partition numbers you are going to use in which raid arrays - if you forget, hit <Esc> a few times to bring you back to the partition editor screen where you can see whats going on.

Tell the installer how to use the raid devices
Once you are done, hit finish - you will be taken back to the partitioner where you should see some new raid devices listed. Configure these in the same way you would other partitions - set them mounts points, and decide on their filesystem type.

Finish the instalation
Once you are done setting up these raid devices (and swap / boot partitions you decide to keep as non-raid), the installation should run smootly.

6) Configuring Swap Space

I mentioned before that the linux kernel automatically manages multiple swap partitions, meaning you can spread swap partitions across multiple drives for a performance boost without needing to create a raid array. A slight tweak may be needed however; each swap partition has a priority, and if you want the kernel to use both at the same time, you need to set the priority of each swap partition to be the same. First, type

As you can see, the second swap partition isn't being used at the moment, and won't be until the first one is full. I want a performance gain, so I need to fix this by setting the priority of each partition to be the same. Do this in /etc/fstab, by adding pri=1 as an option to each of your swap partitions. My /etc/fstab file now looks like this:

As I mentioned earlier, if you ever boot into your instalation with a live cd, you will need to install mdadm to be able to access your raid devices, so its a good idea to at least roughly know how mdadm works. http://man-wiki.net/index.php/8:mdadm has some detailed information, but the important options are simply:

Code:

-A, --assemble
Assemble a pre-existing array that was previously created with --create.

-C, --create
Create a new array. You only ever need to do this once, if you try to create arrays with partitions that are part of other arrays, mdadm will warn you.

--stop
Stop an assembled array. The array must be unmounted before this will work.

will create a raid0 array /dev/md0 formed from /dev/sda1 and /dev/sdb1, with chunk size 4.

When using --assemble, the usage is simply:

Code:

mdadm --assemble md-devicecomponent-devices

for example

Code:

mdadm --assemble /dev/md0 /dev/sda1 /dev/sdb1

Which will assemble the raid array /dev/md0 from the partitions /dev/sda1 and /dev/sdb1

Alternatively you can use:

Code:

mdadm --assemble --scan

and it will assemble any raid arrays it can detect automatically.

Lastly,

Code:

mdadm --stop /dev/md0

will stop the assembled array md0, so long as its not mounted.

If you wish you can set the partitions up yourself manually using fdisk and mdadm from the command line. Either boot up a desktop live cd and apt-get mdadm as described before, or boot up the alternate installer and hit escape until you see a list of the different stages of instalation - the bottom one should read execute shell - which will drop you at a shell with fdisk, mdadm and mkfs etc... available.

Note that if you ever need to create another raid partition, you create filesystems on them in exactly the same way you would a normal physical partition. For example, to create an ext3 filesystem on /dev/md0 I would use:

And that's pretty much it. As long as you have mdadm available, you can create / assemble raid arrays out of identical partitions. Once you've assembled the array, treat it the same way you would a partition on a physical disk, and you can't really go wrong!

I hope this has helped someone! At the moment I've omitted certain aspects of dealing with raids with redundancy (like raid 1 and raid 5), such a rebuilding failed arrays, simply because I've never done it before. Again, I may have a play around and add some more information later (for completeness), or if anyone else running a raid 1 wants to contribute, it would be most welcome.

Re: HOWTO: Linux Software Raid using mdadm

Excellent information and really useful as I am about to setup a RAID 5 myself and very much needed some pointers. I already have a working install of Feisty so I it looks like I will have to install mdadm and go from there. I'll let you know if it works out

Re: HOWTO: Linux Software Raid using mdadm

My feisty install hangs at the exact same spot when one of the discs of my raid5 array is missing. (which I don't understand, since it should still be able to run on the 2 remaining drives, since that's the whole point of raid5) You might try using the instructions on accessing your array via the desktop live cd, to make sure the array is indeed there and not corrupted somehow. If it's corrupted, with a raid0 array, you'd be missing your whole root fs, and the kernel would have nothing else to load.

Re: HOWTO: Linux Software Raid using mdadm

Thanks for this howto!! This is great. I wish I knew that Feisty alternate-install (x86-64) CD didn't have an installer option for software RAID before I nuked my newly installed LTSP server Oh well. A RAID-1/5 HOWTO would be awesome. Maybe I'll try and write one up, although I'm not very experienced at writing HOWTOs.