My mind in a cloud of lily-white..

Author

Mark Pustjens pustjens@dds.nl

NOTE For your convenience a downloadable copy is available here

Introduction

This guide describes how I implemented software raid under RedHat 7.2 and 7.3.

This guide assumes the installation of raid1 (mirroring) on an existing RedHat system. For more information about RedHat go to `www.redhat.com'. I also assume GRUB as the bootloader.

The original disk used in this guide is /dev/hda, the new disk to create the mirror on is /dev/hdc.

The following partitions are assumed on the origional disk:

/dev/hda1   /boot
/dev/hda2   /
/dev/hda3   swap
/dev/hda4   /home

For the creating of this document I relied heavilly on the Software Raid Howto. This document is also available for download on my website

Disclaimer

Although RAID seems stable for me, and stable for many other people, it may not work for you. If you lose all your data, your job, get hit by a truck, whatever, it's not my fault, nor the developers'. Be aware, that you use the RAID software and this information at your own risk! There is no guarantee whatsoever, that any of the software, or this information, is in anyway correct nor suited for any use whatsoever. Back up all your data before experimenting with this. Better safe than sorry.

Preperation

To be able to set up mirroring, you'll need an extra harddisk, besides the one with the running RedHat installation. This harddisk needs to be at least equal or larger in storage capacity than the origional harddisk.

For performance and safety reasons it's recommended the harddisks are connected to seperate ide-busses.

  • Two harddisks sharing a single ide-bus will result in less performance.
  • Since it is very possible a crashing harddisk will also take down you're ide-bus, you should put the disks on seperate ide-busses.

Since RedHat comes with a kernel with raid as kernel modules, you need to build a new kernel, with raid in the kernel itself, not as module. If you don't do this, you won't be able to boot from raid.

Also upgrade your GRUB. Use an official GRUB release, nog a RedHat package.

Also veryimportant_ is to makebackups_ of your data, because something might go wrong.

Step 1: Preparing the new harddisk

The first thing we should do is prepare the new, empy harddisk for raid1. This has to be done as root user. First dump the partition information of the origional harddisk to a file:

# sfdisk -d /dev/hda > hda-partition-info.txt

This command will create a new file in the current location, with information about the partitions on the origional disk.

Since we want raid, we need to edit this file a bit. So open the file with your favorite editor (eg: pico or vim). Everywhere is says Id=83 or Id=82, change the number to fd. The Id field in this file defines the partition type. Where 83 means a standard linux partition, fd means a linux raid auto partition. 82 means a swap partition.

When you're done don't forget to save the file and continue.

Now create the partitions on the mirror disk by issuing the following command:

# sfdisk /dev/hdc < hda-partition-info.txt

Now the new partition table will be written to disk, and the kernel will resync the drive.

Step 2: Creating a raidtab

Now the correct partitions are created, we can start writing a raidtab file. The raidtab file has to be saved under /etc. Open up a new empty file with your favorite editor, eg:

# vi /etc/raidtab

The raidtab file is essential for the creation and activation of raid devices during system startup.

This file shoud have an entry of the following form for each partition. I will provide one example. You will have to write the others yourself.

raiddev /dev/md0
raid-level                     1
nr-raid-disks                  2
persistent-superblock          1
chunk-size                     128
device                         /dev/hda1
failed-disk                    0
device                         /dev/hdc1
raid-disk                      1

The raiddev /dev/md0 defines which device file this raid partition will be. With multiple raid partitions, just increase the 0 for each new entry.

The raidlevel entry defines which raid type you want. I want mirroring, so I type 1. Other raid levels are, of course, possible. Read the Software Raid Howto if you want more info. However, I only have 2 disks, so only a few options are available.

The nr-raid-disk tells how many disks there are in the raid device. We have 2 disks, so we put in 2.

The persistant-superblock field should allways be 1. This options makes sure the raid information is stored on the partition itself, not only in file.

The chunk-size field has no infuence on raid1, but should allways be present

The device entries define which partitions on which disks will be part of this raid device. The device entry should allways be followed by a failed-disk or raid-disk entry.

For /dev/hda1 we set failed disk. This marks the drive as bad. It will not be used when creating/activating this raid device. This way, our origional disk will not be overwritten until we mark it as a good disk. Also note the number after the entry. This defines which device entry we are talking about.

Now we have defined one device as bad, we also define one as good. The good disk in my case is the disk i want the mirror on: /dev/hdc1.

After this, you should add entries like the example above for each partition. Also don't forget to save the file when you're done.

Step 3: Creating the raid devices

Now we should create the raid devices for each entry in /etc/raidtab. Do this by issuing the following command for each raid device:

# mkraid /dev/md0

After creation the raid devices will be automaically be activated. You can confirm this by issuing the command:

# cat /proc/mdstat

Step 4: Creating filesystems on the raid devices

On each raid device you'll have to create a filesystem. I want an ext3 filesystem, so i issue the following command for each raid device:

# mkfs.ext3 /dev/md0

See man mke2fs for information about the options this command takes.

This step will take a long time to complete, so get a cup of coffee, or a glass of beer.

Step 5: Copying the origional partitions to raid

Now the raid devices are activated and have file systems on them, we need to copy the data on the origional partitions to the raid devices. During this and the next steps, make sure no data is altered. You can do this by bringing the system down to runlevel 1. Or you could also make sure no user is able to log in, by unplugging the networking cable.

When you're sure of the above, proceed with the following.

First you need to create a mountpoint where we will temporarilly mount the raid devices. I created a mountpoint like this:

# mkdir /mnt/tmp

The actions described below should be performed for each and every partition to be mirrored.

Mount the raid device:

# mount /dev/md0 /mnt/tmp

Change the path to the mountpoint of the origional partition: (In my case: /boot, /, and /home. swap partitions dont need to be copied).

# cd /boot

Copy all data the origional partition to the mounted raid device (make sure you dont make any typing errors):

# find . -xdev | cpio -pm /mnt/tmp

This command finds all the files in the current working directory (.), without descending to other filesystems/partitions.

The previous 3 actions will take a lot of time, depending on how much data is present on the partitions.

Step 6: Modifying /etc/fstab

Now you should open /etc/fstab with your favorite editor (dont forget to backup).

In this file you should change the first field of every row to correspond with it's raid device. So my raidtab would look something like this:

before:

/dev/hda1      /boot        ext3    defaults    1 1
/dev/hda2      /            ext3    defaults    1 2
/dev/hda3      /swap        ext3    defaults    1 2
/dev/hda4      /home        swap    defaults    0 0

after:

/dev/md0       /boot        ext3    defaults    1 1
/dev/md1       /            ext3    defaults    1 2
/dev/md2       /swap        ext3    defaults    1 2
/dev/md3       /home        swap    defaults    0 0

When you're done don't forget to save the file.

Step 7: Reboot

Now it's time to reboot. When rebooting, open the grub command shell. On a standard RedHat installation that means you push the up and down button in the os selection menu. By pressing up/down the timeout will go away. Now select your kernel with raid support. Then press e to edit this entry. Change or add the kernel argument root to point to the raid device. In my case this would be:

raid=/dev/md1

When done press `b' to boot this kernel with these options. If everything boots up correctly, continue with the following step, otherwise make sure you did everything like described.

Step 8: Fdisk the origional disk

Now you have rebooted, you have your root device on a raid partition. You can veryfy this by issuing the mount command. It should list /dev/mdX as root device. You can also type:

# cat /proc/mdstat

To check the status of your raid devices. Right now they should be in degraded mode. This because we marked the origional disk as a failed disk.

Now use fdisk on your origional harddisk:

# fdisk /dev/hda

You must change the partition types of all the partitions you have mirrored to `fd'.

Or you could use the save partition information we used earlier to create partitions on our new disk:

# sfdisk /dev/hdc < hda-partition-info.txt

This should do the job. We are now ready to add the partitions on the origional disk to the raid device.

Step 9: Completing the raid device

The following actions should be performed for each downgraded raid device. You can check the status of the raid devices using:

# cat /proc/mdstat

Add the partitions on the origional disk (/dev/hda) to the raid devices (/dev/mdX):

# raidhotadd /dev/md0 /dev/hda1

Now the disk in the raid device will be synced. That is, the data on the new disk will be mirrored to the origional disk. If you now check the status you will see how for the sync is complete, and the speed.

You could best wait for the current raid device to be synced. After that proceed with the oher devices.

This step will take a long time, depending of how large your partitions are.

Step 10: Update the raidtab

Now all raid devices are no longer in degraded mode, and they are all in sync we can update the file /etc/raidtab. (verify the status of the devices before proceeding.

Open /etc/raidtab with your favorite editor. Look up all the failed-disk lines, and change them into raid-disk lines. That's all! You now have one thing left to do.

Step 11: Installing GRUB

Now a very important step: installing GRUB. This we have to do manually. (if you know an other way, let me know).

While it is true that we can install GRUB by changing grub.conf, grub does not know how to handle with raid devices. It will install GRUB on only one device, which will be a problem when the first disk failes and you have to boot from the second disk.

To install GRUB, start the GRUB commandline shell. You will get a prompt. Enter the following commands (after tailoring to your situation).

> device (hd0) /dev/hda
> root (hd0,0)
> kernel /vmlinuz-2.4.19-athlon-smp ro root=/dev/md1 vga=791
> setup

Now grub is installed on the first harddisk (/dev/hda). To install on the second harddisk (/dev/hdc), repeat these commands, but change device (hd0) /dev/hda to device (hd0) /dev/hdc.

Now you are all done. Perhaps you could reboot once more to see if it will all work like planned. Instructions for testing the redundancy are provided in the Software Raid Howto, a link to this document is provided at the top of this document.

Special Thanks

I would like to thank Linus Torvalds, for creating the basis of such a fine operation system.

Also would i like to thank Richard Stallman for founding the GNU Foundation. Without him, GNU/Linux as we know it would probably not exist.

Thanks to Jakob Ƙstergaard, for writing the Software Raid HOW-TO. (from which i borrowed the disclaimer).