How to Increase Speed of Software RAID Rebuilding in Linux

While we are learning about RAID devices, Generally we used smaller disks to build a RAID array. In that time when any raid disk failed and we replaced it with a new disk. Then raid rebuilding takes a few minutes or seconds only. But think about if you are running a high storage capacity server or any backup drives which are having raid configured in size of multiple terabytes and you need to rebuild RAID or rebuilding started due to some issues. It takes a lot of time and we run keep running `cat /proc/mdstat` time to time for checking the status.

Now what to do, How can we speed up raid reconstruction speed. This can be done with various ways. But I generally used below method by changes kernel parameter. Follow the below steps

Check Current Recovery Speed and Status

One of servers managed by me raid disk failed, So we replaced it with another disk and configured. RAID rebuilding speed was very slow As you can view below output.

# cat /proc/mdstat

Personalities : [raid1]
md1 : active raid1 sdb2[0] sda2[1]
      4193216 blocks [2/2] [UU]

md2 : active raid1 sdb3[0] sda3[1]
      968372864 blocks [2/2] [UU]
      [=====>......] check = 48.5% (470179584/968372864) finish=7698.8min speed=1078K/sec

md0 : active raid1 sdb1[0] sda1[1]
      4193216 blocks [2/2] [UU]
      resync=DELAYED

unused devices: <none>

Now I checked the kernel parameter configured for minimum and maximum raid rebuild speed using below command.

# cat /proc/sys/dev/raid/speed_limit_max
200000

# cat /proc/sys/dev/raid/speed_limit_min
1000

As per above output i found that raid reconstruction speed is set minimum to 1000 KB/Sec/Disc and Maximum to 200000 KB/Sec/Disc.

Increase Minimum RAID Recovery Speed

As we can see that Maximum speed looks fine, but minimum speed is low, Which we need to increase. Before setting any speed make sure that you are enough CPU and RAM resources. Changes may also affect your other application’s speed running on same server.

To set minimum speed to approx 20MB/sec use following command.

# echo "20000" > /proc/sys/dev/raid/speed_limit_min

Check Again RAID Recover Speed and Status

After making above changes, let’s check raid reconstruction speed again using same command. Now as per below command output speed is better 3426K/sec then earlier.

# cat /proc/mdstat

Personalities : [raid1]
md1 : active raid1 sdb2[0] sda2[1]
      4193216 blocks [2/2] [UU]

md2 : active raid1 sdb3[0] sda3[1]
      968372864 blocks [2/2] [UU]
      [=====>......] check = 48.6% (470793472/968372864) finish=2419.7min speed=3426K/sec

md0 : active raid1 sdb1[0] sda1[1]
      4193216 blocks [2/2] [UU]
      resync=DELAYED

unused devices: <none>

Read more about raid devices http://en.wikipedia.org/wiki/RAID