While we are learning about RAID devices, Generally we used smaller disks to build a RAID array. In that time when any raid disk failed and we replaced it with a new disk. Then raid rebuilding takes a few minutes or seconds only. But think about if you are running a high storage capacity server or any backup drives which are having raid configured in size of multiple terabytes and you need to rebuild RAID or rebuilding started due to some issues. It takes a lot of time and we run keep running `cat /proc/mdstat` time to time for checking the status.
Now what to do, How can we speed up raid reconstruction speed. This can be done with various ways. But I generally used below method by changes kernel parameter. Follow the below steps
Check Current Recovery Speed and Status
One of servers managed by me raid disk failed, So we replaced it with another disk and configured. RAID rebuilding speed was very slow As you can view below output.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdb2[0] sda2[1]
4193216 blocks [2/2] [UU]
md2 : active raid1 sdb3[0] sda3[1]
968372864 blocks [2/2] [UU]
[=====>......] check = 48.5% (470179584/968372864) finish=7698.8min speed=1078K/sec
md0 : active raid1 sdb1[0] sda1[1]
4193216 blocks [2/2] [UU]
resync=DELAYED
unused devices: <none>
Now I checked the kernel parameter configured for minimum and maximum raid rebuild speed using below command.
# cat /proc/sys/dev/raid/speed_limit_max 200000 # cat /proc/sys/dev/raid/speed_limit_min 1000
As per above output i found that raid reconstruction speed is set minimum to 1000 KB/Sec/Disc and Maximum to 200000 KB/Sec/Disc.
Increase Minimum RAID Recovery Speed
As we can see that Maximum speed looks fine, but minimum speed is low, Which we need to increase. Before setting any speed make sure that you are enough CPU and RAM resources. Changes may also affect your other application’s speed running on same server.
To set minimum speed to approx 20MB/sec use following command.
# echo "20000" /proc/sys/dev/raid/speed_limit_min
Check Again RAID Recover Speed and Status
After making above changes, let’s check raid reconstruction speed again using same command. Now as per below command output speed is better 3426K/sec then earlier.
# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sdb2[0] sda2[1]
4193216 blocks [2/2] [UU]
md2 : active raid1 sdb3[0] sda3[1]
968372864 blocks [2/2] [UU]
[=====>......] check = 48.6% (470793472/968372864) finish=2419.7min speed=3426K/sec
md0 : active raid1 sdb1[0] sda1[1]
4193216 blocks [2/2] [UU]
resync=DELAYED
unused devices: <none>
Read more about raid devices http://en.wikipedia.org/wiki/RAID
Leave a Reply