TODO list for MD/RAID - Software RAID for Linux

Patches I have seen on linux-raid

• fixes for superblock checking on Sparc64
  DONE
• raid1 read balancing In linux-2.4.0test1-ac16 via mingo

Patches I would like in the 2.4 time scale

• Apparently the loopback device Oops if the buffer isn't locked, and raid[15] doesn't always lock buffers before passing them down.
  NOT TRUE ANY MORE
• md should range check the address of IO requests before passing them down, otherwise the inevitable error will make md think the device has died. This is particularly imporant for raid 1/5
  TEST DONE IN GENERIC_MAKE_REQUEST.
• /proc/md directory structure with one directory for each active device, each containing one directory for each underlying device with useful information in appropriate files.
• Assume a basic block size of 512 (like everyone else) instead of 1024. Anton Altaparmakov claims this is needed for NTFS.
  ONLY LINEAR/RAID0 HAD A PROBLEM - BOTH FIXED
• Thomas Waldmann reports that "when rebooting/halting, did you notice that the message telling the superblock updates doesn't appear everytime ? " and wonders if this is a problem.
• Make the default blocksize (md_blocksizes) be the max of the blocksizes for the underlying devices, rather than "1024" for Holger.Smolinski@de.ibm.com who has underlying devices with block sizes > 1024 on S/390.
  DONE
• link together raid5 stripes that are waiting for handling by raid5d so that it doesn't have to search the whole cache. This would make it scale to a larger cache better.
  DONE IN EXPERIMENTAL PATCH
• link together raid5 stripes which are available for re-use into an LRU list. This would obviate the need for calling strink_stripe_cache( , max/8) every so often - we could just take the oldest stripe or sleep on a waitqueue.
  DONE IN EXPERIMENTAL PATCH
• Halve the number of buffers that are allocated to raid5 stripes. Currently two buffers per disc are allocated, one for "bh_old" and one for "bh_copy". Most of the time only one of these is in use. The only times when both bh_old and bh_copy need to be valid at the same time are
  1. when computing parity with READ_MODIFY_WRITE
  2. checking parity in handle_stripe_sync
  Case 1 can be handled by
  1. computing the parity of all bh_old to be over-written.
  2. copy bh_new to the buffers
  3. xoring in the parity of the new content of the buffer.
  Case 2 can be handled by computing parity of the whole stripe into the parity buffer - thereby destroying it - and checking if that block is now all zero. Destroying the parity block is of no cost because if it was wrong, we didn't want it, and if it was right, we won't need it as we have all the real data.
  This would make the raid5 stripe cache much more memory efficient.
  DONE IN EXPERIMENTAL PATCH
• Make the size of the raid5 stripe cache configurable via sysctl (or atleast /proc)
• Change the spare-disc handling so that personalities don't "know" so much about spare discs. The personailty module does not need to "know" about spares much at all. If it "finds" a disc that is not MD_DISK_FAULTY and is not MD_DISK_SYNC, then it should be write-only and a recover operation should be started. There is no really need to have it in a "separate" part of the disk array. Doing this would be the first step to allowing more flexability in handling spare discs. If I have two RAID arrays on a system with comparable sized drives, I would lile to be able to only have one spare - at least as big as the largest other drive - and then whenever any drive in either array fails, the spare gets automatically built into that array. I think that it would also make the codea bit tider, but that isn't a very significant facet.
• Change the recovery process so that it happens in a per-array process instead of in mdrecoveryd Currently recovery is done by the central mdrecoveryd and resync is done by per-array raidXsync processes. This feels a bit inconsistant and means that only one array can be recovered at a time. I would prever that the central thread (mdrecoveryd) requested the raid personality to start a recovery process, and the personality could do any initialisation required and then fork a thread which primarily calls md_do_sync. This would leave the main thread free to monitor other arrays.
• Allow an md device to be partitioned in the same way that normal drives are partitioned. This would allow me to have mirrorred boot discs with separate /, swap and /usr partitions, and not have to have several separate raid arrays set up - just one would do. I haven't thought through how best to do this so that current functionality is not disturbed, but I would really like it, and if you don't see a problem I will explore further.
• Allow md.o and xor.o to be configured as loadable modules.- DONE
• Allow the RAID superblock to live elsewhere on disc. md can be told where through ioctl, or when it looks for a superblock, it looks at current place, then at 0k 4K .. 32K

  msdos parition tables seem to leave the first track blank, so this would allow us to mirror an in-use disc without having to move data at all. This would mean that the raid device would have an un-writable block where the superblock was stored, but that should not be a problem.

• tidy up drive accounting
  DONE
• Make sure drive is failed if superblock write fails.

Patches in the 2.5 time scale

• Free up the locking on the raid5 stripes. Currently, only one request can be active per stripe, as raid5_make_request call locks the stripe, and it doesn't get unlocked until the request completes. For read requests, there is never any real need to lock the stripe, though it may make it a bit easier of the target disc is failed. For write there is some need for locking against other writes happening while waiting for lower level writes to complete, but new writes can fairly safely be attached to the stripe. Infact, read requests against blocks on working drives should be able to complete without any interction with raid5d.
  DONE IN EXPERIMENTAL PATCH
• change the raid5 stripe cache so that every stripe is 1 page wide. (This is quite a big change). This would mean that we would effectively have a matrix of sectors in each stripe, "raid_disks" wide and "pagesize/sectorsize" high, and we would need to track the state of each sectors (or 1K block). This would add a fair bit of complexity but it would mean that:
  1. we wouldn't need to worry about changing the buffer_size of the cache, and I think that it is very difficult to get the really right.
  2. we could support filesystems that access the device with varying size blocks. While this isn't actually possible at the moment, I think that it is now an artificial restriction and there are certainly some filesystem implementors (sgi/XFS for example) that would like the extra flexability.
  3. There would be more oportunity for collessing large sequential writes (and reads) because there is less fragmentation. I have definately found that 4K accesses are much faster than 1K access, both for filesystem traffic and for rebuilding.
  I haven't actually tried coding this yet, but I am certainly willing to give it a try if you don't see any major problems.