[Tfug] HDD size and RAID queries

[Harry McGregor]

Hi,

Most of this depends on your threshold for data loss and backup plans...

On 5/8/13 4:49 AM, John Gruenenfelder wrote:
> Greetings all,
>
> It is finally time that I upgrade my file server/Mythbox machine.  It does a
> whole host of other tasks, but those are the primary ones.  Currently, I have
> four 500 GB SATA (3 Gb/s variety) installed, three of which are Samsung
> HD502IJ drives and the fourth of which is a WDC WD5000AAKS-7.
>
> Each drive contains two partitions, one small (~200 MB) and the other
> containing the rest of the space.  The four small partitions are part of a
> RAID-1 array and represent the /boot area of the system.  The other four are
> part of a RAID-5 array.  That block device is then passed through LUKS and
> then *that* block device goes to the LVM2 layer which finally splits it into
> the various volumes.  All volumes are using XFS.
> So far, this setup has worked marvelously.  Originally, I had to hack the
> initrd scripts together to make it work, but now the standard Debian
> initrd+kernel are capable of handling this in an entirely automated manner.
Yes, Debian has all of this built into the installer now, and the 
default scripts.
> But, the time has come to upgrade.  Mostly because of MythTV, I find I am
> constantly on the verge of running out of space.  I've already had to resort
> to some symlink juggling to put things in different places where more space is
> fre.
>
> I have two questions that are related.  I've read a number of articles which
> have presented well reasoned arguments for why RAID-5 should be abandoned in
> favor of RAID-6.  I understand these arguments, yet I'm not sure if *my* big
> is the same as *their* big.  That is, with my RAID-5 setup the space
> efficiency is (1 - 1/n) and with four drives that works out to 2 TB * 3/4 =
> 1.5 TB usable space.  Clearly better than RAID-1 mirroring and I can honestly
> say that, even with some of the heavy I/O that vieo streaming can do, I've
> never had any issues with the RAID-5 speed (parity calculations, extra writes,
> etc.)
I tend to have both a size and a # of drive threshold for Raid-5.

Raid-5 (and six) put a lot of stress on the "good" drives during 
rebuild, this is why double drive failure on a raid5 is far more common 
then the statistics would suggest it should be.

My threshold tends to be 4 drives or a total usable capacity of about 
6TB before I consider Raid-5 as unstable.   A 4 drive raid 5 with 2TB 
drives could take >10 hours to rebuild.

In the enterprise market, most raid5 storage arrays use ranks of 8 
drives, or 7 drives with a hot spare in raid 5.

The storage array I work with uses a distributed mirroring, which 
greatly reduces the stress during rebuild (many to many copy operation, 
etc), and shortens rebuild time.

My personal Raid6 at home with 14 1TB drives takes about 10 hours to 
rebuild from a single drive failure.
> The space efficiency for RAID-6, however, is (1 - 2/n) and with four drives
> that works out to TotalSpace * 1/2 which is the same as RAID-1 mirroring.
> This is the same as RAID-1 mirroring.  RAID-6 is still the better choice
> because it can handle the failure of *any* two drives whereas if I were to use
> RAID-1 I would have to create two mirrors of two drives each and such a setup
> could only handle two drive failures in particular circumstances.

I tend to like Raid-6 for >4 drives.  Are you sure you don't want to use 
more drives?
> I'm looking at buying four new drives, either 2 TB or 3 TB in size.  Part of
> that decision will depend on what type of RAID I use.  RAID-5 with four 2 TB
> drives gives me 6 TB usable space, but only 4 TB if I choose RAID-6.  Using 3
> TB drives, the numbers are 9 TB and 6 TB for RAID-5 and RAID-6, respectively.
Personally with these choices, I would go with 6x 2TB drives in Raid-6 
over 4x 3TB drives in Raid-5 for example
> The argument to ditch RAID-5 in favor of RAID-6 is entirely based on
> probabilities and the likelyhood of encountering an unrecoverable error while
> rebuilding an array.  Am I actually in this realm with these drive sizes?  Is
> my Big Array even remotely close to the Big Array these authors are concerned
> with?  If it's not, then I can stay with RAID-5 and gain an extra 25% usable
> disk space.  Confusing...
I have seen non recoverable read errors far too often on hard drives.  
Remember the % lost is directly related to the number of drives.
>
> The other related question deals with using these new giant drives to begin
> with.  I came across a few vague sentences that *seemed* to indicate that the
> current BIOS methods for accessing large drives (LBA) won't do the trick for
> these 1 TB+ drives and that UEFI is required.  Is this really true?  The
> motherboard they will be used with is not new, nor is it very old either.
> I've used 1 TB+ drives on a much older machine at work, but they were accessed
> via USB2 and FireWire and not via a SATA or eSATA port.  Since I saw this
> mentioned in only one place I tend to think that this isn't true, but I
> thought I should check first.
Many older non-UEFI bios's don't support booting from a GPT partition 
table.  MBR partition tables top out at the 2TB mark.

The simplest is to get a pair of 30-60GB SSDs, and RAID-1 the two SSDs 
for your / and /usr and /var etc  and then use your build LVM on the 
large drives for all of your data mount points.

-Harry