[Tfug] HDD size and RAID queries

[Yan]

It's a bit outdated, but Google actually published a paper about hard drive
failures at Usenix a couple years back:

http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/disk_failures.pdf

IIRC, they reasoned about the size as well as a few other factors. Might be
useful for your hard drive failure question.

- Yan


On Wed, May 8, 2013 at 11:14 AM, Harry McGregor <micros at osef.org> wrote:

> Hi,
>
> Most of this depends on your threshold for data loss and backup plans...
>
>
> On 5/8/13 4:49 AM, John Gruenenfelder wrote:
>
>> Greetings all,
>>
>> It is finally time that I upgrade my file server/Mythbox machine.  It
>> does a
>> whole host of other tasks, but those are the primary ones.  Currently, I
>> have
>> four 500 GB SATA (3 Gb/s variety) installed, three of which are Samsung
>> HD502IJ drives and the fourth of which is a WDC WD5000AAKS-7.
>>
>> Each drive contains two partitions, one small (~200 MB) and the other
>> containing the rest of the space.  The four small partitions are part of a
>> RAID-1 array and represent the /boot area of the system.  The other four
>> are
>> part of a RAID-5 array.  That block device is then passed through LUKS and
>> then *that* block device goes to the LVM2 layer which finally splits it
>> into
>> the various volumes.  All volumes are using XFS.
>> So far, this setup has worked marvelously.  Originally, I had to hack the
>> initrd scripts together to make it work, but now the standard Debian
>> initrd+kernel are capable of handling this in an entirely automated
>> manner.
>>
> Yes, Debian has all of this built into the installer now, and the default
> scripts.
>
>  But, the time has come to upgrade.  Mostly because of MythTV, I find I am
>> constantly on the verge of running out of space.  I've already had to
>> resort
>> to some symlink juggling to put things in different places where more
>> space is
>> fre.
>>
>> I have two questions that are related.  I've read a number of articles
>> which
>> have presented well reasoned arguments for why RAID-5 should be abandoned
>> in
>> favor of RAID-6.  I understand these arguments, yet I'm not sure if *my*
>> big
>> is the same as *their* big.  That is, with my RAID-5 setup the space
>> efficiency is (1 - 1/n) and with four drives that works out to 2 TB * 3/4
>> =
>> 1.5 TB usable space.  Clearly better than RAID-1 mirroring and I can
>> honestly
>> say that, even with some of the heavy I/O that vieo streaming can do, I've
>> never had any issues with the RAID-5 speed (parity calculations, extra
>> writes,
>> etc.)
>>
> I tend to have both a size and a # of drive threshold for Raid-5.
>
> Raid-5 (and six) put a lot of stress on the "good" drives during rebuild,
> this is why double drive failure on a raid5 is far more common then the
> statistics would suggest it should be.
>
> My threshold tends to be 4 drives or a total usable capacity of about 6TB
> before I consider Raid-5 as unstable.   A 4 drive raid 5 with 2TB drives
> could take >10 hours to rebuild.
>
> In the enterprise market, most raid5 storage arrays use ranks of 8 drives,
> or 7 drives with a hot spare in raid 5.
>
> The storage array I work with uses a distributed mirroring, which greatly
> reduces the stress during rebuild (many to many copy operation, etc), and
> shortens rebuild time.
>
> My personal Raid6 at home with 14 1TB drives takes about 10 hours to
> rebuild from a single drive failure.
>
>  The space efficiency for RAID-6, however, is (1 - 2/n) and with four
>> drives
>> that works out to TotalSpace * 1/2 which is the same as RAID-1 mirroring.
>> This is the same as RAID-1 mirroring.  RAID-6 is still the better choice
>> because it can handle the failure of *any* two drives whereas if I were
>> to use
>> RAID-1 I would have to create two mirrors of two drives each and such a
>> setup
>> could only handle two drive failures in particular circumstances.
>>
>
> I tend to like Raid-6 for >4 drives.  Are you sure you don't want to use
> more drives?
>
>  I'm looking at buying four new drives, either 2 TB or 3 TB in size.  Part
>> of
>> that decision will depend on what type of RAID I use.  RAID-5 with four 2
>> TB
>> drives gives me 6 TB usable space, but only 4 TB if I choose RAID-6.
>>  Using 3
>> TB drives, the numbers are 9 TB and 6 TB for RAID-5 and RAID-6,
>> respectively.
>>
> Personally with these choices, I would go with 6x 2TB drives in Raid-6
> over 4x 3TB drives in Raid-5 for example
>
>  The argument to ditch RAID-5 in favor of RAID-6 is entirely based on
>> probabilities and the likelyhood of encountering an unrecoverable error
>> while
>> rebuilding an array.  Am I actually in this realm with these drive sizes?
>>  Is
>> my Big Array even remotely close to the Big Array these authors are
>> concerned
>> with?  If it's not, then I can stay with RAID-5 and gain an extra 25%
>> usable
>> disk space.  Confusing...
>>
> I have seen non recoverable read errors far too often on hard drives.
>  Remember the % lost is directly related to the number of drives.
>
>
>> The other related question deals with using these new giant drives to
>> begin
>> with.  I came across a few vague sentences that *seemed* to indicate that
>> the
>> current BIOS methods for accessing large drives (LBA) won't do the trick
>> for
>> these 1 TB+ drives and that UEFI is required.  Is this really true?  The
>> motherboard they will be used with is not new, nor is it very old either.
>> I've used 1 TB+ drives on a much older machine at work, but they were
>> accessed
>> via USB2 and FireWire and not via a SATA or eSATA port.  Since I saw this
>> mentioned in only one place I tend to think that this isn't true, but I
>> thought I should check first.
>>
> Many older non-UEFI bios's don't support booting from a GPT partition
> table.  MBR partition tables top out at the 2TB mark.
>
> The simplest is to get a pair of 30-60GB SSDs, and RAID-1 the two SSDs for
> your / and /usr and /var etc  and then use your build LVM on the large
> drives for all of your data mount points.
>
> -Harry
>
>
>
> ______________________________**_________________
> Tucson Free Unix Group - tfug at tfug.org
> Subscription Options:
> http://www.tfug.org/mailman/**listinfo/tfug_tfug.org<http://www.tfug.org/mailman/listinfo/tfug_tfug.org>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tfug.org/pipermail/tfug_tfug.org/attachments/20130508/c90c4802/attachment-0002.html>