[Tfug] Dispersed Storage / Distributed backups

[Bowie J. Poag]

Two things come to mind here.

If they're using something like distributed parity to fill in the gaps 
for when one or more of the nodes goes offline, heh, good luck with that 
one, chief. Unless the client comes bundled with a magical psychic 
unicorn that can come up with the missing data out of it's butt, you're 
going to have one hell of a time rebuilding a stripe because the 
neighboring parity data you need to rebuild with may or may not be 
available at any given moment. XOR is XOR is XOR. If you have a stripe 
with parity spread across 11 nodes, and 1 node dies, every last one of 
those 10 remaining nodes needs to be available at some point to rebuild 
the contents of the stripe on the missing node.

Any write operation that you would commit to a stripe 11 nodes wide 
would require that every node be online to participate in parity 
recalculation, period. With nodes constantly coming on and going 
offline, the overhead for writes would be enormous. The volume would 
have to be locked until all 11 nodes were sync'ed with correct parity 
data. With nodes constantly coming in and out of existence, this would 
be an absolute nightmare. Even if you only needed 6, it would still be a 
nightmare, just less horrific.

It's not clear from the description, but i'm assuming this is just 
geographically distributed RAID 0 with quorum enabled. In which case, it 
would probably be so slow as to be impractical, i'd imagine. Like 
mounting a bittorrent image. Eugh.


Cheers,
Bowie



Angus Scott-Fleming wrote:

>'Twould be interesting to set one of these up here in Tucson for mutual backup.  
>I have an unused IP address for setting up a server ...
>
>------- Included Stuff Follows -------
>Cleversafe Dispersed Storage - CleversafeWiki
>
>    The Dispersed Storage Project is the central point of development and idea 
>    exchange for developers around the world to contribute to innovative 
>    storage solutions leveraging dispersed storage methodology.
>
>
>    The Project uses information dispersal algorithms (IDAsTM) to separate data 
>    into 11 unrecognizable DataSlicesTM and distribute them, via secure 
>    Internet connections, to 11 storage locations throughout the world, 
>    creating a storage grid. With dispersed storage, transmission and storage 
>    of data is inherently private and secure. No single entire copy of the 
>    data is in one location, and only 6 out of the 11 nodes need to be 
>    available in order to perfectly retrieve the data.
>
>    Data on the grid remains private and secure in the face of natural 
>    catastrophes, or failures of hardware, connection, facility, or IT 
>    management. Moreover, the individual data slices do not carry enough 
>    information for an unauthorized viewer to determine the original content.
>
>    The Cleversafe Dispersed Storage software includes client software with 
>    both a comprehensive command line interface (CLI) as well as a complete 
>    programming interface (DSAPI) to support any type of storage application. 
>    This software further includes grid server software for creating a 
>    dispersed storage grid. The Cleversafe software also manages metadata for 
>    file systems stored on a Dispersed Storage grid. In addition, this project 
>    includes the multi-terabyte Cleversafe Research Storage Grid at eleven 
>    separate hosting facilities.
>--------- Included Stuff Ends ---------
>http://www.cleversafe.org/wiki/Cleversafe_Dispersed_Storage
>
>Angus
>
>
>_______________________________________________
>Tucson Free Unix Group - tfug at tfug.org
>Subscription Options:
>http://www.tfug.org/mailman/listinfo/tfug_tfug.org
>
>  
>