[Tfug] Small-ish (capacity + size) disk alternatives

[Bexley Hall]

Hi Tyler,

On 2/1/2013 1:51 AM, Tyler Kilian wrote:
> There is no IT installation that will be 100% unattended.  If your
> requirement be that it be based on PC architecture and use PC components,
> then expect PC problems.

I have no "requirement" that it be based on PC components.
Unfortunately, those are the only sorts of things *you* will be
able to purchase and "assemble"!

> There are tons of general automation systems that are firmware based

You've not done your homework.  Or, are still failing to understand
the scope of this project.  See below for some examples of these
"automation systems".

> and if I were a company looking to build a home controller, I would
> leverage something along those lines or spec out the system design
> from scratch.

If you were that company, you wouldn't approach a controller as
ambitious as this!  Its too pervasive.  Too many different types
of I/O's involved.  The sort of effort that would require lots
of "tinkerers" each willing to invest a little of their time
to interface to *their* particular devices (water softener, weather
station, etc.), share those results with others who happen to have
those same devices while SIMILARLY BENEFITING from the efforts
of others to support some of the other devices that they have.

[Gee, sure sounds like the approach taken for open source projects!]

I.e., I have no desire to write drivers and design hardware interfaces
for every model of weather station, garage door opener, PZT camera
base, etc. on the market!  I'll design for what *I* have and let
others peer over my shoulder to see *how* I approached the task in
the hope that they can "tinker" with my *existing* design instead of
having to start from scratch.

[Over the years, I have noticed that very few folks can sit with
a blank piece of paper and "create" -- most need a framework that
someone else has already implemented... that they can "modify"]

> Either way, the instant I want to store video locally or do anything

The drive in question is not used to store video (or audio).
What can you store on a tiny 30G drive -- one/two movies, a
few albums and the rest of the software?  I store multimedia on
external USB drives (currently.  I will migrate this to iSCSI
in the future) using "conventional" filesystems.

Users are already familiar with external USB drives.  And, as
long as I avoid proprietary filesystem formats, they can
"maintain" that media inventory using COTS tools -- a Mac,
a PC, etc. (so I don't have to create tools to do this work)

If such a drive *fails*, the user can go to Costco and purchase
a replacement.  If they lose their media inventory, <shrug>
Next time, buy *two* drives and make a copy (using your PC)
onto the backup drive.

[Why not have a "copy media" utility built into the system?
Because then you'll go to Costco with a defective drive and
try to explain to them how your Frajistat9000 gave you an
"error 27:  replace drive" message.  And they will look at
you with a blank stare:  "Is that a Mac or a PC?"]

> requiring gigs of data locally, I move from the firmware systems to
> requiring hard disks. Disks mean disk failures. That's just how it is.

You seem to misunderstand "firmware".  It has nothing to do with *how*
the "software" is stored (e.g., ROM/FLASH vs. magnetic media).  It
is simply "persistent storage for program/data that controls a device".
I've designed systems where the firmware resided in masked ROM XIP,
FLASH loaded into RAM at IPL, *core* memory, *floppy* disk (you can
smuggle a floppy through customs but a new set of ROMs will often
raise eyebrows and "require" a bribe -- "Gee, those look like they
might be IMPORTANT to you..."), etc.  Even one system that loaded
its firmware over an EIA232 serial link!

A hard disk is just another form of "persistent memory".  In my
case, one in which I can exploit its surplus capacity to use as
virtual memory (considerably more difficult to do this if *ROM*
based firmware) instead of having to add *physical* memory!

[This is called "adding value"]

> If I were a company wanting to make it user friendly, I would design
> an enclosure where the disks are easily pulled (like an Xbox) and
> mirror them, building in the logic to do this all automatically with
> simple notifications.  I would make it as small as possible, but
> ultimately, a system this complex would require a commitment of the
> user of supplying sufficient space.

... and power, and ventilation/cooling, etc.  Easily accessible
(since you are planning on failures), regularly "cleaned" (dust
bunnies, insect infestations, etc.).  All issues that almost
guarantee it will be ages before it is affordable and commonplace
and not just an elitist toy!

["Rich folk" have had such systems for ages!  But, they called
them "household staff/servants"!  Notice how well *that* idea
has caught on... after GENERATIONS!! :-/ ]

> It wouldn't be a cheap solution
> even if it were amazing.  It would take several generations to refine.
> All of this, properly done to meet your goals would be doable, but it
> would be custom.
>
> Instead, your approach is start with a Micro PC and grouse that hard
> drives aren't reliable enough.  Champagne wishes on the proverbial soda
> pop budget.

Gee, one message ago, *you* were stating:
	"I really think a solution exists.  The worry, to me,
         seems unfounded.  If it were me, I'd invest in a good
         quality drive."
Has your opinion changed *that* quickly?

Note that *my* approach was to start with a fully custom solution
built from the ground up -- so I can control form factor, power
consumption, cost, performance, reliability, etc.  I.e., the same way
I would design a medical instrument, gasoline pump or slot machine.

But, that excludes others ("tinkerers") from being able to replicate
the design and makes it considerably more expensive for them to
*purchase* the necessary "custom" hardware (by a factor of 3 or 4).

Should I similarly exploit other "professional" design methodologies
that make *my* job easier (proprietary IDEs, VHDL tools, PCB layout
tools, etc.) and expect those same "tinkerers" to make the sorts of
investments that I've (already) made?  Everyone has a reflow oven
and SMT rework tools at their disposal, right?  Reballing arrays
is now taught in grade school, if I'm not mistaken (?)

I opted to steer my design along a path that could be *emulated*
with COTS hardware.  Granted, those "tinkerers" would pay higher
power bills and have to find larger spaces in which to stash their
implementations (tower cases, etc.).  But, they'd be able to do
so.  I wouldn't have locked them out by making choices that had
no obvious COTS parallels that they could take advantage of.

> If you really wish to build something like this, you are going to have
> to accept some limitations and/or be prepared for custom engineering.

So, you've confirmed my earlier suspicions:  custom engineer the
system for myself (this is what I do professionally) and force
any others who might have wanted to benefit from my efforts to
go to a retail outlet to *purchase* something equivalent -- whenever
mainstream vendors advance to this level of sophistication.

[I'm looking to *retire*, not manufacture toys!]

> Your desires are simply beyond the scope of homebrew tinkering.

I.e., don't bother making it an FOSS offering -- it's *beyond* the
capabilities of "tinkerers" to replicate, enhance, etc.  That
definitely makes it a lot easier!  I can just steal the memory
subsystem from another (similar) project and build on that.

Others with similar interests need not fret!  There are COTS
solutions available for them!  (almost)

Heck, for a couple of thousand dollars, you can get the whole house
"music distribution" capabilities I've targeted:
     <http://www.sonos.com/shop/>

Maybe someone will design a similar whole home *video* solution
as well (of course, that will be a separate system with much of
the same costs replicated as the above).  Of course, you
probably won't be able to just plug a "spare" widescreen LCD
monitor into that box... they'll undoubtedly want to sell
you that, as well!

I suppose you could purchase a COTS VoIP system with a software
PBX so you can have "smart" voice comms within the home (I'm
sure the sonos system can't be easily adapted to that purpose.
Oh, well... all those existing "audio paths" but only available
for playing music!)

We already know you can purchase a surveillance system for a few
hundred dollars.  With an appropriate mixer or PiP TV, you can
probably arrange to view that surveillance video *live* while
enjoying a movie, etc.

And, for $300, Nest makes a sexxy thermostat!
    <http://www.nest.com/living-with-nest/?ref=gmc>

There are "smart" irrigation controllers that will interface
with environmental monitoring systems (temperature, rainfall,
dew point).  Some you can even talk to remotely:
    <http://www.weathermatic.com/content/smartlink-network-contractors>
Maybe someone will devise a web link that allows them to
query local forecasts to *anticipate* tomorrow's weather
conditions as well (sure, you don't want to be watering the
yard WHILE it's raining.  Or, soon *after* it rained.  But,
you also wouldn't want to water *today* if it is going to rain
tomorrow!)

SmartHome will let you piece together various assorted bits of
automation to tackle your laundry, appliance, garage door, etc.
needs (well, sort of...):
    <http://www.smarthome.com/_/index.aspx>

There's even a commercial service that will let you control
bits of your home remotely via your iPhone (for $2/day).

All you'll (tinkerer) have to do is figure out how to get all
these things "integrated" so they can talk to each other (the sum
is greater than the parts).  *Then*, figure out how to give them
"smarts" so they can anticipate your needs instead of just sitting
dumbly awaiting your "commands".

Should be a nice little "weekend project", eh?  :>

Please spend some time *researching* the scope of such a project
before *casually* tossing out "If I were doing this..." advice.
Everything is ALWAYS easy and obvious -- when "you" aren't tasked
with making it happen!

People come up with "solutions" to all the world's problems every day!
Yet, somehow, those problems persist...

    "Genius is one percent inspiration,
    ninety-nine percent perspiration."
       -- T A Edison

    "God in the detail"
       -- anon