Smarter Technology Tragedies, Part Two - Needed: User-Proof IT
Needed: User-Proof IT
Parts One
and Two in this series show two cases of powerful technology gone tragically
wrong: One because of poor execution and usage; the other because of flawed, murky
underlying models and runaway automation.
What are
we, as a society and as technology professionals, supposed to do about all
this? Frankly, I’m not sure.
There’s
the obvious motherhood and apple pie moral: Test your code and systems like the
life and death matters they may be—or be prepared to pay the price.
Beyond
that, a vague feeling of helplessness sets in. After all, technologists, to use
a poor analogy, simply design and make the guns. Others aim and shoot, for good
or evil. What can IT really do?
Both
tragedies have obvious current political implications, which we won’t get into
here. Suffice it to say that regardless of your politics, it’s pretty clear
that both medicine and hidden investing need far more transparency to avoid worse
future sorrows.
The thing
that jumps out most is the need to protect ourselves from ourselves. That is,
thinking about how we can build systems that make human error much harder to
make. And much less serious when it does occur. Cars have speed governors.
Nuclear weapons have elaborate fail-safe systems that in theory minimize the
chance of accidents and a madman/woman starting a worldwide holocaust. Science
fiction robots (the good ones, anyway) are programmed to do humans no harm.
Today we
know better than ever that those same humans, especially in matters of security—
computer and otherwise—are always the weakest link. Perhaps the next generation
of programs and systems should make it a priority to deliver new levels of
personalization and ease of use along with more powerful protections that make
it much harder to do harm—even at our busiest, laziest, greediest, most
distracted moments. Great power requires great caution and protections.
The irony
of a man who earned his living solving computer problems being struck down by
one was not lost on Scott Jerome-Parks, the CIBC computer analyst, according to
The New York Times. Nor should it be
lost on us.
Good point...Posted on: 02-19-10 | By: Joe MaglittaBuilding computer models and programs flawed foundations expands the wisdom of Garge In, Garbage Out - GIGO.
In the case of climate modeling, allegations of scientists massaging/falsifying/surpressing research data further weakens the foundations. (More of that in an upcoming Slaps and Claps. http://www.smartertechnology.com/c/a/Technology-For-Change/Smarter-Technology-Slaps-and-Claps-Part-3/)
Thanks for weighing in Barry.
Now, About those climate modelsPosted on: 02-18-10 | By: Barry WilliamsDoes anyone in the "environmental" movement understand the implications implicit in this story?
The "science" is far from settled. Just like the Quants who could not account for the true cause of their downfall -- CHAOS --, environmental researchers cannot attempt to model huge, complex systems that are grossly and subtly affected by CHAOS and rely upon their models.
