TECHNOLOGY FOR CHANGE

Smarter Technology Tragedies, Part Two

Joe Maglitta | Date: 01-29-10 | Comments

System failures in advanced radiation therapy machines and secretive trading programs highlight technology�s -- and users� -- destructive potential.

Table of Contents:

Smarter Technology Tragedies, Part Two - Needed: User-Proof IT

Needed: User-Proof IT

Parts One and Two in this series show two cases of powerful technology gone tragically wrong: One because of poor execution and usage; the other because of flawed, murky underlying models and runaway automation.

What are we, as a society and as technology professionals, supposed to do about all this? Frankly, I�m not sure.

There�s the obvious motherhood and apple pie moral: Test your code and systems like the life and death matters they may be�or be prepared to pay the price.

Beyond that, a vague feeling of helplessness sets in. After all, technologists, to use a poor analogy, simply design and make the guns. Others aim and shoot, for good or evil. What can IT really do?

Both tragedies have obvious current political implications, which we won�t get into here. Suffice it to say that regardless of your politics, it�s pretty clear that both medicine and hidden investing need far more transparency to avoid worse future sorrows.

The thing that jumps out most is the need to protect ourselves from ourselves. That is, thinking about how we can build systems that make human error much harder to make. And much less serious when it does occur. Cars have speed governors. Nuclear weapons have elaborate fail-safe systems that in theory minimize the chance of accidents and a madman/woman starting a worldwide holocaust. Science fiction robots (the good ones, anyway) are programmed to do humans no harm.

Today we know better than ever that those same humans, especially in matters of security� computer and otherwise�are always the weakest link. Perhaps the next generation of programs and systems should make it a priority to deliver new levels of personalization and ease of use along with more powerful protections that make it much harder to do harm�even at our busiest, laziest, greediest, most distracted moments. Great power requires great caution and protections.

The irony of a man who earned his living solving computer problems being struck down by one was not lost on Scott Jerome-Parks, the CIBC computer analyst, according to The New York Times. Nor should it be lost on us.