I've just finished Malcolm Gladwell's new book Blink: The power of thinking without thinking. He's not suggesting we abandon careful thought. Rather, he shows that how we select the things we consider when we make a decision determines in large part how good our decisions are.
We called it analysis paralysis when I worked for IBM many years ago. We all want to be as secure in our decisions as we can be. Often that means we engage in rampant data collection even when the factors that we gather don't have clearly demonstrated correlations to our concerns. So we over-sample all the inputs we can get our hands on and delude ourselves that we're better able to make decisions because we have tons of data.
I digest Gladwell's thesis in this book down to this point: Accumulating and considering too much uncorrelated data produces inferior decisions when compared with decisions based on properly filtered data. When we have just the right information, we can make decisions very quickly indeed. That's when we start to blink, rather than to plod along.
The art is to know how to gather just the right information to make a good decision. Gladwell examines a wide variety of examples where experts in a particular domain jump precisely to the right conclusion because they know what's important to know, and they're not distracted by other factors. He examines Cook County General's evidence-based rubric to determine which patients to admit for cardiac care. Doctors formerly over-admitted patients who might be having a heart attack. Their new decision tree uses a few easily determined factors to assess who needs acute cardiac care. There's a particularly fascinating story about how a battle-seasoned Marine officer out-planned a high-tech set of new military tools.
But how do we know when we have just the right information - the right filtration of raw data - to make the best judgment possible. Gladwell has no magic bullet, save one. It's vitally important not to mask out important data through bias. He makes clear points about visual cues often being detrimental to musical judgment, for example.
Sometimes we can become expert observers in a domain through a program of study. In my own field I believe the best software design experts come to it through a variety of experiences - most of them bad ones. That's why IBM was caught in analysis paralysis. They'd been through far too many software development projects gone wrong. Each time something bad happened they added another rule to follow in the future. Bureaucratic procedures accrete one innocent rule at a time.
Something happened to me as I became a better software designer. I started to envision the systems I create as tangible, extended entities that I could examine in my mind's eye. Bad systems just 'look bad'. They're not aesthetically pleasing. (Just try telling that to your manager...) That's not the same thing as having a beautiful formal structure. About twenty years ago I was part of a team that looked at some telephony software. It was faithfully designed using the best software engineering practices. Unfortunately it could only process about three phone calls a minute. Formal beauty didn't translate into a feasible solution to the problem which required capacities maybe ten thousand times larger.
This case proved the worth of an old friend's maxim: solve the whole problem, not ninety percent of it. This unfortunate design solved most of the problems - but failed to meet critical performance levels. Building in performance from the start would have forced a completely different design.
So I've learned to have a sense about when a system is likely to pan out. Having that sense, and it is an aesthetic one, is probably little different from a museum curator needing years of being around art to be able to distinguish great art from a fraud. There's no way to impart this sense of software design internals to someone apart from real experience. Tons of books try to do it; I'm unconvinced that they actually do. Nothing teaches software design like building a house of cards, watching it collapse, and figuring out how not to do it again.
Do try this book. Again, there are no magic bullets here - other than avoiding hidden biases and determining those factors that are really important.
Comments