Noise — humbling lesson in inaccuracy by Kahneman, Sibony and Sunstein

Noise has an extraordinarily tough act to follow. Thinking, Fast and Slow, the lead author’s last book, is a towering masterpiece. Many people who care about rationality and good judgment consider it to be the best book on the topic ever written (I am one of them). Noise is not a sequel, but it is being pitched and packaged as a companion volume, setting it up for a tough comparison.

Daniel Kahneman, Olivier Sibony and Cass Sunstein do live up to that earlier book in one specific sense: they make a convincing case that Noise’s topic is just as important as that of Thinking. The older book was about bias, the way our judgments are wrong in consistent, predictable ways. The new book is about how judgments are predictably inconsistent — scattered apparently randomly around the target, rather than veering off in a particular direction.

Random scatter in estimates and predictions may seem benign, if the average judgment is not too bad. So it is with the much-discussed cases of “the wisdom of crowds”, where a large group’s average guess at, say, the number of jelly beans in a jar turn out to be remarkably accurate. But in most contexts judgments that err in different directions do not cancel out. If one convict receives an excessive sentence, and another gets off way too easily, justice has not been done in aggregate. It has been violated twice. An insurance company that pays out too much to one claimant and much too little to another has lost money in one case and is likely to lose customers in the other.

The authors marshal evidence from law, medicine, business and elsewhere to show that noise does at least as much harm as bias and that it is not irreducibly random. Its various causes can be isolated and resisted. Interestingly, even in cases where the target does not have a single objective value, such as jury awards for damages, noise is demonstrably dangerous, because it is unfair and undermines credibility.

Still, the book faces a challenge, which is parallel to the challenge that decision makers will face when they try to eliminate noise. Bias comes in a large number of different, interesting forms, from loss aversion to overconfidence to confirmation bias, and Thinking was a varied tapestry. Noise is a single, deep problem, which the present book pursues with dogged persistence. So it’s a bit of a harder go.

Most important is a commitment to humbleness about our ability to make nuanced judgments accurately

Bias is simply sexier than noise: it feels wrong or even wicked, whereas noise comes from a very human failure to think in a disciplined, statistical way. Eliminating noise from judgments is less a matter of enlightenment than grinding out the right procedures (the authors call it “decision hygiene”). Make sure to think about each case not as unique but as a representative of a larger, statistically regular class; be ready to renounce your gut instinct; break down complex judgments into smaller ones; let members of a group come to truly independent conclusions before aggregating them; and so on.

Most important of all of these is a commitment to humbleness about our ability to make nuanced judgments accurately. Decades of research shows that people are not good at this, and substitution of simple, clear rules for individual discretion almost always renders better results.

This creates a deep tension that is the subject of some of the book’s most interesting passages. We are committed to the notion that in judgments of many sorts — legal rulings, college admissions, job applications, performance reviews, and so on — each individual deserves a unique hearing, acknowledging their particularity. We also want, quite rightly, the opportunity to shift our decision criteria on the fly as the world and our values change. But the evidence is clear. Giving individual decision makers room to tailor their judgments in response to seemingly crucial details of a case almost always makes decisions in the aggregate less accurate, less fair, and more costly.

Many of us are accustomed to thinking about the trade-off between what statisticians call type 1 and type 2 errors: roughly, balancing the risk of believing in something that isn’t there against the risk of failing to believe in something important. You can be more sensitive, or less wrong, but usually not both. Noise suggests that there is an analogous and equally important trade-off in judgment, between allowing flexibility and creativity and limiting random errors. It’s a balance we all have to strike.

Noise: A Flaw in Human Judgment, by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein, William Collins, RRP£25/ Little, Brown Spark, RRP$32, 463 pages

Robert Armstrong is the FT’s US finance editor

Join our online book group on Facebook at FT Books Café