The Signal and the Noise

How do you fancy reading 450 pages on earthquakes, the spreading of diseases, climate change and terrorism? Quite. I groaned too.

But Nate Silver’s The Signal and the Noise: The Art and Science of Prediction is a page-turner. He manages to mix history with a variety of topics in a way that is entertaining and educational. After a brief mention of big data, an historical lesson on the spread of scientific knowledge from the 15th-Century, and a discussion around terms such as ‘prediction’ and ‘forecast’, Silver takes us on a roller-coaster of various failed and successful predictions in financial markets, baseball, weather forecasting, earthquakes, diseases, gambling on basketball, IBM’s Deep Blue grandmaster-beating chess program, poker, the stock market, climate change and global terrorism.

There are recurring themes throughout the book, many of which you may have heard before from authors such as Douglas Hubbard, but each chapter is fascinating in the intricacies. It includes excellent practical advice such as giving ranges rather than specific estimates, and that aggregated forecasts are often more accurate than individual ones. Again we hear about the over-confidence in our ability to estimate and we are encouraged to learn from the uncertainty that is around us: once we are willing to acknowledge that “perfect prediction is impossible” then we will be in a better place. One of my favourite quotes is from Dr. Ozonoff of the Harvard School for Public Health, who says that “It should be a given that whatever forecast we make on average will be wrong … usually it’s about understanding how it’s wrong, and what to do when it’s wrong, and minimizing the cost to us when it’s wrong.”

But Silver takes us deeper. He acknowledges human error and introduces us to the problem of separating the important stuff (the signal) from the mass of irrelevant ‘noise’ around it. Along with lessons in correlation and causality (there is a correlation between an increase in ice-cream sales and the amount of forest fires, but there is no causality), he offers an easy introduction to Bayesian analysis. The idea that we have prior information that we add to the mix, and that as this information changes over time our outlook changes, challenges the misconception that a good prediction shouldn’t change: “the right attitude is that you should make the best forecast possible today – regardless of what you said last week, last month, or last year.” Add that to the cone of uncertainty and enjoy.

So who is this book for? I’d say anyone who is asked “when will it be finished?” will enjoy this book. It’s not always joyful reading (the last few pages are talking about the chances of a nuclear terrorist attack killing millions of people) but it is captivating and offers skills that you can apply on a daily basis, knowledge on how other fields use prediction and pointers on where to learn more from real life.

I’ll leave you with a gem from the book: “If you can’t make a good prediction, it is very often harmful to pretend that you can”.

The Signal and the Noise: The Art and Science of Prediction is available from Amazon in paperback  for £7.49 or on Kindle for £6.64