Wednesday, May 20, 2015

Halfway Book Review: The Signal and the Noise

This post should make my librarian wife quite happy, as it is a book review.  And actually, a book that she picked out for me to read, Nate Silver's The Signal and the Noise.

I'm currently only halfway through the book, but I have a lot of thoughts, and not sure I'll get around to writing a blog entry when I finish.  To summarize, the book is an explainer for why some predictions are good, and why some are bad.  The book is written from a simplistic perspective, such that even people with no background in statistics can understand the underlying concepts.

Silver walks the reader through several fields where predictions are made: predicting baseball player performance, predicting earth quakes, predicting the weather, predicting economic downturns, and predicting election results.  He takes us through the reasons that many fields are bad at predictions (earthquake research) and some are fairly good (weather predictions, by the government).  

A big downside to this book, if you are someone who makes predictions for a living, and your job depends on it, is that there are a lot of stories of failed predictions.  Specifically anecdotes about when researchers were confident in their prediction and confidence levels, but ended up being horribly wrong.  If you make predictions for a living, some of these stories are nightmare inducing, or at least sleep reducing.

A theme that runs through the first half of the book is that context matters a lot in predictions, and that forecasters need to understand things that aren't represented in data.  Examples of this may be how well a political candidate speaks in public or if a baseball player likes to party or takes care of his body.  These may not be evident in original data, but will impact eventual outcomes.  (My guess is that this later turns into a pitch for bayesian statistical methods, but we'll see.)

This take on context in political predictions takes me back to an earlier post on this blog, where I was critical of  Nate Silver for making a potentially out of context prediction about the Kansas 2014 Governor's race.  My specific criticism was that I felt a forecast that gave Paul Davis a major edge was seriously lacking in Kansas social and historical context. Upon reading this book, the political context was important (which Silver would undoubtedly agree with) and I was just the advantaged forecaster with 30+ years of living in Kansas (a lot of contextual knowledge).  

In summary, I definitely recommend this book to anyone who 1. it won't keep up at night and 2. who wants to understand the way statistical predictions work.  My wife has read this book, and I think it gave her some great insight into the way I think about and approach problems.  But you don't have to take my word for it (SORRY ALWAYS WANTED TO DO THAT).

No comments:

Post a Comment