Thursday, March 19, 2015

Political Predictions, Context Matters

Starting about a year ago (March 2014) I started hearing predictions about politics in my home state that ran counter to everything I knew about Kansas politics.  Pundits, journalists, and pop statisticians (read: Nate Silver) had started predicting that Sam Brownback would lose the Kansas governor's race.
Brownback is very conservative and so is Kansas, generally.  He won the 2010 election in a 63% to 32% landslide that was never really any kind of race.  So why would he be so vulnerable only four years later?

The answers lay deeply in Kansas politics of the last four years, and you can read more on it at many different sources.

The short answer is this: Brownback cut taxes which caused a budget crisis, then tried to cut education, didn't give employees a raise, cut art spending, etc.  There were additional criticisms largely because Brownback isn't progressive on women's issues (abortion), was against gay marriage beyond the point that it was socially palatable, and didn't support unions.  Here's the Daily Show take on it:

Back to statistics.  The idea that Brownback would lose this election was still ludicrous to me, even with the craziness of the past four years. Here's how I saw each side:

Brownback Loss Factors:

  • Polling data was mixed, but generally showed a small Paul Davis lead.
  • The narrative in the mainstream media was generally that Brownback had really screwed over Kansas, and that people would be stupid to vote for him.

Brownback Win Factors:

  • Prior election results (30+% victory).
  • More motivated conservative base, and not a strong progressive candidate running against.
  • Mid-Term election with a (African American) democrat in the Whitehouse.

As an analyst it's tempting to rely only on polling data (or on modeling polling data) and projecting this, because that seems the most quantitative option.  But I didn't have a good feeling about those polling results.  And that gets us to the point of this blog entry: Context Matters.  

The day before the election Nate Silver was still predicting a Davis win, but I was not.  Here are our projections:  Silver: Davis has an 82% chance of winning.  Me: Sam Brownback wins by 2-5%.

The result of the election was a 3.5% Brownback victory.

So why were outside analysts so wrong about the Kansas election?  I see a few reasons, generally related to the model lack of context:

  • Models didn't account for level of voter motivation. (all below reasons feed this)
  • In mid-terms, the out-of-power party (president not in whitehouse) has higher motivation (more pissed off).  
  • Kansas has a strong conservative skew, consisting of many people with a visceral hatred of Obama, making them especially motivated.
  • Kansas conservatives vote for what they believe to be meta-right/wrong issues such as abortion and gay marriage.  They generally believe that if they don't vote correctly on these issues, they burn in hell (quite motivating, if you believe in that sort of thing).
  • Going into election day, there was a "shame" in saying you supported Brownback.  I assumed many Brownback voters would deny their allegiance, thus making polling inaccurate. 
Prior to the election I developed a model that looked at broader context, history, and accounted for this being a midterm (also looked at Presidential Polling data).  My model was similar to Silver's, in being logistic regression, but added value of the context of Kansas politics in a mid-term election. The added context variables, however, were important in allowing me to make a more accurate prediction of the outcome of the election.

Point of this all:  if you want to make accurate predictions, make sure you properly account for voter motivation, and more broadly, contextual factors.

No comments:

Post a Comment