Friday, March 6, 2015

On Weird Metrics

In my job, I spend a lot of time telling people they're looking at the wrong metrics.

Yesterday I came upon an article that many people have read this week, reacting with varying levels of shock, outrage and apathy.  The article was Justin Wolfer's New York Times story on the "glass ceiling" which pointed out mainly that more men named John run companies than women in total run companies. (Found here)  There's a fun death penalty analysis coming, but first I'll be nerdy about metrics...

My first reaction to this story was... what a bizarre freaking metric.  I understand what they are trying to do here, and it's basically a rhetorical ploy like this:

X is so much bigger than Y, that even G (a subset of X) is four times the size of Y.  
This device is used quite a bit, the most common use I can think of is when we talk about California having a bigger economy than most countries.  Basically, the US economy is so large, that even California has more going on than most other countries.  But why is this more powerful than just saying the US is the worlds largest economy and is X% larger than its nearest competitor?  It's not.  But people feel like it is a useful rhetorical device to invoke emotion and scale.

The root problem with this strategy is that it creates a compound metric where you're measuring both the size of the comparison group (in this case women running large companies) and the relative frequency of the subgroup.  The relative frequency of the subgroup will vary over time, space, and other dimensions that correlate, further perverting the metric.  The direct and meaningful metric here is simply the ratio of men to women running large companies.  

I don't miss the point that Wolfers is trying to make with his article, which is that fewer women run large companies.  But I wondered if people on the other side of the argument (Men's rights activists versus Women's rights activists) applied the same strategy, would they see similar results.

Men's rights activists often argue things like men live shorter lives and are more likely to be imprisoned or executed.  So let's look at execution numbers, because they're easily available.  I downloaded the Death Penalty Database from deathpenaltyinfo.org, so my analysis is as accurate as the data they compiled.  Then I just categorized names, nothing fancy, no name rooting or stemming.
From Wolfer's article, the ratio of men named John to Women running large companies is 1.29. (5.3% versus 4.1%)

Here are some ratios from my analysis:


As you can see from the analysis, the ratios are much higher than in Wolfer's article, so do I conclude that recent execution gender disparity is more severe than large company CEO gender disparity?  Maybe.  

But in statistical terms, it's more valid just to say that women make up 1.1% of all executions since 1976 in the United States versus 4.1% of all CEO's.

2 comments:

  1. If you have been paying any attention to Scottish politics, you will know there are more giant pandas in Scotland than Tory MPs.

    How about that as a bizarre metric?

    ReplyDelete
    Replies
    1. That's a great one!

      Though I have to admit, I googled "giant pandas in Scotland" to better understand the metric.

      It points out an important point I don't think I made clearly in the original post. These types of metrics are powerful rhetorical devices, but generally not great "measurements," and thus statisticians and serious economists should avoid them.

      FYI, for anyone reading this and not familiar with Scottish Politics, it appears that there are 2 Giant Pandas in Scotland, 59 total MP's, only one of which is a Tory.

      Delete