Tuesday, August 25, 2015

Comparing Sentiment: #ksleg v #ksed v #Royals

Yesterday's post was fairly popular (quite a few page views) so I thought I would extend it a bit.  What about comparing sentiments over various topics rather than over time?  It's easy to point my code at three separate topics on twitter, use my emotion/polarity identifying algorithm on each set, and compare the results.  I also pull out a couple of current events references below.


For practical methodology, I pointed the algorithm at three separate hashtags that are used quite a bit in my region:

  • #kleg: Hashtag used for Kansas government matters.
  • #ksed: Hashtag used for Kansas education matters.
  • #Royals: Hashtag used for the Kansas city Royals baseball team.
After compiling the tweets into a dataframe, I ran some simple statistics, and validated with statistical tests.  


First, when we compare the overall polarity of the tweets, people are most negative #ksleg, followed by #ksed and #Royals.  This makes some sense, people are most negative about politics, and more positive about baseball (my conclusion yesterday gave a general hypothesis to why).  I would expect people to be more positive about public education however the hashtag in question is one on more politically charged policy-related education issues.  More on that below.

In the emotion analysis, we see a little shift in negative emotion towards #ksed.  A lot of anger and sadness on that hashtag.  The Royals bring out about the same amount of anger as Kansas government, and slightly more sadness.  But education wins negative emotion (I dig into this below).

The #Royals are the most "surprised" overall, but you can see how people would have a "surprised" emotion, especially when live-tweeting a game.  

The anger at education was weird to me, so I wanted to dive in a little bit.  Here's what I found:
  • Prior to early yesterday afternoon, #ksed tweets were very positive.
  • Early afternoon yesterday the tweets turned more angry (50% increase in negativity in tweets using hashtag #ksed).
  • I checked reality, and this happened, which appears to have spurred the negative twitter storm.
So the negative tweets are at least explainable, and coincide directly with a real-life event.  

One final note on something I noticed in data: a cluster of extremely negative tweets, all referencing the word "fraud."  I realized that these tweets were referencing an issue I've talked about a few times before, Kansas Election Fraud.

Anyways, I dove into tweets using the terms "election fraud" related to Kansas.  Findings were interesting.  First, this is the most negative collection of tweets I've ever analyzed.  Second, only two real emotions were popping up, Anger and Fear.  Here's what the polarity looks like (can be compared to polarity above for reference).

No comments:

Post a Comment