I downloaded that data and plan on making this data available here on the blog for other analysts to look at. If this is popular, I will think of posting more generated datasets to this site. I also think this could give some readers of this blog a taste of what social media data looks like, and what it's like to work with it.
METHODI used the Twitter API to pull down the most recent 500 tweets for each current sitting US Senator on Saturday night, December 2nd 2017. For this I used a list of Twitter handles. A couple of notes:
- I found this list on the internet, and made the obvious changes, so if you find any errors please let me know.
- Senators often have multiple Twitter handles, so I'm hoping I have the right one for this type of policy discussion.
If you have any issues on this blog you can either comment on this blog (open comments) or hit me up directly on Twitter at @leviabx.
The data I pulled down is a list of tweets found here. This data was pulled mid-day on December 2nd so it includes tweets from immediately after tax bill passage. A few notes on (some of) the data fields:
- text: This is a cleaned version of the text from the original tweet.
- original_text: this is the original text from the tweet.
- created_at: the UTC timestamp of when the tweet was sent. This is a standardized time that is hours ahead of US eastern time.. impact: subtract 5 hours to get it to DC time.
- emotions(anger, anticipation, positive, negative): for the user's convenience I ran this data through a sentiment algorithm (see Plutchik's 8 emotions).
- tax: a TRUE/FALSE indicator of whether "tax" was mentioned in the tweet.
- tl: this is a link to go look at the tweet directly in browser (just copypasta it to the browser). I wrote this piece of code 3 years ago and have no clue what I meant by "tl".
- screen_name: the screen name of the senator who sent the tweet.
- geo fields: There are a ton of geo location fields for Twitter data.. mostly to be ignored as it's only filled out on opt in from the user.
- retweet/fav counts: number of times an individual tweet is retweed or favorited.
|WordCloud of related tax related tweets from Senators during the week of tax reform.|
Playing with this data can be interesting and somewhat fun. Here are some use cases you can do, from least to most technical:
- Find your Senators and see what they Tweeted this week.
- Sort the spreadsheet by "created_at" and follow the tweets by the timeline of bill passage and after bill passage time.
- Find tweets you like/dislike (search, emotion, names), then use the "tl" field to go to the Tweet directly and react.
- By sorting the emotion fields, find the Senators who were the most happy (trust, joy) versus least happy (anger, disgust) about the bill.
I'm not going to work on this dataset extensively, but I did pull together the happiest and angriest tweets regarding tax reform:
The awful #GOPTaxPlan will hurt commuter benefits and transportation projects. As if your daily commute wasn’t bad enough already. https://t.co/9IAcFfATX4— Sen Dianne Feinstein (@SenFeinstein) November 30, 2017
The TrumpTax bill is shameful on so many levels, but the parts attacking students are particularly awful. Why in the world would we raise taxes on grad students? Oh, right--billionaires need their tax cuts. Give me a break! https://t.co/6YPQXEZxYH— Sheldon Whitehouse (@SenWhitehouse) November 28, 2017
Good news: we finally have a tax bill to look at.— Senator Patty Murray (@PattyMurray) December 2, 2017
Bad news: we can’t read it.
Worst news: we may be voting on it in just a few hours. pic.twitter.com/deiAOCscol
The Senate has taken an important step toward providing real tax relief for Americans & we must continue working to get a bill to @POTUS's desk this year. #TaxReform— Senator Roy Blunt (@RoyBlunt) December 2, 2017
The Senate’s historic tax reform bill would help boost the economy, create jobs, simplify the tax code and provide meaningful tax relief to hardworking Americans. https://t.co/E0CyPzSumw— Mike Enzi (@SenatorEnzi) December 2, 2017
Top economists: the Senate Finance Committee tax proposal will lift the economy, could boost GDP 3-4 percent long term. https://t.co/KXuxvIdJsv #taxreform— GOP Senate Finance (@GOPSenFinance) November 27, 2017