Sunday, December 3, 2017

Twitter Data Related to the Passing of the Senate Tax Bill

Over the past few weeks I have been heavily analyzing Twitter data: looking for methods to find mass-blocking patterns (I've tweeted on this extensively at @leviabx and may write that analysis up in the future).  A few nights ago saw some users on Twitter wanting people to save tweets of Senators leading into passing of controversial tax legislation-I thought-hey that's actually super easy.  

I downloaded that data and plan on making this data available here on the blog for other analysts to look at.  If this is popular, I will think of posting more generated datasets to this site. I also think this could give some readers of this blog a taste of what social media data looks like, and what it's like to work with it.


I used the Twitter API to pull down the most recent 500 tweets for each current sitting US Senator on Saturday night, December 2nd 2017.  For this I used a list of Twitter handles.  A couple of notes:
  1. I found this list on the internet, and made the obvious changes, so if you find any errors please let me know.
  2. Senators often have multiple Twitter handles, so I'm hoping I have the right one for this type of policy discussion.

If you have any issues on this blog you can either comment on this blog (open comments) or hit me up directly on Twitter at @leviabx.


The data I pulled down is a list of tweets found here.  This data was pulled mid-day on December 2nd so it includes tweets from immediately after tax bill passage. A few notes on (some of) the data fields:
  1. text: This is a cleaned version of the text from the original tweet.
  2. original_text: this is the original text from the tweet.
  3. created_at: the UTC timestamp of when the tweet was sent.  This is a standardized time that is hours ahead of US eastern time.. impact: subtract 5 hours to get it to DC time.
  4. emotions(anger, anticipation, positive, negative): for the user's convenience I ran this data through a sentiment algorithm (see Plutchik's 8 emotions).
  5. tax: a TRUE/FALSE indicator of whether "tax" was mentioned in the tweet.
  6. tl: this is a link to go look at the tweet directly in browser (just copypasta it to the browser). I wrote this piece of code 3 years ago and have no clue what I meant by "tl".
  7. screen_name: the screen name of the senator who sent the tweet.
  8. geo fields: There are a ton of geo location fields for Twitter data.. mostly to be ignored as it's only filled out on opt in from the user.
  9. retweet/fav counts: number of times an individual tweet is retweed or favorited.
WordCloud of related tax related tweets from Senators during the week of tax reform.


Playing with this data can be interesting and somewhat fun.  Here are some use cases you can do, from least to most technical:
  1. Find your Senators and see what they Tweeted this week.
  2. Sort the spreadsheet by "created_at" and follow the tweets by the timeline of bill passage and after bill passage time.
  3. Find tweets you like/dislike (search, emotion, names), then use the "tl" field to go to the Tweet directly and react.
  4. By sorting the emotion fields, find the Senators who were the most happy (trust, joy) versus least happy (anger, disgust) about the bill.


I'm not going to work on this dataset extensively, but I did pull together the happiest and angriest tweets regarding tax reform:

First angriest:

Now happiest:


  1. myTectra Placement Portal is a Web based portal brings Potentials Employers and myTectra Candidates on a common platform for placement assistance

  2. It's good that such blogs exist. I will wait for your next post!
    Latest reviews

  3. I gotta most loved this site it appears to be exceptionally useful . Buy Twitter Retweets Cheap

  4. I haven’t any word to appreciate this post.....Really i am impressed from this post....the person who create this post it was a great human..thanks for shared this with us. about Isro

  5. تركيب جسم النمل يمكن تمييز النمل الأبيض من الأنواع الأخرى من الحشرات، من خلال الانتباه لتركيبة جسمه، حيث يمتاز بعدة أمور ومنها: امتلاك النمل الأبيض جسم مستطيل، حيث إنه لا يمتلك أي خصر في مركز جسمه. امتلاك النمل الأبيض قرون استشعار مستقيمة. امتلاك النمل الأبيض أجنحة متساوية الشكل والحجم، كما تمتاز أجنحتها بكونها أطول من جسدها.
    شركة مكافحة النمل الابيض ببريدة
    شركة مكافحة النمل الابيض بالاحساء
    شركة مكافحة النمل الابيض بخميس مشيط
    شركة مكافحة النمل الابيض بالجبيل

  6. Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with extra information? It is extremely helpful for me. Buy Twitter Likes

  7. Delicious food, beautiful nature, amazing people what more do you want from a country!! Everyone should visit the Albanian Alps. Albanian Roots

  8. Incredible model in LinkedIn Today that was propelled in March this year.check

  9. You made an article that is interesting.You are so amazing.
    Thanks a million and please keep up the effective work. clipping photoshop

  10. Attend The Analytics Training Institute From ExcelR. Practical Analytics Training Institute Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Analytics Training Institute.
    ExcelR Analytics Training Institute

  11. Cool stuff you have and you keep overhaul every one of us
    ExcelR data science course in mumbai

  12. Very good and informative article. Thanks for sharing such nice article, keep on updating such good articles.
    MS Office training in Chennai | MS Office training in Madipakkam

  13. Very good and informative article. Thanks for sharing such nice article, keep on updating such good articles.
    Basic Computer Course in Chromepet | Basic Computer Course in Chennai

  14. Awesome post. Really you are shared very informative concept... Thank you for sharing. Keep on updating...
    MCSA exam center in Chennai | MCSA exam center in pallikaranai

  15. Awesome post. Really you are shared very informative concept... Thank you for sharing. Keep on updating...
    UI Path training center in Chennai | UI Path training center in pallikaranai

  16. Very good and informative article. Thanks for sharing such nice article, keep on updating such good articles.
    Adobe Photoshop training center in Chennai | Adobe Photoshop training center in Velachery

  17. Very good and informative article. Thanks for sharing such nice article, keep on updating such good articles.
    MS.Excel Training center in Chennai | | MS.Excel Training center in Velachery

  18. Very good and informative article. Thanks for sharing such nice article, keep on updating such good articles.
    PCB Design training in Chennai | | PCB Design training in Besant Nagar

  19. Part B premium consistently for each full year time span you were qualified for Part B however not took on it-yet Bubba maintained a strategic distance from the punishment by notbuy modafinil cheap with free shipping,buy modvigil in australia

  20. I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the end. I would like to read newer posts and to share my thoughts with you.

    business analytics course in malaysia

  21. This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value of providing a quality resource for free. smm panel

  22. Odds are you have found out about Twitter and that isn't astounding. Anyway to fruitful with it you need a decent bit by bit preparing has one of the top rankings out of all that you will discover offered out there. Twitter Video Download

  23. Incredibly best man toasts, nicely toasts. is directed building your own by way of the wedding celebration as a result are supposed to try to be witty, amusing and consequently unusual as well as. best mans speech Insta followers

  24. The social media world is dynamic; be that as it may, the activity of the PR individual continues as before: produce positive inclusion across media to help assemble brands. best smm panel

  25. I like to follow people in twitter who say something of value, just like this.

  26. One of these devices that was not intended for advertising yet rather was an informal organization is Twitter. get twitter followers