Monday, April 27, 2015

My Data Science Toolkit

People often ask me what software, programming languages and tools they need to learn to enter the analytics/data science field.  Some focus on products currently creating a lot of buzz like Hadoop or Apache Drill, but I spend the majority of my work time in other products that I consider to be much more important (I work in SQL more than Hadoop, no really!).

Anyways, I decided to compile my top 5 software tool list for Data Science.  This of course isn't an exhaustive list of tools I use, but it's a good start for any new data scientist.  If a job applicant came to me with just this list of tools, as well as a great statistics background, I would likely hire them immediately.


  1. R: R is still my primary statistical and analytical tool.  As I've mentioned before on this blog, we are running as server version that live-decisions transactions on the fly, processing and predicting on individual transactions in less than a second.  I still find R to be a robust and diverse system for statistical algorithm programming, and to have a great array of machine learning algorithms, as well as a large user base to consult with when I have problems.  Homepage.
  2. Python: While I don't use Python for statistical programming, there are still a number of non-statistical applications for Python.  Generally speaking, when I feel like I'm breaking new ground doing something, I seem to be using Python somewhere in the process.  Most recently I've used Python to spool up loops of large processes and to convert speech to text and dump results into a database for later analysis. Homepage.
  3. QGIS: Most of the data I encounter has a spatial element of some type, sometimes important, and sometimes not.  While both Python and R can process GIS data and display it, I prefer QGIS because it's designed specifically for GIS data, and has a lot of out of the box tools that help a GIS novice like myself. Also, it has Python internals, so if the functionality isn't there, I can script it out in a language I already know! It's like a smaller version of the industry standard ArcGIS, but is free and open source.  With ArcGIS's high cost, I have never been able to make a business case for it, especially with QGIS just a free download away.   Homepage.  
  4. SQL: Sure, nosql databases and Hadoop are all the rage right now, but 90% of the data that I have to analyze are in some kind of SQL database.  Obviously, there are a lot of flavors; when I'm doing a project on my own, I tend towards PostGRES. I was also quite impressed with the new versions of MySQL workbench, which is approaching the functionality of the exclusively commercial Microsoft SQL Server Management Studio.
  5. Notepad ++: A text editor, seriously?  In reality, I use vim quite a bit too when in Linux production, but Notepad ++ is a tool I use on a daily basis.  Why?  Primarily because Notepad ++ works across all languages I use, and can use it for my deploy scrips, SQL, R, Python, XML, and any other language I may encounter.  Homepage.







96 comments:

  1. There are lots of information about hadoop have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get to the next level in big data. Thanks for sharing this.


    Big Data Training Chennai
    Big Data Training
    Big Data Course in Chennai

    ReplyDelete
  2. Thanks for your informative guide on SQL technology. As we all know, SQL is most used database tool and your article offer clear insight about the technology. Your blog is recommended for freshers and experienced ones looking for ideal blog to learn SQL technology.
    .Net Training in Chennai
    PHP Training in Chennai
    Big Data Training in Chennai

    ReplyDelete
  3. commission express franchiseI found this informative and interesting blog so i think so its very useful and knowledge able.I would like to thank you for the efforts you have made in writing this article.

    ReplyDelete
  4. This comment has been removed by a blog administrator.

    ReplyDelete
  5. This comment has been removed by a blog administrator.

    ReplyDelete
  6. Really awesome blog. Your blog is really useful for me.
    Thanks for sharing this informative blog. Keep update your blog.
    Oracle Training In Chennai

    ReplyDelete
  7. Best Java Training Institute In ChennaiThis information is impressive; I am inspired with your post writing style & how continuously you describe this topic. After reading your post, thanks for taking the time to discuss this, I feel happy about it and I love learning more about this topic..

    ReplyDelete
  8. noocube Nootropics also called smart drugs, memory enhancers, neuro enhancers, cognitive enhancers, and intelligence enhancers. Find detailed Noocube review, effects & facts. http://www.noocube-nootropics.com/

    ReplyDelete
  9. Extreme Nutritions Selling Bodybuilding,Weightloss,Women's health and Sexual Products for All and Get the Best Prices on Extreme Nutritions Supplements Legal Steroids http://extreme-nutritions.com/

    ReplyDelete
  10. Posts about history are cool! Nice one.Read these PhenQ customer reviews to find out does it work? Review of side ... It is not available at any local store close to you and not at any retailer. www.phenqreviewstore.com buy phenq

    ReplyDelete
  11. Posts about history are cool! Nice one.Read these PhenQ customer reviews to find out does it work? Review of side ... It is not available at any local store close to you and not at any retailer. www.phenqreviewstore.com buy phenq

    ReplyDelete
  12. Posts about history are cool! Nice one.Read these PhenQ customer reviews to find out does it work? Review of side ... It is not available at any local store close to you and not at any retailer. www.phenqreviewstore.com buy phenq

    ReplyDelete
  13. You can make good use of the constant Crazy Bulk coupon and the Crazy Bulk discount http://buycrazybulksteroids.com/

    ReplyDelete
  14. Hi, Here i am portraying you least complex approach to send SMS utilizing ASP.NET and API from SMS sender suppliers.woo sms

    ReplyDelete
  15. Be sure to get the best deal on Phen375. Beware of cheap imitations. Check out our Phen375 review and find out more!

    ReplyDelete
  16. Interesting post. I Have Been wondering about this issue, so thanks for posting.
    Top Legal Anabolic Steroids
    Crazy Bulk Reviews
    Crazy Bulk Review

    ReplyDelete
  17. Virginia Web Design I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post.

    ReplyDelete
  18. These provided information was really so nice,thanks for giving that post and the more skills to develop.giving articles really nice.

    hadoop training in chennai

    ReplyDelete
  19. Interesting post. I Have Been wondering about this issue, so thanks for posting.
    Crazy Bulk Buy Legal Anabolic Steroids Online

    ReplyDelete
  20. Crazy bulk is one of the ideal options to build quality and strongest muscles masses just in 2 to 3 weeks. This is the greatest product for muscle growth that provides the good muscle size. The person who is looking for the right muscle gaining supplement they can choose this crazy bulk product to gain powerful muscles as well as the crazy bulk helps to increase your stamina. http://www.crazybulkstacks.com/

    ReplyDelete
  21. This was a great and interesting article to read. I have really enjoyed all of this very cool information Buzz Applications

    ReplyDelete
  22. Male Extra™is one of the natural male enhancement pills that really work by aiding your erection size, controlling your ejaculations.Read complete user reviews here

    ReplyDelete
  23. The stock that is accessible will NEVER be less expensive than it is currently.crazy bulk

    ReplyDelete
  24. Having green tea after a blowout can be mind boggling to back off the discharge of Insulinphenq

    ReplyDelete
  25. It is additionally ready to expand the digestion system of the body with the goal that one uses up a great deal of vitality and tat way blaze the additional calories to lessen weight all the more adequately.phen375

    ReplyDelete
  26. SizeGenetics is the most comfortable male size extender available. For more details visit

    ReplyDelete
  27. There are millions of people around the globe who are interested in losing weight.superphenq

    ReplyDelete
  28. It ought to be the item line the item is critical from a viewpoint that no ought to ever get paid unless the item is soldlegal steroids
    The corporate pioneers, for example, the proprietors and VP of offers are basic to the development of the organization and the maintainability of that development.purchase steroids
    In system advertising Facts tell and stories offer so the story is most likely the greatest component that will help you make deals.sale legal steroids
    when you are searching for a system promoting organization and searching what the potential for generational riches it is about planning, timing, timing.order online

    ReplyDelete
  29. Wel done great work i have got here thank you for sharing,
    Forskolin Fuel

    ReplyDelete
  30. To keep ourselves up to date with the current trend is not an easy task in IT. But we can, through quality and worth able content like this. Thanks for sharing this web page. Please write more articles like this in future.
    PHP training in Chennai | PHP course in Chennai

    ReplyDelete
  31. Virginia Web Design I wanted to thank you for this great read!! I definitely enjoying every little bit of it I have you bookmarked to check out new stuff you post.

    ReplyDelete
  32. V Tight Gel is the best solution for vagina tightening available in market. Consist of natural and quality ingredients.

    ReplyDelete
  33. we talked to readers who knocked off 10, 25, even 60 pounds with some easy tweakssale legal steroids

    ReplyDelete
  34. Looking for proven, effective Dianabol for Sale without side effects? Discover premium Dbal-Max Pills currently helping guys build mass safely. http://www.dbalmax-reviews.com/

    ReplyDelete
  35. Rolex Watches Authentic Mens & Ladies Rolex Datejust, President Watches for Sale at JavyEstrella.com.

    ReplyDelete
  36. Thanks for sharing this informative content that guided me to know the details about the training offered in different technology.
    digital marketing course in chennai | digital marketing training in Chennai

    ReplyDelete
  37. web development company in patna:-
    rkvit solutions pvt ltd is a leading software company in patna. we, bring successful years in it world along with the team of highly professional expertise. our service includes website development in Patna, website design in patna, web design in patna, seo company in patna.
    for more details:
    http://rkvitsolutions.com

    ReplyDelete
  38. PhenQ is simple and clear that is why the main ingredients and the working is mentioned on each piece of this product so that the customers can get the idea of what actually will be done with their body to reduce their weight.PhenQ diet pills have been the best-selling weight loss option for over a year and its perfect for people who are not losing any weight despite being on a low-calorie diet.

    ReplyDelete
  39. PHP provides the best option to build the website where we can design our website in a very interactive manner that provides better functioning in data management.
    PHP Training in Chennai | PHP course in Chennai

    ReplyDelete
  40. Well Said, you have furnished the right information that will be useful to anyone at all time. Thanks for sharing your Ideas.
    Web Designing Course in Chennai | web designing training in chennai

    ReplyDelete
  41. I usually do not use body building supplements. However, a friend referred Crazy Bulk body building supplements and I decided to give these a shot. Must say, I am really impressed. Just 30 days of regular use of the bulking and cutting products, and I am now a loyal customer. Visit the official website http://www.crazybulkstacks.com/ to know more.

    ReplyDelete
  42. Updating with the latest technology and implementing it is the only way to survive in our niche. Thanks for making me this article. You have done a great job by sharing this content in here. Keep writing article like this.
    SAS Training in Chennai | SAS Course in Chennai

    ReplyDelete
  43. There are times when you suffer the disease being treated immediately, before things got worse. Thank you for the information:)

    ReplyDelete
  44. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it. The python programming language is very popular and most widely used.
    Python Training in Chennai | Python Course in Chennai

    ReplyDelete
  45. Breast Actives
    is a 3 step all natural solution for breast enlargement, combining breast pills, cream and a breast exercise and massage program.http://www.breastactivesbuy.com/

    ReplyDelete
  46. Flat Belly Flush is a 10-day program that helps you minimize belly fat and lose up to ten pounds.for more visit http://itsallabouthealth.net/flat-belly-flush/

    ReplyDelete
  47. There are hundreds of contractors available nowadays, but finding the one that does not only charge less money, but also does a professional job – could be a hell of a difficult task; and even if you luckily find one, the problem of choosing the best waterproof flooring downriver MI solves this problem and offers its customers the best Luxury Vinyl Tile flooring in the best installation and product rates. Read more: http://lvtflooringmichigan.com/installing-the-right-waterproof-flooring-in-downriver-michigan

    ReplyDelete
  48. Packers and movers in jaipur@
    http://www.movingsolution7.in/packers-and-movers-jaipur/
    Packers and movers in jodhpur@
    http://www.movingsolution7.in/packers-and-movers-jodhpur/

    ReplyDelete
  49. One weight loss program that has gone above and beyond expectations to promote weight loss is Wake Up Lea.to know more about this program visit now http://itsallabouthealth.net/wake-up-lean/

    ReplyDelete
  50. " cognitive enhancement supplements Find out which Cognitive Enhancers Really Work: Brain Training, Drugs, Vitamins, Meditation & Exercise.dont buy before reading any review. http://www.noocube-review.com"

    ReplyDelete
  51. " Nootropics Nootropics, or smart drugs, are being used around the world by students, entrepreneurs, small business owners, athletes, poker players and housewives. http://www.noocube-review.com/"

    ReplyDelete
  52. Updating with the latest technology and implementing it is the only way to survive in our niche. Thanks for making me this article.
    Node JS training in chennai | Node JS training institute in chennai

    ReplyDelete
  53. Thanks for sharing information to our knowledge, it helps me plenty keep sharing…
    Java Training in Chennai | JAVA Course in Chennai

    ReplyDelete
  54. Male Extra pills are famous on the market. But you have to know some side effects and results. My prof review will help you. http://www.maleextrareviewguide.com

    ReplyDelete
  55. I have read your blog its very informative and impressive. Keep Updating.ERP software chennai|ERP in chennai

    ReplyDelete
  56. Testosterone, the male hormone, is basic to you remaining a vivacious, solid, and sexually dynamic person. Low testosterone triggers weariness, tiredness, sorrow, weakness, being overweight, weight pick up, fat, muscle misfortune, fragile bones, and winding down sex drive — even glucose and cardiovascular issues.
    testosterone booster reviews

    ReplyDelete
  57. Zone Téléchargement,Télécharger sur Zone Téléchargement ou Regardez en Streaming Gratuit des Films, Séries, Qualité HD en Français ou en Vost. zone de téléchargement

    ReplyDelete
  58. Zone Téléchargement,Télécharger sur zone de téléchargement ou Regardez en Streaming Gratuit des Films, Séries, Qualité HD en Français ou en Vost.

    ReplyDelete
  59. having more sex can make you live longer Male Extra pills are famous on the market. But you have to know some side effects and results. My prof review will help you.

    ReplyDelete
  60. packers and movers in india@
    https://quicksoluction.com
    packers and movers in delhi@
    http://quicksoluction.com/packers-and-movers-in-delhi.html
    packers and movers in mumbai@
    http://quicksoluction.com/packers-and-movers-in-mumbai.html
    packers and movers in pune@
    http://quicksoluction.com/packers-and-movers-in-pune.html
    packers amd movers in gurgaon@
    http://quicksoluction.com/packers-and-movers-in-gurgaon.html

    ReplyDelete
  61. I really like the dear information you offer in your articles thanks for posting. its amazing
    http://bestdietpillswork.com/phenq-review/

    ReplyDelete
  62. how to make your vagina tighter Many women also consider v tight gel after having children. During natural childbirth the vaginal muscles expand to form the birth canal for your baby. http://vtightgelreviewer.com

    ReplyDelete
  63. Testmax Nutrition is designed to enhance and maximize your genetic potential.Increases Virility Enhances Performance Maximum Sex Booster Optimum Testosterone Levels.

    ReplyDelete
  64. legal anabolic steroidsLegal Anabolic Steroids” for many performance enhancers that little three word phrase has a nice ring to it and at the same time seems like something out of fantasy land. http://www.legalmusclesteroids.com

    ReplyDelete
  65. Hi, Really your post was very informative. Today's internet era learn Hadoop Online Training will helps you to reach your goal.Selenium Training

    ReplyDelete
  66. Nice sharing. R is a language and environment for statistical computing and graphics. Want to make a career in R Programming. Learn R Programming Online Training course @ GangBoard. We are the best provider of online training on evergreen technologies.

    ReplyDelete
  67. You have done really great job. Your blog is very unique and informative. Thanks. Devops Online Training | Data Science Online Training

    ReplyDelete
  68. Great post!I am actually getting ready to across this information, Selenium Training Institutes in Chennai I am very happy to this commands.Selenium Training in Chennai | Selenium Course in Chennai

    ReplyDelete
  69. Find great deals for Penomet Premium Hydro Penis Pump Enlargment Kit System for Male Enhancement. Shop with confidence
    penomet vs bathmate

    ReplyDelete
  70. Nice blog. Thank you for sharing. The information you shared is very effective for learners I have got some important suggestions from it. erp in chennai.

    ReplyDelete
  71. hi admin.it was really good blog.My Data Science Toolkit including R,python,java,SQL explanation wise good.I get more knowledge from that blog.Thanks for posting the valuable blog.get more details about.....
    Dot Net Training in Chennai
    Android Training in Chennai
    Selenium Training in Chennai
    Hadoop Training in Chennai

    ReplyDelete
  72. provides details about v tight gel which helps in getting tighter vaginal walls and improves ...

    ReplyDelete
  73. Nice to be visiting your blog again, it has been months for me. Well this article that i've been waited for so long. http://www.bookcrossing.com/mybookshelf

    ReplyDelete
  74. provides details about v tight gel which helps in getting tighter vaginal walls and improves ...

    ReplyDelete
  75. http://m.gamevilusa.com/forums/member.php?439955-Riggssiy

    ReplyDelete
  76. http://www.fixya.com/users/iris-riggs?up=1
    http://www.atlasobscura.com/users/riggssiy
    http://www.psu.com/forums/member.php/369093-Riggssiy

    ReplyDelete
  77. I actually enjoyed your post. The way you delivered your view/opinion, was very insightful. Your contribution to this community will be very fruitful to us.car service alexandria va

    ReplyDelete