Thursday, January 15, 2015

On audit-ability in machine learning

Again a moment of honesty:  I started my career working for the Kansas Auditor's office (known as the Kansas Legislative Division of Post Audit) working on school funding, government efficiency and fraud.  I held the absurd title of "Principal Data Mining Auditor"... but.. that was a long time ago (5 years).  I don't regret my experience, though after leaving I swore I'd never work with auditors ever again, I just like being more creative than that.

Fast forward five years, and I'm working in financial services, and suddenly auditing is key again.  This is partially due to financial managers wanting to understand the "innards" of models, but also due to outside auditors and the government (CFPB) wanting to audit our decision-making algorithms.
As a result, I sometimes have to make a decision about which algorithms to use not based on performance, but based on the ability of government auditors to understand what I do.

For instance, I use an ensemble learning process (multiple algorithms) for part of our decision making, part of which uses an SVM..  I can only train the SVM on a truncated data set, missing several variables including some employment and age information, because.. in simple terms, I have to be able to prove that the output function provides a continuous one-direction first derivative.

So, this is my short list (or generalization) of audit-able versus non-auditable methods:

Can be audited:
Multivariate Regression
GLM (logistic)
Spline Regression
Simple Decision Trees
Naive Bayes


Difficult to audit:
Artificial Neural Networks
Support Vector Machines
Relevancy Vector Machines
Random Forest

24 comments:

  1. So interesting to read about auditing. I am studying this in school right now and I just think I never understood what an accountant really does and the purpose for having them around. This was a fascinating read and I am very much looking forward to reading other posts. Good to know because I might want to go into this.

    Kent Gregory @ ARMATURE Corporation

    ReplyDelete
  2. Of course, your thoughts are applicable, but for the audit, my assistant prefers electronic forms Field Operations with Mobile Forts is still much more productive than before. You should. And also download a demo version to get acquainted with the interface and functionality of this software see it here . Good luck in the audit. Thank you for the post a lot!

    ReplyDelete
  3. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command.

    Data Science Online Training|
    Hadoop Online Training
    R Programming Online Training|

    ReplyDelete
  4. Long Description Riskonnect is the trusted, preferred source of Integrated Risk Management technology, offering a growing suite of solutions on a world-class cloud computing model that enable clients to elevate their programs for management of all risks across the enterprise. Riskonnect allows organizations to holistically understand, manage and control risks, positively affecting shareholder value GRC software

    ReplyDelete
  5. Great blog.you put Good stuff.All the topics were explained briefly.so quickly understand for me.I am waiting for your next fantastic blog.Thanks for sharing.Any coures related details learn...

    Data science training in Marathahalli|
    Data science training in Bangalore|
    Hadoop Training in Marathahalli|
    Hadoop Training in Bangalore|

    ReplyDelete
  6. Your article is really an amazing with useful content, thank you so much for sharing such an informative information. keep updating.
    AWS Training Institute in Chennai | Best AWS Training Center in Velachery | AWS Training in Perungudi | AWS Training in Kanchipuram

    ReplyDelete
  7. Great post and informative blog.it was awesome to read, thanks for sharing this great content to my vision. This is a great inspiring article.I am pretty much pleased with your good work. You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post..

    Best Python Training in Chennai | Python Certification Training Centers in Chennai | Python Exams in Chennai | Python Exam Center in Chennai

    ReplyDelete
  8. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…This is a great inspiring article.I am pretty much pleased with your good work. You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post..

    Best Python Training in Chennai | Python Certification Training Centers in Chennai | Python Exams in Chennai | Python Exam Center in Chennai

    ReplyDelete
  9. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…
    Embedded System Training Institute in Kanchipuram | Embedded Training Center in Kanchipuram

    ReplyDelete
  10. Great post....Thank you for posting the great content……I found it quiet interesting, hopefully you will keep posting such blogs…Keep updating your creative and awesome blog.

    Best Python Exam Center in Chennai | Python Certification Training in Chennai | Python Training in Chennai | Python Exams in Chennai

    ReplyDelete
  11. This is a great inspiring article.I am pretty much pleased with your good work. You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post,.Thank You For Sharing This Information..
    Best CCNA Training Institute in Chennai | CCNA Training Center in Velachery

    ReplyDelete
  12. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…
    Best Linux Training Institute in Kanchipuram | No.1 Linux Training Center in Kanchipuram

    ReplyDelete
  13. Great post and informative blog.it was awesome to read, thanks for sharing this great content to my vision. This is a great inspiring article.You put really very helpful information. Keep it up. Keep blogging. Looking to reading your next post..
    Best Python Exam Center in Chennai | Python Certification Training in Chennai | Python Training in Chennai | Python Exams in Chennai

    ReplyDelete
  14. Your article is really amazing with useful information you are shared,its very helpful for everyone.Thanks a lot for sharing such an informative information.keep updating..
    Best Python Exam Center in Chennai | Python Exams in Chennai | Python Exams in Velachery

    ReplyDelete
  15. Very informative post and useful post so happy to read this article share it for everyone.Pretty post. Whatever we accumulated data from the web journals, we ought to actualize that in basically then no one but we can comprehend that precise thing obviously.

    Linux Training Institute in Chennai | Linux Training Institute in Velachery

    ReplyDelete
  16. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…
    Best Java Training Institute in Chennai | Java Training in Velachery

    ReplyDelete
  17. It’s really a nice and helpful piece of information. I’m satisfied that you just shared this helpful information with us. Please stay us informed like this. Thanks for sharing.
    MatLab Training Institute in Chennai | MatLab Training in Velachery

    ReplyDelete
  18. Nice Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one,keep updating..

    Best AWS Training Institute in Taramani | No.1 AWS Training Center in Taramani

    ReplyDelete
  19. Your article is really amazing with informative information,you are shared.Thanks a lot for sharing this wonderful blog.keep updating such a excellent post with us.
    Embedded System Training in Tambaram | Embedded Training in Tambaram

    ReplyDelete
  20. Very informative blog. Helps to gain knowledge about new concepts and techniques. Thanks a lot for sharing this wonderful blog.keep updating such a excellent post with us.
    Best MatLab Training Institute in OMR | No.1 MatLab Training Center in OMR

    ReplyDelete
  21. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.
    Best CCNA Training Institute in Guindy | No.1 CCNA Training Institute in Guindy

    ReplyDelete
  22. Very informative blog. Helps to gain knowledge about new concepts and techniques. Thanks a lot for sharing this wonderful blog.keep updating such a excellent post with us.
    AWS Exam Center in Chennai | AWS Certification Exams in Chennai | AWS Exams in Velachery

    ReplyDelete
  23. Thank you for your post. This was really an appreciating one. You done a good job. Keep on blogging like this unique information with us.
    Best Embedded Training Institute in Thiruvanmiyur | No.1 Embedded Training Institute in Thiruvanmiyur

    ReplyDelete