Cloudera Fast Forward Labs is a machine intelligence research company.

Post

The Business Case for Machine Learning Interpretability

Last week we launched the latest prototype and report from our machine
intelligence R&D team:
Interpretability.

Our prototype shows how new ideas in interpretability research can be used to
extract actionable insights from black-box machine learning models; our report
describes breakthroughs in interpretability research and places them in a
commercial, legal and ethical context. This research is relevant to anyone who
designs systems using machine learning, from engineers and data scientists to
business leaders and executives who are considering new product opportunities.

We will host a public webinar on
interpretability on September 6
2017, where we’ll be joined by guests Patrick Hall (Senior Data Scientist at
H2O, co-author of Ideas on Interpreting Machine
Learning)
and Sameer Singh (Assistant Professor of Computer Science at UC Irvine,
co-creator of LIME, a model-agnostic tool
for extracting explanations from black box machine learning models). There will
be lots of opportunities for the audience to ask questions, so we hope you’ll
join us!

In this post we’re going to look at the business case for interpretability.
What advantages are there to building interpretable machine learning systems?

The Power of Interpretability

A model you can interpret is one you, regulators, and society can more easily
trust to be safe and nondiscriminatory. A model whose decisions can be
explained opens up the possibility of new kinds of intelligent products. An
accurate model that is also interpretable can offer insights that can be used
to change real-world outcomes for the better. And a model you can interpret and
understand is one you can more easily improve. Let’s look at these in more
detail.

Enhancing Trust

If we are to trust a machine learning model to perform accurately, and to be
safe and non-discriminatory, it is important that we understand it.

A paper by Rich Caruana and
colleagues gives a
powerful example of the danger of deploying a model you do not understand. They
describe a model that recommended whether patients with pneumonia should be
admitted to hospital or treated as outpatients. This model was interpretable,
and it was immediately obvious by inspection that the model had acquired a
lethal tendency to view pneumonia patients who also have asthma as low-risk.
This was wrong, of course. Asthma patients with a pulmonary infection
should be admitted to hospital. In fact they often are, and this was the
problem. Such treatment makes their prognosis better, and this pattern was
present in the historical training data.

This problem was only spotted because the model was interpretable. We discuss
what makes a model interpretable in more detail in the report. Similar risks
lurk in any model applied in the real world.

Satisfying Regulations

In many industries and jurisdictions, the application of algorithms is
constrained by legal regulations. Even when it’s not, it should be constrained
by ethical concerns.

It is extremely difficult to satisfy regulations or ethical concerns if the
model is uninterpretable. For example, the US Fair Credit Reporting Act
requires that agencies disclose “all of the key factors that adversely affected the credit score
of the consumer in the model used, the total number of which shall not exceed
4.” It’s difficult to satisfy this regulation if your credit model is a deep
neural network. Difficult, but as we show in the report, thanks to new
research, not impossible!

Explaining Decisions

Some kinds of interpretability allow you to offer automated explanations for
individual decisions. A model that makes a prediction is useful, but a model
that tells you why the prediction was made is even more powerful. Our prototype
product, Refractor, demonstrates this. It offers explanations for the
reasons customers are predicted to leave a subscription business. This allows
the user to identify both global weaknesses in the product, and individual
complaints. Most excitingly, it raises the possibility of intervening to change
outcomes.

Refractor is built on top of LIME, a
model-agnostic tool that can be applied to a trained black-box model, so data
scientists don’t need to change the way they build their models to apply this
technique. Our report describes our application in conceptual and technical
terms, and looks closely at how it could be deployed in a business.

Improving the Model

Finally, interpretability makes for models that simply work better. Debugging
an uninterpretable, black-box model is time consuming, and relies at least in
part on trial-and-error. Debugging an interpretable model, however, is easier
because glaring problems stand out. Having insight into the attributes that are
likely causing trouble can motivate a theory about how the model works, which
saves time when tracking down problems.

How to Access our Reports and Prototypes

The future is algorithmic. White-box models and techniques for making black-box
models interpretable offer a safer, more productive, and ultimately more
collaborative relationship between humans and intelligent machines. We are just
at the beginning of the conversation about interpretability and will see the
impact over the coming years.

We offer our research on interpretability in a few ways:

Annual research subscription (for individuals and corporate members)

Subscription and advising (research and time with our team)

Special projects and workshops (help to build a great data product or
strategy)