Black Box vs White Box approaches to develop AI models
From personalising recommendations on Netflix, Amazon, etc to optimising power supply from electrical grids, Artificial Intelligence (AI) is rapidly changing the face of science, business and everyday life, and in every application, an AI model serves the purpose. Irrespective of the domain the model hails from, be it Machine Learning (ML), Deep Learning (DL), Reinforcement Learning (RL) or even any other domain, the developers employ either a Black Box approach or a White Box approach.
In this blog, we will provide a brief introduction to the two approaches, followed by the pros and cons of each of the two approaches. We will also look at some examples where these approaches have been exploited and how they have turned out. Whether good or bad, we will keep it as a mystery for now, a mystery which we shall unravel further down our journey.
A Brief Introduction
Black Box and White Box approaches are general methodologies that are not limited to AI. They are used to develop conventional software, they are used to test software, and among a multitude of other uses, they are also used to develop AI models. In order to give a formal definition to them, we will go through an example of developing an AI model using both the approaches.
Let’s say that we want to develop an AI model ‘H’, which based on our inputs ‘X’, predicts ‘Y’, i.e., H(X) = Y. If we develop the model using the black-box approach, we can see and exploit the results in any way that we may want, but what we cannot do is understand the logic behind why H predicted Y for X. In contrast, if we develop the model using the white-box approach, we will obtain both the results as well as the logic behind those results. Now, at this point, one might hop into the boat of the white-box approach, but we will shortly see why this decision may backfire in some cases. So, without any further ado, let’s begin with the pros and cons of both the approaches.
White Box Approach
If a model is developed with white box approach at its core, then, by definition, the decision making process of the model will be completely transparent, and with the increasing use of AI models in automated decision making in each and every domain, this becomes essential to a great extent. It enables the users to audit the decisions made by the AI models, if and when needed. This includes answers to questions such as “Why the model made a particular decision?”, “What are the most influencing variables for a particular decision?”, “What are the assumptions taken by the model to reach a particular decision?”, and many more.
This interpretability becomes even more important, when the risks involved are exponentially greater. A simple example could be AI models involved in the healthcare domain. In this scenario, human lives are at risk, and hence, it is of utmost importance for the medical practitioners to make sure that the model is making legitimate assumptions and is getting influenced by the appropriate factors.
However, at the same time, this interpretability, which is supposed to make these AI models much more reliable; exposes them to vulnerabilities which are often exploited for unacceptable purposes. Let me present an astounding example that I came across in the Batch of 30th March, 2022. Researchers at Collaborations Pharmaceuticals built a drug discovery model to design useful medical drugs, and in the process, they developed a ranking function that penalised toxic drugs and rewarded drugs with greater biological impact. In an experiment, the researchers reversed the toxicity factor, prioritising the deadliest compounds with the greatest effect, and the model went on to generate 40,000 toxins in just a matter of 6 hours, some of them being actual chemical warfare agents that weren’t even present in the initial dataset.
This discussion very clearly begs the question, “If white-box approach introduces vulnerabilities, then should we employ black-box approach to develop our AI models?”. Let’s answer this question in the upcoming section.
If you are unaware of the Batch, it’s one of the best newsletters on Artificial Intelligence, that exists out there. I have attached the link to the same towards the end of this blog!
Black Box Approach
As you might be already familiar by now, a model that is developed with black-box approach at it’s core provides minimal interpretability to it’s users. This makes the black box approach a poor choice for applications where the risks involved are considerably great. However, at the same time, this saves the model from exposing its vulnerabilities.
Consider an example in which a bank, needs a model that can predict whether a loan should be given to an applicant or not, based on a multitude of factors, such as the applicant’s credit score, debt-to-income ratio, size of down payment, liquid assets, value of the collateral, and many more. Let’s say we begin our modelling with a Decision Tree model (a highly interpretable model, i.e., an approximation of white box approach). Now, somehow a broker got hands on the model, and thanks to its interpretability, he/she discovered that the model seems to accept applicants having high-valued collateral, despite of having low credit score and high debt-to-income ratio. Using this fact, the broker got loans approved for a number of applicants, and made a hefty commission.
Now, consider the same example, in which we used a Random Forest model (comparatively less interpretable, i.e., an approximation of black box approach). In this scenario, even if the broker got hands on the model, it is highly unlikely for the broker to exploit it, thanks to its opaque nature. This is what makes the black box approach a more suitable one when the risk of exposing vulnerabilities is too high.
Using the black-box approach also comes with an additional advantage. Models employing the black-box approach doesn’t have any constraints as far as its interpretability is concerned. In this scenario, it is not uncommon for these predictive models to be such complicated functions of the variables that no human can understand how the variables are jointly related to each other to reach a final prediction. However, this tends to give the models an extra boost in terms of its performance, since the only goal in this scenario is to perform better and make more profits, and when this happens, corporations often tend to settle in on the black-box approach, or at least, this is what has been happening since many years.
If you are interested to learn more about Decision Tree & Random forest models, and why Decision Tree models are more interpretable than Random Forest models, then you can find some great resources towards the end of this blog!
Are the tables turning?
If you are one of those who thought that the blog would have been concluded by now, then let me assure you, I was one of you, until I came across this article by Cynthia Rudin and Joanna Radin. This article highlights the trend that has recently surfaced, i.e., the growing concerns of the common populace about how their lives are affected on a day-to-day basis with the increasing use of AI, and how this very trend is strong arming the corporations to shift towards the white-box approach. Additionally, this article raises a very significant question, i.e., “Why are we using black-box models in AI when we don’t need to?”. It also explains how the belief that accuracy must be sacrificed for interpretability is inaccurate, and provides reference to some of the most noteworthy efforts taken up by researchers to tackle this belief.
I would urge each and everyone of you to read this amazing article, but for those of you, who aren’t interested (till now), let me arouse you a bit more with an amazing anecdote described in this article.
In 2018, a major AI challenge was held, with the goal of creating a sophisticated black box model for the dataset and explaining how it worked, i.e., transitioning it from the black-box approach to the white-box approach. Prior to the announcement of the challenge winners, the audience was asked to participate in a thought experiment in which they were diagnosed with cancer and required surgery to remove a tumour. On the screen, there were two images. One image showed a human surgeon who could explain anything about the surgery (i.e., white-box approach), but had a 15% chance of causing death during the procedure. The other image depicted a robotic arm capable of performing the surgery with a 2% probability of failure. In this scenario, complete trust in the robot was necessary; no questions could be asked of it, and no detailed explanation of how it arrived at its judgements could be given (i.e., black-box approach). The audience was then asked to raise their hands to indicate which of the two candidates they preferred to perform life-saving surgery. All but one hand voted for the robot.
While it may appear obvious that the black-box approach is winning, this begs a very significant question, i.e., “Why must the robot be a black box? Would the robot lose its ability to perform accurate surgery if it was enabled with an ability to explain itself?”, and the answer to this, you very well know where to find.
For those of you who are interested in finding out the answer, you can find the link to this article towards the end of this blog!
A little about ME 👋
You can safely skip this section, if you have no interest in knowing the author, or you already know me. I promise that there are no hidden treasures in this section. I am an Artificial Intelligence Enthusiast and a Musicophile. Currently, I am exploring something old, and meanwhile, learning something new. I really hope that you liked this blog, and if you did, do put your hands together 👏 and if you would like to read more blogs, then #StayTuned. Connect with me on LinkedIn and Twitter.
