Machine Learning is a very hyped subject in all areas of knowledge due to the modernization brought forward by new technology models worldwide. This field is often described as the peak of automation and a mind-opener for its contributions to other fields like artificial intelligence and data science. However, some risks are often ignored by those who profit the most from using these tools.
As with most inadequate uses of statistics out there, the most significant risk in machine learning lies in poor methodology and lack of rigour. Those who wish to transform their business by implementing a modern automated decision-making model are prone to overlooking those elements. This article focuses on companies’ impact on the lives of their employees and stakeholders when automation is implemented. Precautions must be taken to ensure that implemented models are both technically and ethically sound. The same awareness needs to be brought to everyone who uses these tools in any field.
In the search for optimal efficiency and efficacy in business, managers will often reach out to new technologies to solve latent problems and improve processes and profits. In this journey, hiring big data teams and automating processes and positions is seen as a natural step to growing organizations, bringing an exponentially increasing complexity to the business environment. This complexity overwhelms most managers with several changes, modernization and, most importantly, the sudden rise in data availability. In this never-ending eyre to perfect-companyland, the first big issue with machine learning arises: automating processes becomes more important than improving processes.
At this point, the biased thought process of managers and owners brings a considerable risk to companies: the idea that rising profits and growing businesses will come as a natural consequence of automatization. It might seem evident that automating is the way to go when everything looks good on the surface. After all, if the organization appears to be thriving, the logical thing to do next would be to reproduce successful recipes on a larger scale. Right? Well, not necessarily. The issue is that what is efficient and optimal in individual processes might transpose poorly to different contexts.
When this happens, huge risks might be overlooked. Let’s consider this scenario for a moment: a family company that originated in a middle-class white household in São Paulo. The first employees were all friends and acquaintances of the founding brothers and their two male cousins. There is a big chance that most, or all, of those initial members of the company were also white, male, middle-class and lived in São Paulo. But why exactly would that be a problem? Let’s take a look at what might happen next.
Consider that the company is now expanding and that new employees from different backgrounds are being hired. The older employees will likely have better working performance than the newer ones since they are more familiar with the company’s culture, processes and general policies. Suppose we pick this moment to “improve” the company’s recruitment process with an automated machine learning algorithm. The procedure is meant to determine which characteristics make individuals perform better at work. The same algorithm will look for these exact same “successful” features in new candidates. In that model, it is very likely that being white, male, middle-class and living in São Paulo would be interpreted as predictors for better performance.
A good data team could detect that bias and exclude those factors from the model training. Still, companies often import algorithms that have been designed for different uses. These machine-learning models are often black-boxed. Knowing what factors were chosen as the best predictors for a model is challenging. That is because we will only have access to the predictions for which candidates will be the best performers during the recruitment process. I am in no way suggesting that company owners and managers are intentionally biased. After all, they might be just laymen when it comes to understanding data and how it should be cared for. That being said, the automation of a poorly defined and optimized process can lead to structural racism, sexism and elitism within company grounds.
It is essential for business owners and managers to take the hard path to implement new technology. They must make sure that the process that is being automated has been defined, improved and reviewed thoroughly. They also need to consider the ethical implications of the variables chosen as possible predictors for the model being utilized. In this scenario where automated recruitment is based on performance predictors, it is essential to think critically about what can be a defining factor to a machine learning model. For instance, if there is no apparent reason for men to perform better in a given role than women, why keep this factor in your model? In a company with hundreds of employees and balanced demographics, if men are frequently evaluated as better performers than women (even though they have the same level of education), you’re likely dealing with a biased evaluation process.
Bringing awareness to possible biases, discrimination issues that may be overlooked, or processes inefficiencies before deciding to automate the area is not just an extra step to ensure that technology works correctly. Searching for authentic organizational truth in each context is also essential. This logic can be transposed to any situation and field that seeks modernization and automation to solve a given problem. The ethical responsibility for this transformation is not only in the hands of those who implement these models. Those who decide to automate without considering the consequences of such a decision are also to be blamed. In short, machine learning models lack the critical thinking to check if the data utilized in its training is a good representation of any given reality. Ensuring this incredible tool is being used ethically, correctly, and for the right reasons is, and always will be, a human responsibility.
Last but not least, we must consider the possibility of intentional wrongdoing. Shocking, I know! But intentional bias does happen. That’s why understanding how machine learning utilizes statistics and data to make predictions and foresee the future is essential. But most importantly, it gives us the means to tackle both intended and unintended biases.
Ill-intended decision-makers can easily justify wrongdoing by hiding behind black-boxed models for prediction. On the surface, their decisions may look like the simple result of the model’s supposedly neutral and statistically correct predictions. Nonexpert stakeholders frequently overlook this handling due to their lack of knowledge on the subject, sanctioning the bias themselves and taking part in this pernicious plot of discrimination and inequality. Knowing how new technologies work, even at a conceptual level, can lead us to correct a bias instead of reinforcing and validating it.
The most commonly used machine learning algorithms must be trained with a database. Selecting specific data that proves one’s point is a malicious strategy frequently used by those who seek to manipulate results in their favour. By using a biased database in the model’s training and then utilizing the trained model on the actual database, the algorithm will gladly reproduce the bias of its training in all of its predictions. Requesting the training database and results, for example, can be crucial in identifying wrongdoers and bringing justice to the environment in question.
We should never accept the most modern technology as the best, nor should we idealize the results of new technology implementations. Knowledge is power, and the worst risk is the one we are unaware of. The difference between surfing the hype of new technology and falling off the plank is doing adequate research and having ethical responsibility. Always update your knowledge on how to safely swim before blaming the tide for drowning you in an ocean of machine failures and mistakes.
