Artificial Intelligence (also called “AI”, if needed read our 5 minute read post : What is Artificial Intelligence?) is a much fantasized area of research. It’s capabilities today have often been exaggerated so much so that people or companies have compared them to those of the human brain. If we are indeed far enough from this stage of “general AI” it is undeniable that recent key achievements in the field of Machine Learning have made real-world business applications possible.
With this enthusiasm and flurry of experiments all over the world a crucial question needs answering : what are the main ethical* risks posed by AI ? This paper will attempt to cover the opportunities and main pitfalls machine learning systems can fall in and the solutions that we recommend and implement today.
* By ethical we mean that they comply by moral principles and values that are shared by members of a society. A set of values and principles have been agreed upon during the Human Rights declaration by the United Nations in Paris in 1948 but are challenged by numerous states and entities today. We state our ethics in the end of this post.
AI carries incredible opportunities for actors big and small around the world. Here are a few sectors for which AI can make a huge difference.
I.1 In the field of agriculture
AI can be used in the field of agriculture to identify with great precision where farmers should water their crops and where they must intervene in order to save them from an environmental threat. This is achieved through deep learning models trained on satellite images as well images taken by cameras on farm tractors.
I.2 In the field of transportation
Car accidents are estimated to cause 1.25 million deaths in the world each year. They are also hugely costly, causing hundreds of billions of dollars in damage each year in the U.S. What’s more the National Highway Traffic Safety Administration (NHTSA) estimates that 94% of auto accidents are caused by human error.
Reducing human input has the potential to result in fewer deaths and injuries and reduced economic damage if automated driving systems are up to the task of taking over. Autonomous vehicle technology relies heavily on AI technology.
I.3 In the field of health
AI has shown itself incredibly valuable in assisting doctors in making the right decisions for their patients. Recently an artificial intelligence has been developed that is better at spotting breast cancer in mammograms than expert radiologists.
The AI outperformed the specialists by detecting cancers that the radiologists missed in the images, while ignoring features they falsely flagged as possible tumours.
I.4 In the field of homeland security
Police departments and secret services can benefit from AI by gaining precious time in identifying know criminal faces or vehicle licence plates in CCTV footage and photos published online. AI can also improve weapon detection at airports scanning luggage by automatically detecting suspicious luggage. This can prove greatly helpful in preventing an attack on the population.
I.5 In the field of financial services
The banking sector has undergone a large scale transformation since the 2008 banking crisis with an emphasis put on compliance. AI can help compliance officers with the many challenges they have : automatically extract and synthesize huge amounts of information to verify that transactions and new accounts are compliant. Process internal recordings to ensure no breach of protocol has been made. In the ultimate aim to prevent a new risk for the bank and the system as a whole.
As often with new and powerful technologies Artificial Intelligence carries risks on different levels. After defining the notion of bias we will describe theses risks.
Bias is commonly defined as an inclination or prejudice for or against some person, some group or something.
- It may be conscious : for example choosing a banana to eat by it’s yellow color : we may have a conscious bias to choose yellow bananas because we have prior experience or scientific knowledge of bananas being ripe when they are yellow
- Or it may be unconscious : associating negative feelings with a community even though you are openly inclusive (as the famous Harvard implicit test on race has demonstrated)
“Two quite opposite qualities equally bias our minds – habits and novelty.” – Jean de la Bruyère (Les Caractères ou les Moeurs de ce siècle, 1688)
Bias is a very widespread reality, as Jean de la Bruyère notes bias can be forged by habits, repeating the same choices and being unwilling to change, whereas something new can trigger a positive or a negative bias just by being new. The field of cognitive biases is an extremely active field that has yet a lot to reveal on inherent human biases.
Statistics and the field of machine learning have their own mathematical definition of bias. Bias as well as Variance are the main sources of errors between a machine learning model’s predictions and reality :
- Bias error is due to over simplified models that fail to capture the complexity of reality
- Variance error is due to over complexified models fit perfectly the training data but fail to generalize well
II.2 Bias in > Bias out
Education, employment, health, access to credit : even today these important components of our society are prone to biased behaviour and decisions.
In the law enforcement domain multiple studies have shown existing biases :
- A 2011 study has shown that British judges were more merciful after lunch than in other moments of the day.
- The Open Justice initiative in California has shown LAPD’s negative bias towards black people : with 28% of persons stopped by the police being black when they make up only 9% of the local population
If past data is treated as ground truth then AI models trained on the data will reproduce existing social biases. This has already been verified on multiple occasions :
- In the field of human resources Amazon has built and since removed an AI model that was meant to suggest the top 5 best candidates among a pool of more than 100 resumes to managers. Among other problems, women were under ranked by the model. This was due to the fact that the existing workforce at management level is today predominately white and male
- The British Government has issued in September 2019 a report informing police of the risk of bias in current crime prevention systems
Key Issue #1:
Whenever an AI is developed with human related data biases are bound to be present in the training data.
Acknowledging existing biases is the first step that will set a zero-bias target in the choice or development of an AI system. For the AI system to be fair a bias analysis needs to be carried out on past data so that gender or ethnicity biases can be balanced prior to AI training. Because our societies’ bias’ are undergoing a long term change (for instance deleting the gender pay gap) it is also crucial for AI systems to have a continuous bias mitigation feature.
II.3 Intrusive AI
Today’s AI systems are based on calculations, a sum of binary operations that are made by our computers : they lack emotions and are unable to be sincerely empathetic and understand our emotions. They can only simulate and reproduce trends they have learned (with more or less randomness).
These limitations make systems claiming to recognize emotions, mental health, personality or other interior states inherently flawed. A call to ban this technology have recently emerged.
However, automatically identifying what content triggers a given reaction (e.g a hate emoji on Facebook) on a given segment of the population (e.g suburban single mothers in Ohio) and using this information to create similar contents to trigger these emotions again is already a troubling capability.
The Cambridge Analytica scandal has revealed how targeted advertisements have attempted (and arguably succeeded) to influence voters on a massive scale. In the case of the american 2016 presidential campaign the company aggregated over 80 million Facebook profiles and processed a huge amount of personal information to detect patterns among voters and design tailored messages to provoke desired emotions and reactions.
Key Issue #2:
The accumulation and aggregation of highly personal data (political view, family status, sexual orientation, …)
Regulators should strengthen data protection and regulation. Businesses should be wary of the data they process and use the minimum amount of data to carry out their processes.
On the 25th of may 2018 the European Union finally gave a regulatory framework to protect citizen’s data (GDPR) and recently the state of California has also adopted a Consumer Privacy Act (CCPA) .
These frameworks are essential to encourage best practices in businesses and help avoid a new Cambridge Analytica from happening.
II.4 AI for good or bad ?
The third ethical issue in regards to AI is one linked to the very application in which the AI is supposed to make an improvement.
Elon Musk along other prominent figures in AI have called for a ban on autonomous killing robots. The initiative is much needed however it is important to note that autonomous killer robots are unfortunately only one step further in the development of killing robots which already exist and are used today.
Killer robots illustrate the augmentation with AI of an unethical product or means (killing a fellow man).
The same can be said of mass surveillance or genetic selection and modification of animal living cells. If Artificial Intelligence can assist in performing these actions the resulting models are unethical by design as their task done by a human is already unethical.
To be perfectly clear : this argument does not mean it is Ok to develop killer robots, quite the contrary. It shows that AI models can never be ethical if the underlying goal in itself is unethical.
Key Issue #3 :
Augmenting existing unethical means (e.g genetic selection) or products (e.g lethal weapons) with AI.
The development of AI at scale is the perfect occasion to reconsider a part of our activities, especially those that are questionable in regards to our set of values and moral principles. Investment and development of AI should go into activities which create sustainable value for the human community and should not bring harm or destruction in any way.
III. Our engagement
As we have seen in the first section AI can prove hugely valuable to society and to workers, lawmakers, bankers, teachers and many more professionals. However it is important to remain strongly vigilant to ensure that AI is ethical by design.
As an AI company we commit to carrying out these key precautionary measures when introducing AI into existing processes for our clients :
- Perform a bias analysis : we make descriptive statistics on demographics to see how past decisions have varied according to variables such as gender or ethnicity. If the study proves an existing bias then unbias the data on which the AI will learn past trends. We recommend our clients to also be proactive and launch an internal communication campaign to underline past biases and promote better human decisions
- We keep only the strict amount of data relevant to the given process : we carefully review with the business what are the mandatory data. If we need to process unstructured data such as images, videos or texts we anonymize it as much as possible. This way we guarantee our treatment to be GDPR compliant and we also avoid our models adopting unwanted biases if a supposedly neutral variable (for example gender) takes in fact a positive or negative weight
- We advise our clients from the risks of going fully automatic : Machine learning (which fuels the vast majority of AI systems), is based on probabilities. You can have a confidence score on one prediction but it will only be a statistic based on past observations. One can think of it as a weather forecast, it can be extremely accurate, and is extremely useful, but remains a forecast all the same, with a (small) probability to fail. When making decisions as critical as to lend or not money or to give access to property, education or health a human validation is mandatory
- Before working on a project and installing our solution we ask ourselves and the client : will the system do good ? Are there any chances a highly negative outcome can occur ? We carefully weigh the pros and cons and make sure there are safeguards that will easily identify and prevent the system from getting out of control
We at Datakeen are focused on producing machine learning technology that is free of existing biases or is not detrimental to any group of people or community.
We work hand-in-hand with our clients in ensuring their input data is unbiased through preliminary analysis and that their use of AI does not breach any ethical boundaries. This is part of our DNA and makes us proud as an AI company.
I hope you enjoyed the read and invite you to read our future publications.
About the author
Passionate and practitioner of Machine Learning and it’s deployment in real-life business cases, Gaël directs AI product development at Datakeen and is committed to making AI as ethical as it is efficient.
Here are a few interesting reads related to the issue at stake :
- List of cognitive biases
- The Bias and Variance tradeoff
- AI Now Institute reports
- The Guardian view on automating poverty
- Top AI researchers call on Amazon to fix racial bias in Rekognition
- A Moral and Legal imperative to ban Killer Robots
- The California Consumer Privacy Act (CCPA)
- The General Data Protection Regulation (GDPR) for the EU