How Taking a Mindful Approach to AI Cuts Down on Biased Data Sets

Biased Datasets and Ethical AI

AI-powered solutions need data sets to be effective. And the creation of those data sets is fraught with an implicit bias problem at a systematic level. All people suffer from biases (both conscious and unconscious). The biases can take any number of forms: geographic, linguistic, socio-economic, sexist, and racist. And those systematic biases are baked into data, which can result in AI products that perpetuate and magnify bias. Organizations need a mindful approach to mitigate against bias creeping into data sets.

Examples That Illustrate the Bias Problem

One notable example of this data set bias that garnered a lot of negative press at the time was a resume reading solution that favored male candidates over females. This is because the recruitment tool’s data sets had been developed using resumes from over the past decade when a majority of applicants had been male. The data was biased and the results reflected that bias. 

Another widely reported example: At the annual Google I/O developer conference, Google shared a preview of an AI-powered dermatology assist tool that helps people understand what’s going on with issues related to their skin, hair, and nails. The dermatology assistant underscores how AI is evolving to help with healthcare — but it also highlighted the potential for bias to creep into AI in the wake of criticism that the tool is not adequate for people of color.

When Google announced the tool, the company noted:

To make sure we’re building for everyone, our model accounts for factors like age, sex, race, and skin types — from pale skin that does not tan to brown skin that rarely burns.

Google, Using AI to help find answers to common skin conditions

But an article in Vice said Google failed to use an inclusive data set:

To accomplish the task, the researchers used a training dataset of 64,837 images of 12,399 patients located in two states. But of the thousands of skin conditions pictured, only 3.5 percent came from patients with Fitzpatrick skin types V and VI—those representing brown skin and dark brown or black skin, respectively. 90 percent of the database was composed of people with fair skin, darker white skin, or light brown skin, according to the study. As a result of the biased sampling, dermatologists say the app could end up over- or under-diagnosing people who aren’t white.

Vice, Google’s New Dermatology App Wasn’t Designed for People With Darker Skin

Google responded by saying it would refine the tool before releasing it formally:

Our AI-powered dermatology assist tool is the culmination of more than three years of research. Since our work was featured in Nature Medicine, we’ve continued to develop and refine our technology with the incorporation of additional datasets that include data donated by thousands of people, and millions of more curated skin concern images.

Google, Using AI to help find answers to common skin conditions

As much as we might hope AI and machine learning programs could correct for these biases, the reality remains: they are only as smart as their data sets are clean. In an update to the old programming adage garbage in/garbage out, AI solutions are only as strong as the quality of their data sets from the get-go. Without a correction from programmers, these data sets don’t have the background experience to fix themselves – as they simply have no other frame of reference.

Building data sets responsibly is at the core of all ethical artificial intelligence. And people are at the core of the solution. 

Mindful AI is Ethical AI

Bias doesn’t happen in a vacuum. Unethical or biased data sets come from taking the wrong approach during the development stage. The way to combat bias errors is to adopt a responsible, human-centered, approach that many in the industry are calling Mindful AI. Mindful AI has three critical components:

1. Mindful AI is Human-Centered

From the inception of the AI project, in the planning stages, the needs of people must be at the center of every decision. And that means all people – not just a subset. That’s why developers need to rely on a diverse team of globally-based people to train AI applications to be inclusive and bias-free.

Crowdsourcing the data sets from a global, diverse team ensures biases are identified and filtered out early. Those of varying ethnicities, age groups, genders, education levels, socio-economic backgrounds, and locations can more readily spot data sets that favor one set of values over another, thus weeding out unintended bias.

Take a look at voice applications. When applying a mindful AI approach, and leveraging the power of a global talent pool, developers can account for linguistic elements such as different dialects and accents in the data sets.

Establishing a human-centered design framework from the beginning is critical. It goes a long way toward ensuring that the data generated, curated, and labeled meets the expectation of the end users. But it’s also important to keep humans in the loop throughout the entire product development lifecycle. 

Humans in the loop can also help machines create a better AI experience for each specific audience. At Pactera EDGE, our AI data project teams, located globally, understand how different cultures and contexts can impact the collection and curation of reliable AI training data. They have the necessary tools they need to flag problems, monitor them, and fix them before an AI-based solution goes live.

Human-in-the-loop AI is a project “safety net” that combines the strengths of people – and their diverse backgrounds with the fast computing power of machines. This human and AI collaboration needs to be established from the beginning of the programs so that biased data doesn’t form a foundation in the project. 

2. Mindful AI Is Responsible

Being responsible is to ensure that AI systems of free of biases and that they are grounded in ethics. It is about being mindful of how, why, and where data is created, how it is synthesized by AI systems, and how it is used in making a decision, decisions that can have ethical implications. One way for a business to do so is to work with under-represented communities to be more inclusive and less biased. In the field of data annotations, new research is highlighting how a multi-annotator multi-task model that treats each annotator’s labels as separate subtask can help mitigate potential issues inherent in typical ground truth methods where annotator disagreements may be due to under-representations and can get ignored in the aggregation of annotations to a single ground truth. 

3. Trustworthy

Trustworthiness comes from a business being transparent and explainable in how the AI model is trained, how it works, and why they recommend the outcomes. A business needs expertise with AI localization to make it possible for its clients to make their AI applications more inclusive and personalized, respecting critical nuances in local language and user experiences that can make or break the credibility of an AI solution from one country to the next. For example, a business should design its applications for personalized and localized contexts, including languages, dialects, and accents in voice-based applications. That way, an app brings the same level of voice experience sophistication to every language, from English to under-represented languages.

Fairness and Diversity

Ultimately, mindful AI ensures solutions are built upon fair and diverse data sets where the consequences and impact of particular outcomes are monitored and evaluated before the solution goes to market. By being mindful and including humans in every part of the solution’s development, we help ensure AI models stay clean, minimally biased, and as ethical as possible.