84 questions to ask before training an AI model

A group of artificial intelligence experts and data scientists have published a voluntary code for training an AI model safely.

This checklist, published as an open letter from the World Ethical Date Foundation, consists of 84 questions which developers need to consider at the start of a project training an AI model.

Questions for developers include how they will prevent an AI product from incorporating bias, and how they would deal with a situation in which the result generated by AI results in law-breaking.

Other considerations in the voluntary framework include the data protection laws of various territories, whether it is clear to a user they are interacting with an AI, and whether human workers who input or tag data being used to train the AI were treated fairly.

The 84 questions are divided into three chapters: questions for individual developers, questions for a team to consider together, and questions for those testing the product.

84 questions to ask before training an AI model

Training – Data Selection and Ingestion

Me – The necessary questions to ask myself before starting this part of the process. Answers should be saved so you can look back at how your thought process evolves.
– What are the reasons for me selecting this training data, and how does that selection align with my intention for this model?
– How was this training data sourced, is there any protected or copyrighted material in the training data such as Personally Identifiable Information (PII), Payment Card Industry (PCI) data, and Protected Health Information (PHI), and have we considered GDPR, CCPA or any other systems for managing data sources?
– Do I have experience in using similar data for AI or Machine Learning models in the past or is this the first time I am using this data source?
– If I’ve used this data previously, were there any issues that resulted from the models that were trained on this data historically?
– If this is the first time I’ve used this data, what are my expectations for the impact this data will have on the models outputs?
– Have you used a Model Card to communicate risks and how will this document be updated with the collaboration team?
– What do I hope the data will do to this model, what is my intention of the outcome and how do I expect the training data will impact that performance?
– Are there any features that could act as instrumental variables (eg. Granular neighbourhood level data) for disenfranchised populations, and if so, do you have sufficient controls to ensure that you are not perpetuating biases that can be learned from the training data?
– Can I summarise what is in this training data in a way that captures the essence of the data in a way a non-data scientist can understand?
– Do I feel rushed or pressured to input data from questionable sources?
– Can I cite my source of the training data?
– What biases may be acting on my selection of this data?
– Am I considering biases I have that I don’t understand and am I sharing my logic with a larger group who can help me identify my bias being deployed when selecting data?
– How do I think this training data will benefit this model?

We – The questions we should ask ourselves as a working group before starting this part of the process. Answers should be saved so we can look back at how our thought process evolves.
– What is the group’s intent for training this model?
– Who collaborated in the process of building the training data strategy and selection?
– Is the team of people who are working on selecting the training data from a diverse set of backgrounds and experiences to help reduce the bias in the data selection?
– What are the likely biases inherent in this team that selected the training data?
– Does the whole team understand where the train data came from, and can they explain it back in their own words?
– Is a Model Card being used to communicate risks and did each contributor create their own document that can be shared with the larger team?
– Have we considered the EU AI Act or any other regulation being proposed or that is already in place?
– Is there any protected or copyrighted material in the training data such as Personally Identifiable Information (PII), Payment Card Industry (PCI) data, and Protected Health Information (PHI), and have we considered GDPR, CCPA or any other systems for managing data sources?
– What percentage of my training data am I saving for testing and how are we selecting it?

It – The questions we should ask ourselves of the algorithms or models before starting this part of the process. Answers should be saved so we can look back at how our thought process evolves.
– Do I have means to compensate should part of this data set be discovered to be illegal, unreliable, or unacceptable at some point in the future?
– What data is needed to train thoughtfully and with intention?
– Is the full data set of known origin, explainable, and beneficial for the model?
– What are the likely biases inherent in the data?
– Is there any personal identifiable information, protected data, or copyrighted material?
– Am I prepared to handle liability for the model if part of the data set causes any legal issues in the future?
– What are potential unintended uses/consequences, including subsequent level outcomes that the training data could teach the model?
– What are the likely biases that could be amplified through the training data being added to the model?

Building – Creation or Selection of Algorithms and Models

Me – The questions I should ask myself before starting this part of the process. Answers should be saved so you can look back at how your thought process evolves.
– What do I intend for this model to do, and why am I training it?
– If I am running reinforcement learning, how will this model optimise in the live environment and is it possible my selection of outcomes to test is biassed?
– If I am deploying transfer learning, what are the possible biases that the transferring process will uncover?
– If I am running ensemble models or systems that train each other is there a chance that new bias or bad data collection practices will enter the system?
– How do I think this model will perform and what are some examples of desired outputs I am hoping to see?
– What are my human biases that I possess that impacted my goals and reasoning?
– If I didn’t write the model from scratch, where did it come from and how was it initially trained?

We – The questions we should ask ourselves as a working group before starting this part of the process. Answers should be saved so we can look back at how our thought process evolves.
– Whom did I collaborate with in the process of training this model and building the strategy?
– What human biases are in this group, and have we considered if the working group is diverse enough to capture differing points of view?
– Who were the collaborators building the model and strategy?
– Who are the stakeholders and are all the stakeholders engaging in this step of the process?
– Have we considered the EU AI Act or any other regulation being proposed or that is already in place?
– Were these models trained on any protected or copyrighted material such as Personally Identifiable Information (PII), Payment Card Industry (PCI) data, and Protected Health Information (PHI), and have we considered GDPR, CCPA or any other systems for managing data sources?

It – The questions we should ask ourselves of the algorithms or models before starting this part of the process. Answers should be saved so we can look back at how our thought process evolves.
– Where did the model come from or was it developed from scratch?
– What is the intended use of the model once it is trained?
– If we didn’t create the model ourselves, do we understand the intent of the original creator of the model?
– What are potential unintended uses/consequences, including subsequent level outcomes?
– What are the likely biases that could be amplified through the model?
– Do we have examples of these models with similar data being deployed previously and what were the intended and unintended outcomes?
– What are the possible dangers of the model and do we have a plan for the worst case scenarios?
– What type of measure have I put into place as precautionary measures?
Who controls the model?
– How will continued compliance with laws and regulations be monitored and implemented?
– How will biases be discovered and resolved?
– How can the model be shut down and under what circumstances must that happen?
– Are there any self interested stakeholders or realities of funding that may stop that from happening?
– What are the contractual requirements that have been entered into that will dictate the management and usage of this model?

Testing – Managing Test Data and Tagging

Me – The questions I should ask myself before starting this part of the process. Answers should be saved so you can look back at how your thought process evolves.
– Is the data I’m using for testing sufficient for analysing how the model performs?
– Is user testing in the live environment being considered and how will the model be adapted if it turns out that the user behaviour outcome is unwanted.
– What do I think is the best approach for evaluating the outcome of this model before I speak with the larger group about our approach?
– Am I confident that the team and I are aware of the impact our work can have and have we thought about all the potentially bad outcomes that could result from our work that we should be testing for before we go live?
– Do I believe we have been thorough in our testing strategy and deployment to ensure we have unearthed any issues that could arise from the models outputs?
– Did the results of the training data and feedback from taggers match the original intent I had when creating the model. If not, what differed and how do I know it differed?

We – The questions we should ask ourselves as a working group before starting this part of the process. Answers should be saved so we can look back at how our thought process evolves.
– How do we evaluate the performance and outcomes of this model?
– Are we aware of where the testing data come from and is the testing data appropriate to be representative in relation to the vastness of the training data?
– If the data is tagged by people, who are the people, are they being humanely treated?
– What instruction did taggers receive before they tag the data that might impact their opinion?
– What questions are the taggers or working group asking of the data?
– Does the tagging strategy align with the training data and model creation strategy?
– Is there an appropriate amount of diversity on my data tagging team?
– What human biases might impact the tagging or testing process?
– Is user testing in the live environment data being collected and how will it feed back into an iterative process?
– Are the user group a diverse audience that is representative of the future total user group?
– Is it possible that bias is a programmatic result of the model?
– Whom did I collaborate with in the process of building the testing or tagging data strategy?
– Are product teams involved in the process of retrieving user data and sharing human logic with the larger group?
– Was the intention of the working group realised now that the model has run and been tested? If not, explain in detail with quantifiable data.

It – The questions we should ask ourselves of the algorithms or models before starting this part of the process. Answers should be saved so we can look back at how our thought process evolves.
– Did the results of the test or tagged data match our intent for the model outputs?
– If we are testing our models to determine accuracy, how much do the results we are seeing match up with the core intent of what the model will do?
– Will I now tune the model based on the outputs of the testing or tagged data? If so, start the process from Step 1.
– If deploying reinforcement learning, how can model outcomes amplify bias and influence new data from users?
– Have we considered the output of the model against the EU AI Act or any other regulation being proposed or that is already in place?
– Have we tested for protected or copyrighted material in the output of the model such as Personally Identifiable Information (PII), Payment Card Industry (PCI) data, and Protected Health Information (PHI), and have we considered GDPR, CCPA or any other systems for managing data sources?

Vince Lynch, a board director for the World Ethical Data Foundation and CEO of IV.AI, told the BBC: “Some of the main points are to start by focusing on intent: what do we want our models to be doing? What do we expect them to do before we start building them? And what’s the outcome of what we’re trying to achieve?

“As we’re building the models, what are the steps we are trying to achieve that outcome? What is the data, where does it come from and how is it structured?

“Then, working as a group, are we all aware of what’s going on with this model? Is it more than just me who’s thinking about it as a data scientist? How will people be testing the output? Will more people involved in testing than building it, giving us feedback on what we’re seeing?”

What the draft EU AI Act means for regulation – Information Age speaks to EU data protection, intellectual property and technology experts about the business implications of the EU AI Act

Lynch said that AI development is still in the “Wild West” stage but that those cracks in the foundations are becoming more apparent, as people are having conversations about intellectual property and how human rights are considered in relation to AI.

Using this checklist before embarking on training artificial intelligence could save companies a lot of money before an AI model veers off course.

For example, said Lynch, if an AI model has been trained using some data which is copyright protected, it could cost hundreds of millions of dollars to completely rebuild the model, without having asked the right questions first.

“It’s not an option to just strip it out – the entire model may have to be trained again … it is incredibly expensive to get it wrong.”

Tim Adler

Tim Adler is group editor of Small Business, Growth Business and Information Age. He is a former commissioning editor at the Daily Telegraph, who has written for the Financial Times, The Times and the... More by Tim Adler

84 questions to ask before training an AI model

AI experts and data scientists have published a checklist of questions developers should ask before embarking on training an AI model

84 questions to ask before training an AI model

Further reading

Tim Adler

84 questions to ask before training an AI model

AI experts and data scientists have published a checklist of questions developers should ask before embarking on training an AI model

84 questions to ask before training an AI model

Further reading

Tim Adler

Related Topics

Related Stories

3 ways AI is set to transform the energy sector

Biggest AI myths in customer experience

How insurtech is using artificial intelligence

Only 22% of IT staff fully understand capabilities of AI tools

Related Stories

3 ways AI is set to transform the energy sector

Biggest AI myths in customer experience

How insurtech is using artificial intelligence

Historical education bolstered by AI-powered digital human