84 questions to ask before training an AI model

AI experts and data scientists have published a checklist of questions developers should ask before embarking on training an AI model

A group of artificial intelligence experts and data scientists have published a voluntary code for training an AI model safely.

This checklist, published as an open letter from the World Ethical Date Foundation, consists of 84 questions which developers need to consider at the start of a project training an AI model.

Questions for developers include how they will prevent an AI product from incorporating bias, and how they would deal with a situation in which the result generated by AI results in law-breaking.

Other considerations in the voluntary framework include the data protection laws of various territories, whether it is clear to a user they are interacting with an AI, and whether human workers who input or tag data being used to train the AI were treated fairly.

The 84 questions are divided into three chapters: questions for individual developers, questions for a team to consider together, and questions for those testing the product.

84 questions to ask before training an AI model

Vince Lynch, a board director for the World Ethical Data Foundation and CEO of IV.AI, told the BBC: “Some of the main points are to start by focusing on intent: what do we want our models to be doing? What do we expect them to do before we start building them? And what’s the outcome of what we’re trying to achieve?

“As we’re building the models, what are the steps we are trying to achieve that outcome? What is the data, where does it come from and how is it structured?

“Then, working as a group, are we all aware of what’s going on with this model? Is it more than just me who’s thinking about it as a data scientist? How will people be testing the output? Will more people involved in testing than building it, giving us feedback on what we’re seeing?”

What the draft EU AI Act means for regulationInformation Age speaks to EU data protection, intellectual property and technology experts about the business implications of the EU AI Act

Lynch said that AI development is still in the “Wild West” stage but that those cracks in the foundations are becoming more apparent, as people are having conversations about intellectual property and how human rights are considered in relation to AI.

Using this checklist before embarking on training artificial intelligence could save companies a lot of money before an AI model veers off course.

For example, said Lynch, if an AI model has been trained using some data which is copyright protected, it could cost hundreds of millions of dollars to completely rebuild the model, without having asked the right questions first.

“It’s not an option to just strip it out – the entire model may have to be trained again … it is incredibly expensive to get it wrong.”

Further reading

How generative AI regulation is shaping up around the world – With generative AI developments heating up globally, we take a look at the regulation state of play for regions across the world

Avatar photo

Tim Adler

Tim Adler is group editor of Small Business, Growth Business and Information Age. He is a former commissioning editor at the Daily Telegraph, who has written for the Financial Times, The Times and the...