How AutoML is helping to bridge the AI skills gap

In 2017, over half of senior artificial intelligence (AI) professionals stated that a lack of qualified personnel in their field is the single biggest barrier to AI implementation across businesses. As more and more companies choose to explore AI, deep learning and machine learning (ML), this knowledge gap begins to cause some serious problems. The solution may be in de-humanising ML functions by using automatic (also known as augmented or assisted) ML techniques.

The AI skills gap

AI experts are costly, with a reported average annual salary of $314,000. But before you can even worry about affording an AI expert, you have to find one. A recent study by Element AI concludes that there are only 22,000 qualified computer scientists, worldwide, who are capable of building AI systems. Of these, only 3,000 are actually seeking work, and there are an estimated 10,000 available positions currently for AI experts in the US alone.

Here’s a solution to the AI talent shortage: Recruit philosophy students

A partial solution to the problem of lack of artificial intelligence talent might lie with recruiting philosophy students. Read here

Too few scientists have the knowledge or experience needed for work in the industry, but as the AI industry evolves so fast, it is hard for academic institutions to keep pace with the changing needs of corporations and to provide graduates with the needed skills.

As an increasing number of companies, sectors, and markets branch into AI-related technologies, the gap between required and available personnel widens, and the need for another solution becomes more urgent. Small, unqualified or overwhelmed teams of developers are more likely to make mistakes, to try and apply existing models inappropriately to new data, or to miss problems with the data. All of these issues can cause significant damage to the systems being developed.

What is AutoML?

There are several different names for automated ML, including AutoML, Augmented ML and Assisted ML. All describe an evolving machine learning tool that can help developers and businesses address complex scenarios without the use of full AI systems. AutoML revolves around two fundamental concepts: data acquisition and prediction.

Data acquisition is necessary for any machine learning or AI system, and the quality and quantity of data can determine the overall reliability, efficacy, and usability of any system, regardless of its purpose.

Taking the next step with AI adoption — overcoming the problem of data

In an exclusive interview with Information Age, Ben Lorica – chief data scientist at O’Reilly Media – discusses the resurgence of AI, AI adoption in the UK and how companies can overcome the limitations of various factors holding back the technology. Read here

AutoML assumes that any person or company already has a large, reliable dataset. Users are able to input their dataset, identify labels, choose sections of code or applicable methodologies, click a button, and generate a trained, optimised model that is ready to predict. Thus, all of the steps between data acquisition and prediction and handled by AutoML.

This is a significant change from the traditional ML workflow and enables analysts to stay focused on the business problem rather than getting lost in the process. Many platforms are compatible with Android and iOS so that models can be smoothly and quickly integrated into mobile applications.

How does AutoML fill the skills gap?

AutoML allows smaller, overwhelmed, or understaffed companies achieve the level of analytical sophistication they need without having to find and hire dozens of deep learning experts. Aside from the distinct advantages that this has on time and the bottom line, the use of AutoML also reduces room for error.

AutoML products guide users through the process of creating and using new algorithms in an easy-to-use and comprehensive setting, minimizing error or misapplication of code. They also allow a greater range of personnel to use ML, as the barrier to entry for AutoML products is lower than for AI. With user-driven interfaces, even novice developers can create useful technologies quickly. And this saves time and increases productivity, as less energy is spent on building a lot of institutional knowledge.

AutoML products

Many companies have already started developing and packaging AutoML tools. Microsoft’s Azure ML is a cloud-based service that allows users to upload their data and use a simple drag and drop interface to design code and thereby create neural networks for analyzing their dataset. No data science or coding knowledge is required and the program works from anywhere, on any device.

Google has a similar service called Cloud AutoML, which helps developers build and train deep-learning models. Cloud AutoML has some off-the-shelf models available. Google’s product offers a graphical interface that allows users to train image detection and object recognition models, as well as access to natural language processing.

A guide to artificial intelligence in enterprise: Is it right for your business?

While true artificial intelligence is some way off, businesses are taking advantage of intelligent automation, like machine learning, to improve business operations, drive innovation and improve the customer experience. Read here

Amazon’s SageMaker focuses on ease of use while delivering accelerated development of learning models and deployment to the AWS cloud platform. It also includes off-the-shelf algorithms, as well as an integrated Jupyter notebook.

These and other AutoML products allow companies to avoid large teams of developers, dozens of data experts, or the necessity of hiring seasoned AI specialists. The simplicity of these tools also allows smaller and less skilled teams to develop and implement the technology they need. Given the persisting shortage of AI experts and the ever-increasing pace of AI development and implementation, AutoML is a smart solution with a lot of potential for growth.

Dr Ken Sanford, the US lead Analytics Architect for Dataiku
Dr Ken Sanford, the US lead Analytics Architect for Dataiku

Editor's Choice

Editor's Choice consists of the best articles written by third parties and selected by our editors. You can contact us at timothy.adler at