The skills needed to succeed as an AI prompt engineer

Alan Jacobson explores the skills required for future success as an AI prompt engineer — also referred to as an 'AI whisperer'.

The use cases for AI and, more specifically, generative AI have captured the collective imagination of business leaders and decision-makers across every industry, leading to skills being shifted towards new job roles such as that of the prompt engineer.

With the explosion of data creating a fertile ground for AI to deliver data intelligence at scale, enterprises are racing to infuse AI technology into strategies, products, and operations to expose new opportunities for driving growth and streamlining processes for greater efficiencies. There’s also a massive opportunity for generative AI to profoundly impact the development of data-driven cultures that will forever change how data workers find insights.

From text-mining to PDF data extraction and natural language processing (NLP), generative AI and large language models (LLMs) are lowering barriers to entry, empowering everyone to ask analytical questions while engaging the world’s imagination in how we understand and use data. However, even with the dawn of more accessible generative AI helping bridge the technology and skills gap, its application is only as good as an organisation’s ability to use it.


80% of UK tech start-ups investing in skills to mitigate gapsWith almost half of small businesses surveyed by the Department for Education citing a lack of skilled staff as a concern, investment in this challenge proves widespread among UK tech start-ups.


The baseline for generative AI success

The rapid ascent of generative AI has made AI technology a global sensation. Recent Alteryx research highlights this trend for AI-driven insights; 89 per cent of companies currently using generative AI reported seeing either substantial (34 per cent) or modest (55 per cent) benefits of the technology. As its meteoric rise continues, its true potential will only be realised by access to high-quality data and a mature analytics culture capable of engaging with the data needed to educate it. To accelerate this journey, it is essential for everyone to understand how to uncover valuable insights from these tools. They must be able to ask the right questions, implement the right data techniques, and yield helpful outcomes. So, what skills and roles will lead the pack and ensure companies move beyond the hype and into widespread production across the enterprise, by ensuring everyone can unlock the power of AI?

One of the biggest myths regarding data analytics and AI is that you need to have a solid background in coding and databases. For a long time, working with data required technical expertise in various programming languages, like SQL, Python, SPSS, or SAS. But it’s important to remember that the path into the world of AI does not always have to be via a degree in data science, computer science, special training or advanced coding skills. Similar to the shifts in finance and engineering from the advent of computers and graphing calculators, gone are the days when you needed a specialist data science degree to start out with AI.

The skills required to develop the AI-ready workforce go beyond the scope of pure technical ability. Generative AI introduces new, intuitive and compelling ways for the business user — the accountant, the supply chain analyst, the merchandising analyst — to solve critical challenges with data by putting the power of better decision-making at everyone’s fingertips.

From critical thinkers to creative problem solvers and active listeners, every organisation already has a large pool of talent with the right combination of essential knowledge and interest in the technology and soft skills suitable for generative AI to be unleashed to its full potential. Empowering this talent through access to data and self-service, low-code/no-code analytics removes the complexity of data science. It makes AI accessible to all — giving everybody the ability to start using AI from day one by empowering non-technical users to build and automate processes without needing to write code.


How to become a software engineerBritain has around two million technology jobs left unfilled, yet even a junior developer can earn £39,000 a year. We explain how a novice can become a software engineer.


Developing the AI talent of the future

Despite its fast-moving pace from news headlines to business use cases, organisations still recognise the importance of keeping humans in the loop to train, oversee the implementation and ensure generative AI success.

There is an incredible amount of work involved in harnessing the power of generative AI to make its outputs reliable, secure, and trustworthy. Without the context of the business problem, the knowledge of the data used to educate the AI or the domain knowledge to interpret the results, it is difficult to help identify areas for further development, while ensuring that the model delivers consistent results that deliver business value.

To guarantee the answers from generative AI are correct, you need to ensure models have access to high-quality datasets created by governed analytics processes and developed by experts in your organisation who understand the shape of data and have been upskilled to become empowered with generative AI capabilities. The prompt engineer, or AI whisperer, can put skills towards easily building, training, and deploying AI models across the business — from sales to finance and marketing — and monitor and interpret the results to help further shape their development.

Correctly worded prompts are critical to coaxing the desired results from generative AI models, so it’s no surprise that prompt writing has become an art form. Currently, one of the hottest jobs in tech, prompt engineers with a strong background in the domain they are working in who understand AI technology are also critical to ensuring the quality of AI-generated output. Delivering the decision intelligence required to provide real-time split-second business insights from AI requires prompt engineers who not only understand the underlying business challenges but can translate them into easy-to-use and easy-to-understand prompt text for the generative AI to solve using proper tone and accurate information.

Businesses are at a pivotal point in the journey to harnessing generative AI for business value, and decision-makers are increasingly engaged with implementing AI to deliver insights. The tipping point to success will be data-ready humans with the business context to the questions, and the skills, to train the generative AI to solve their business problems. However, AI’s potential for inaccurate results based on developers with low domain knowledge, or domain experts who have not been properly trained to leverage AI, can sink any company’s strategy to harness these new technologies. The successful organisations will be those that have developed their domain experts with the key critical thinking, domain knowledge and appropriate analysis skills to become the AI whisperers of the future.

Alan Jacobson is chief data and analytics officer at Alteryx.

Related:

Tech leaders profile: seeing AI as an opportunityJohn, Mike and Kevin from N-able’s leadership team explain why you should be leaning into AI and how to embrace the opportunity mindset.

Related Topics

AI Skills
Generative AI