Natural language processing (NLP) makes it possible for computers to read text, hear speech, interpret it, measure sentiment and determine which parts are important. Today’s machines can analyse more language-based data than humans, without fatigue and in a consistent, unbiased way.
“NLP is important because it helps to resolve ambiguity in language and adds useful numeric structure to the data for many downstream applications, such as speech recognition or text analytics,” Kayne Putman, analytics consultant at SAS UK & Ireland says. “That means it can be applied in a wide variety of business use cases such as fraud and risk analysis or extracting insights on customer behaviour. Fundamentally, it is essential to have good data analytics in place and to know what you want to achieve before introducing NLP into the environment.”
Development of natural language processing
The biggest change in natural language processing over the last ten years has been the move away from more traditional pattern recognition in words and move towards machine and deep learning approaches. There have been recent advances in terms of the technique of ‘word embedding’ which allows some degree of semantic meaning to be ascribed to individual words, or groups of words.
“The result is a high-dimensionality numeric vector that allows further model training and analysis to occur,” Jos Martin, senior engineering manager at MathWorks, says. “Once some of the semantic meaning has been transferred into the numerical space, many other recently developed deep learning techniques become available to NLP system designers.”
For example, many systems have taken advantage of long short-term memory (LSTM) types of recurrent neural networks, helpful in learning more about the relationships between words in sentences, paragraphs and other linguistic blocks. These types of network allow the designer to predict what the next word in a sequence might be or ascribe probabilities to the next several words.
Six surprising ways businesses are impacted by RPA, OCR and NLP
More than just a chatbot
In traditional natural language processing, words were just words – somewhat meaningless phrases that lack the meaning established by wider context. This is how we would typically categorise a chatbot, which is limited by the information fed to it in real-time.
“For advanced natural language processing , we can improve contextual understanding by representing words as vectors of numbers,” Johan Toll, executive director transformations at IPsoft says. “Instead of just understanding words as words, this enables the machine to understand word similarities and phrase similarities in very flexible ways.”
For example, understanding that the word ‘contract’ has a very different meaning in a legal context to than in a gangster movie. Unlike with a chatbot that understands just a single-stream of information and language, this enables a multi-faceted conversation that is tailored as per the required context, in addition to the type of language used.
According to Toll, advanced NLP is a facilitator. “In fact, most AI developments and digital innovations include some form of intelligent process automation (IPA), with McKinsey estimating that 50 to 70 per cent of tasks within companies are automated,” he adds. “It is this automation, facilitated by NLP, that enables the development of new AI applications, allowing organisations to accurately source, organise and identify vast amounts of valuable information. “
NLP to break down human communication: How AI platforms are using natural language processing
Augmenting customer experience
With the growth of eCommerce and the expanding global marketplace, more consumers are shopping online than ever before. To meet this increasing demand, organisations are adopting a combination of next generation technologies including NLP to augment their customer experience, improve brand reputation and boost sales. Research conducted at Aspect found that 92 per cent of respondents recognise the value of natural language processing in modern customer service.
“A key requirement for any forward-thinking organisation is to offer a versatile omnichannel experience where customers have the option of using automated self-service capabilities including virtual assistants and desktop or mobile chatbots, alongside traditional means of communication,” Stephen Ball, senior VP Europe & Africa at Aspect explains. “The true rewards of automated self-service can only be reaped if organisations implement these new technologies properly, and gaining a comprehensive grasp of natural language processing (NLP) technology is key to this.
“It is extremely important to realise that successful natural language processing integration is a process that takes time and effort and requires investing in AI and associated technologies that can be easily adapted to suit the needs of the company, and most importantly, are advanced enough to meet the complex and shifting demands of the modern customer. To make this happen, it’s crucial that businesses offer NLP and technology training to upskill staff as well as work with external partners to gain this relevant AI experience in the short and medium term.”
Gartner: debunking five artificial intelligence misconceptions
Adding emotion to natural language processing
The ability to understand whether someone is frustrated or angry from the tonal inclination, the words they choose and from pauses between sentences was until recently considered a uniquely human skill. A blossoming area of research is Emotion Analysis.
Emotion Analysis is a form of natural language processing that looks to determine the emotions of the writer from text, the emotions could be fear, anger, disgust, frustration, agitation or even sadness. “One such application of this fascinating technology will be in call centres – a helpline operator could adapt their advice and language to better meet a customer’s need, without the caller ever having to explicitly state their frustration level,” Sally Epstein, machine learning engineer, Cambridge Consultants comments.
A broader area of innovation is Sentiment Analysis, this is used to determine whether a portion of freeform text is positive, neutral or negative. “Using Sentiment Analysis it is possible to efficiently search huge quantities of text documents, social media posts, or product reviews to extract meaningful trends,” Epstein adds. “This insight is already being used by businesses to assess consumer satisfaction for their brand.”
Most importantly these techniques can be quickly scaled to draw insight from different regional dialects and languages.
What’s next for natural language processing ?
As for the future, better voice control and speech recognition will be a big area for natural language processing. Whilst speech recognition has got significantly better over the last several years, there is still some way to go before it is good enough to be used more broadly. Part of the advances in this area will come with better models.
Another big area will be the successful application of transfer learning to natural language processing. It is only within the last year that it has become possible to have deep learning pre-trained models for transfer learning in natural language processing , which is significant because transfer learning has been around for longer for computer vision. This is a huge breakthrough because it allows the user to benefit from the same pre-trained model, albeit with some tweaks, in all sorts of text analytics tasks – from sentiment analysis to question-answering.
Article by Mark Venables.
