What is the technology behind natural language processing?
Simply put, NLP, or natural language processing, is the technology that allows for real-world data to be understood and processed by computers.
It starts in the 1950s, with Alan Turing’s paper “Computer Machinery and Intelligence” , where he asked: Can machines think? What he meant is: Can machines think like humans, therefore can machines understand facts and data from human reality?
NLP is a form of artificial intelligence which uses several disciplines, like computational linguistics, to try to make structured sense of of unstructured human communication. In a sense, it converts human words (speech and text) into binary code.
Seventy years ago, punch cards were the only way for humans to communicate with a computer. Today, computers can ‘understand’ many human languages, execute spoken or written orders, infer insights from them and apply them to the future. For example, if you had viewed a red jumper online once, the computer would understand that you like red jumpers, and would show you all the items it knows of corresponding to “red” + ”jumper”.
NLP works by analysing infinitesimal components of human language – what would be the ‘atoms of human communication’. These might be words or phrases or descriptions from your product catalogue. These “atoms” it then compares to its own database of words and recordings, which it categorises and extracts meaning from them using semantic analysis – in effect, understanding them.
This means it can make connections humans may not. Today, with massive processing power available cheaply, the inferences it can make for individuals’ preferences is astonishing.
Why is it coming to the market now?
NLP has been in development for many, many years. The wider market is only taking notice now, however, because (1) the real-world applications of it and other AI technology are becoming better understood by business, and (2) we finally have the broad computational power to actually do something with it. It also doesn’t hurt that companies like ours have been able to use its vertical industry knowledge, for example, to go beyond the generic models and create experience -focused algorithms that drive real results, like increased conversions or revenue.
Up until recently, very few organisations outside of academia or the government had the computing power to actually run the deep learning algorithms needed to understand and draw correlations from text. With recent advances, not only can we quickly consume and draw meaning from vast amounts of textual data, but we now also have the real time technology (with the advent of technologies like Flink and Kafka) to process it along with user behaviour, extract the requisite insights, and respond in real time with the appropriate action or decision.
NLP to break down human communication: How AI platforms are using natural language processing
As Gartner notes, the advent of text data is sparking wide interest in a range of potential advanced AI applications. The use of NLP in the field of customer experience, for example, to extract sentiment from text messages and chat, has paved the way for more broader industry adoption and practical applications.
In the field of experience personalisation, we’re just starting to explore the possibilities and are excited by the results we’ve seen with use of NLP, for example, to extract insights from product catalogue descriptions, reviews, attributes and other text and gain a deeper understanding of our customers’ products and how they relate to each other – hypo-allergenic, 100% cotton, etc.
It’s only a matter of time before companies across industry will be using deep learning technologies like NLP to extract hidden insights and combining it with their specific business experience and expertise to innovate and differentiate themselves.
Should organisations be applying NLP? Can you provide some practical enterprise case studies/examples?
One sector we have clearly seen how it works is retail, where many of our clients belong. Fast fashion and retailers that have large product turnover or introduce whole new catalogs seasonally can especially benefit from the use of NLP.
Personalisation has traditionally worked best when you have tons of behavioural data for each product. But, what do you do if a large block of your catalogue is new or niche and just doesn’t have any data to go along with it? Until recently, the best one could do was offer up “top-sellers” recommendation as a complement.
Natural language processing gives a voice to digital processes
With NLP, we can actually take the textual data — such as the product description, style and fit notes, ratings and reviews, etc. — extract insights about that product and others related to it, and then use that deeper understanding to immediately provide recommendations and offers complementary to the new or long-tail product.
This is a game changer for retailers/brands that regularly introduce or quickly cycle through new styles or seasonal catalogs, making it possible to personalise and expose shoppers to relevant products through cross-sell offers from the start.
What next for NLP?
As brands and retailers continue to adopt NLP, use cases will continue to evolve. What we see next is expanding the application of NLP to include journey predictions, speech based extensions, faceting for search, and more. NLP with deep learning is the future of e-commerce shopping.
Answers provided by Raj Badarinath, VP Ecosystems, RichRelevance
