LLMs won’t replace NLUs. Here’s why

‍Understanding LLMs and NLUs

To better understand the relationship between LLMs and NLUs, we need to know their distinct capabilities.

1. Large language models (LLMs): LLMs are AI models trained on vast amounts of text data to generate human-like text based on given prompts. They excel at things like text completion, question-answering, translation, and summarization. However, they’re pretty limited when it comes to deeper understanding and interpretation of context—and that results in occasional errors or nonsensical output.
2. Natural language understanding (NLU) systems: NLU is a subdomain of NLP that focuses on enabling machines to comprehend and interpret human language. NLU systems are designed to analyze text and extract meaning, context, sentiment, and intent. They play a crucial role in applications like chatbots, voice assistants, and sentiment analysis.

Why LLMs won't replace NLUs

1. Context and meaning: Although LLMs can produce coherent and contextually relevant text, they often struggle to understand the deeper meaning behind it. NLU systems, on the other hand, are specifically designed to interpret context and semantics. This makes them more suitable for tasks requiring a precise understanding of language.
2. Intent recognition: LLMs aren’t designed to recognize the intent behind a user's query or statement. NLU systems excel in this area by identifying the purpose of a user's input and enabling more accurate and responsive interactions in applications like chatbots and voice assistants.
3. Robustness and efficiency: NLU systems are typically more focused and efficient, requiring fewer computational resources than LLMs. This makes them more suitable for real-time applications and resource-constrained environments.
4. Customizability: NLU systems can be customized for specific domains, industries, and applications, resulting in more accurate and relevant responses. In contrast, LLMs are trained on general-purpose datasets, making customization more challenging.
5. Ethical considerations: LLMs have a reputation for generating biassed or harmful content based on the data they’ve been trained on. NLU systems, with their more focused approach, can be designed to mitigate these risks by carefully selecting training data and implementing safety mechanisms.

LLMs and NLUs are complementary technologies

LLMs have revolutionized the field of NLP by generating human-like text, while NLUs continue to excel in understanding and interpreting language. By combining the strengths of both approaches, we can develop more advanced, efficient, and accurate AI applications that better understand and serve our needs.

Practically speaking, it’s likely that we’ll see NLU used to determine user intent, which hands off to the LLM to perform the conversational flow for that intention.

We’ll see continued improvements in both LLMs and NLU systems as AI research progresses. By appreciating their unique strengths and fostering collaboration between these technologies, we can unlock their full potential and create a future with more intelligent and helpful AI-powered tools.

And if you’re wondering, yes—GPT-4 helped me write this post.

Header image by Anni Roenkae.