Large language models (LLMs), despite their ability to generate human-like content, have one serious flaw: they tend to hallucinate answers. This is because an LLM is a probabilistic model, which means that it generates the most probable answer to the user’s prompt, relying on its training data. The most probable answer, however, is not necessarily the correct one. In other words, we can say that LLMs do not store facts but only probabilities. This feature makes the wider business adoption of generative AI models quite a challenge. No company will risk building either an internal or customer-facing solution that makes mistakes that can potentially lead to serious business problems.
Fortunately, there are strategies that can help us curb LLM hallucinations and thus harness the power of generative AI in business. These strategies are retrieval augmented generation (RAG), fine-tuning, and prompt engineering. In this blog post, I will walk you through all of them, explaining their pros and cons, and their potential business use cases.
What is retrieval augmented generation?
A retrieval augmented generation mechanism enhances large language models (LLMs) by integrating external knowledge dynamically during the text-generation process. This method relies on two main components:
A retrieval system that utilizes a neural network to encode both the query and relevant documents into a dense vector space. When a query is received, the retrieval system identifies and retrieves the most relevant documents from a substantial database, such as a company’s knowledge base, treating these documents as a specialized corpus. The retrieval is driven by the similarity between the query’s vectors and those of the documents.
A sequence-to-sequence model: after retrieval, the documents are concatenated with the query to form an extended input for the sequence-to-sequence model. This model, typically based on transformer architecture, then produces the final text. It leverages the context from the retrieved documents to enhance the accuracy, relevance, and depth of the generated text.
By integrating RAG with LLMs, these models extend beyond their pretrained knowledge to include real-time, specific information from external sources. This capability significantly improves the utility of LLMs for tasks that require current knowledge or specialized information beyond their training datasets. Essentially, RAG enables the development of systems that can ingest large volumes of documents, allowing users to ask complex natural-language questions and receive precise answers from extensive, diverse datasets.
Read more on AI
Finding the best LLM—a guide for 2024Small language models (SLMs)—a gentle introduction
Generative AI implementation in business: how to do it responsibly and ethically
AI in manufacturing: Four nonobvious implementations
Ten business use cases for generative AI virtual assistants
Generative AI in knowledge management—a practical solution for enterprises
Is ChatGPT dreaming of conquering the world? Generative AI in business
The power of generative AI at your fingertips—Azure OpenAI Service
Unleash your superpowers with new Power Platform AI capabilities
Curbing ChatGPT hallucinations with retrieval augmented generation (RAG)
What is LLM fine-tuning?
Fine-tuning a large language model (LLM) is tailoring a pretrained model to perform specific tasks with greater accuracy. Initially, these pretrained models are trained on extensive data sets, gaining a broad understanding of language. However, without additional training, they often lack the precise skills needed for specialized tasks. Fine-tuning adjusts a model’s parameters using new datasets that mirror the tasks it will perform, thereby enhancing its performance. This process involves training the model on specific examples or domain-specific data that directly aligns with the desired outcomes, such as detecting sentiment in text or composing emails.
Selecting the right dataset for fine-tuning is crucial. The dataset must be large enough and diverse enough to avoid overfitting, where the model learns the training data too well and performs poorly on new data. It also needs to be highly relevant and clean—free from errors and inconsistencies. For example, in sentiment analysis, the training set should include a broad range of emotions and sources to ensure the model can accurately interpret various texts. If the model is being fine-tuned to generate emails, the prompts should be clear and detailed, encouraging the model to produce the right tone and content.
By fine-tuning LLMs with targeted, high-quality data, these models become much more than just broad language processors; they become specialized tools capable of sophisticated tasks. This makes them incredibly valuable for business applications that require nuanced understanding and responses, enhancing their effectiveness in specific domains.
What is prompt engineering?
Since large language models (LLMs) have the ability to learn new tasks on the fly, prompt engineering allows users to guide LLMs to generate desired outputs by providing precise instructions through well-crafted user inputs. This method involves drafting and refining input prompts that steer the AI’s generation of outputs using various techniques. By incorporating detailed instructions and contextual information, prompt engineering significantly enhances the AI’s performance on specific tasks.
The benefits of prompt engineering are manifold: it ensures the AI’s responses are more relevant and accurate; improves the model’s efficiency by reducing the need for retraining or extensive data inputs; and can even mitigate issues like catastrophic forgetting, where an AI loses previously acquired knowledge when adapting to new tasks.
Despite its relatively recent emergence as a field, there is abundant literature on prompt optimization, offering a variety of techniques and guidelines for prompting strategies. More advanced methods include zero-shot, few-shot, and chain-of-thought prompting.
Zero-shot prompting challenges a model to tackle tasks it has not been specifically trained for, testing its ability to generalize from existing knowledge to new situations without prior examples. This method evaluates the model’s intuitive understanding and flexibility.
Few-shot prompting, also known as in-context learning, supplies the model with a small number of examples, or “shots,” to guide its responses. These examples serve as contextual information, enabling the model to better comprehend the task and produce the desired output. This method significantly enhances the model’s accuracy for specific tasks.
Chain-of-thought prompting (CoT) introduces a sophisticated technique where the model receives step-by-step instructions, breaking down complex tasks into manageable reasoning steps. This method not only improves the model’s understanding and output accuracy but also ensures the results are logically coherent and contextually appropriate.
Prompt tuning involves optimizing the behavior of an AI platform by leveraging automated approaches, optimization techniques, and specialized software or tools to find the best prompt for a given task. It includes fine-tuning prompt parameters to maximize the performance of the LLM and using optimization algorithms to adjust prompt templates based on a dataset of questions and answers.
RAG vs. fine-tuning vs. prompt engineering—which strategy to choose
The choice of strategy depends on the desired outcome. In many cases, prompt engineering, particularly with advanced techniques, is enough to produce the desired output quickly and simply. It is the least technically complex option, and it is accessible to nearly all users. However, the results can vary between users and may occasionally be inaccurate or distorted (“hallucinations”).
If the goal is to consistently obtain accurate and precise answers grounded in verified knowledge, retrieval augmented generation (RAG) is more suitable. This approach involves a more complex setup, creating a RAG pipeline where the large language model (LLM) fields user queries and generates responses based on reliable sources such as knowledge bases or technical documentation. Additionally, RAG is a good solution when data privacy is crucial. In the RAG pipeline, the LLM and the knowledge base are separated, ensuring that no part of confidential company data ends up in a publicly accessible model. That is why the RAG approach is often utilized in enterprise knowledge management systems, where we need to balance between the power of generative AI with security and confidentiality requirements.
If neither prompt engineering nor RAG meets the requirements of a specific use case, fine-tuning an LLM may be necessary. However, this approach comes with significant drawbacks: it can degrade model performance if not executed correctly, and it requires high-quality, consistent data, as well as substantial computing resources.
In particular contexts, however, fine-tuning is beneficial, especially when dealing with narrow domain-specific applications, such as adapting a model for medical use. It can also resolve issues when prompt engineering becomes too cumbersome—if the model frequently fails to deliver the exact desired output and a fixed response format is needed, fine-tuning might be more efficient.
Fine-tuning also proves useful when speed, security, and hosting costs are critical, allowing organizations to manage models directly on their own infrastructure, thus ensuring data privacy. Finally, it can be advantageous when a model needs to understand specific cultural or linguistic contexts, ensuring it recognizes local idioms or terms.
It is also worth noting that an approach consisting in building small language models has been gaining ground recently. These models do not have as many parameters as GPT-4 or Gemini but are still powerful and capable of performing well in specific tasks, especially in environments where response speed and latency are crucial.
Wrap-up
When choosing the best strategy to address hallucinations in LLMs, it is crucial to find a balance between your goals and the complexity of the chosen approach. Prompt engineering is recommended as the starting point because it is straightforward to implement, does not require technical expertise, and uses minimal computing resources. However, prompt engineering typically only works for user-specific scenarios and tasks and may not consistently yield similar results for different users.
For situations where consistency, accuracy, and data privacy are paramount, building a retrieval augmented generation system is a suitable choice. The RAG approach is particularly beneficial for enterprises that produce large volumes of documents dispersed across various systems and knowledge bases.
Fine-tuning is the most technically complex option and also demands considerable computing resources. It should be considered a last resort. But in scenarios where precision and customization are crucial, fine-tuning can be the most effective approach, ensuring the model meets the specific needs of the task at hand.
If you are still unsure which strategy is the best option for you, or have a business-specific use case, we are eager to help. At Fabrity, we build AI-powered solutions ensuring enterprise-grade security and confidentiality. Drop us a line at sales@fabrity.pl so we can discuss the details.
