Customers expect quick answers, but finding the right information is not always easy.
In many cases, the answer to a customer’s question already exists—on a website, in a product manual, or in a company’s internal documentation. The problem is that the customer cannot find it, while the employee who could help is busy with other work.
RAG offers a practical way to make information more accessible and easier to find. A customer asks a question in their own words, and the assistant responds based on materials selected by the company. If the application also displays its sources, users can verify where the answer came from. Such an assistant can support customers on a website as well as employees in an internal system, while ensuring that each person can access only the information they are authorized to view.
What Does RAG Stand For?
RAG stands for Retrieval-Augmented Generation. In general, RAG belongs to the field of artificial intelligence, or more specifically, to methods used with generative AI and language models. The system first retrieves information relevant to the question and then asks a language model to formulate an answer based on it. RAG therefore combines information retrieval with text generation.
We can think of a language model as a capable new member of the customer support team. It communicates well but does not yet know our services or internal rules. It needs the right information, and RAG selects it based on the question being addressed.
A language model does not automatically have access to the latest content on a company website, internal documents, or new working procedures. The user interacts with a specific application or AI assistant, while RAG works with the company’s information in the background.
What Happens After a Question Is Submitted?
Imagine a common situation: a website visitor wants to book a specific service. Instead of browsing through several pages, they can enter their question in a chat. A short process then takes place in the background:
- The system finds sections of documents related to the question.
- It sends the selected information to the model together with the question and instructions on how to respond.
- The model creates an answer based on the provided information. The application can also add links to the sources used.
The first step is to prepare the content itself. Documents are usually divided into smaller, semantically coherent passages and stored in a search index. In addition to exact keywords, the system can also search for texts with a similar meaning. This is where embeddings come in. Embeddings are numerical representations of text that allow the system to compare a question with the available documents.
In a typical application, this process remains in the background. The user sees the question, the answer, and, where applicable, the sources on which the answer is based.
How We Used RAG in a Client Project
In a project for a private clinic, we developed a customer support assistant. The AI assistant works with selected content from the clinic’s website and uses it to answer questions about services, contact details, and bookings.
The system searches in two ways. Keyword search helps it identify exact service names and specific terms, while semantic search helps it understand questions phrased in natural language. The results are combined, and the most relevant passages are passed to the language model. The model is instructed to answer based on the provided materials and to support factual claims with links to the relevant sources.
When preparing the content, we preserved the structure of the website, including headings, paragraphs, and lists. Sections containing reviews and navigational content are filtered out so that they are not confused with information about services and operations. After all, the quality of an answer also depends on which passages the system finds and sends to the model.
How It Works Technically
The entire process can be divided into four steps:
- Preparing the sources
First, the sources from which the RAG system will retrieve information need to be prepared. The content is divided into smaller, semantically coherent passages and stored in a search index together with information about its origin. An embedding – a numerical representation of its meaning—is also created for each passage. Whenever the source content changes, the index must be updated.
- Retrieving relevant passages
After a question is submitted, two types of search run simultaneously. The first uses keyword matching and is particularly effective at identifying service names, product names, and specific terms. The second uses embeddings to compare the meaning of the question with the meaning of the stored passages.
The results from both searches are then combined using Reciprocal Rank Fusion, which gives preference to passages that rank highly in both result lists. The model ultimately receives no more than the six most relevant passages.
- Generating the answer and adding sources
The selected passages are added to the prompt for the language model, together with the source name, section, URL, and a numerical reference marker. The model is instructed to answer based on these materials and to use the markers to identify factual claims.
Once the answer has been generated, the backend retains only the markers associated with the sources retrieved for that specific question. This prevents the answer from linking to sources that were not provided. However, it is still necessary to test whether each claim is genuinely supported by the source being cited. - Passing the request to another tool
RAG works with information and generates answers. If the system needs to perform a specific action, such as changing an existing booking, it requires a separate tool and an integration with the relevant application. In our solution, such a request triggers a form or is forwarded to customer support.
When Is RAG Worth Considering?
RAG is most valuable when answers depend on a company’s own information and that information is scattered across multiple sources or changes regularly. A typical example is customer support, where the same services and procedures need to be explained repeatedly. RAG can also help employees search internal policies or support sales teams working with product documentation.
Whether the solution truly helps must be verified in real-world use. The number of conversations alone does not reveal whether users received useful answers. It is more important to track how many users found the right information without having to search further, how quickly they found it, and which questions still required human assistance.
Sometimes Less Technology Is Enough
RAG is not necessary for every AI use case. When processing a short document as a one-off task, the document can be included directly in the prompt. If someone simply needs to find and open a known page, a conventional search function may serve them better.
In addition to RAG, language models can also be adapted through a process known as fine-tuning. However, these approaches serve different purposes. With RAG, the model receives supporting information when generating an answer. Fine-tuning changes the model’s behavior using training examples. If a company document changes, RAG does not require the generative model to be retrained; instead, the source and search index need to be updated. The two approaches can therefore complement each other.
To provide the exact status of an order, calculate a price, or modify a booking, an application needs access to a database or the relevant interface. Retrieving text cannot perform such an action, although it can explain the process and direct the user to the appropriate next step.
An Answer Is Only as Good as Its Sources
RAG can reduce unsupported answers, but it cannot prevent them entirely. The search process may retrieve an unsuitable passage, while the model may interpret the correct source incompletely or incorrectly. A link to a source does not automatically support every sentence in an answer. This is why testing should evaluate the quality of retrieval and the quality of the answer separately.
But working with the data does not end there. Sources need to be maintained, and the solution should be tested using questions that people actually ask. When RAG works with internal documents, it must also respect access permissions. Each user should receive only the information they are authorized to access.
Start with a Specific Problem
A successful RAG pilot does not need to cover all company information from the outset. Start with one area where questions frequently recur and searching for answers places an unnecessary burden on customers or employees. Gather reliable source materials, list the most common questions, and define what a useful answer should contain. Even the first test will show whether the solution can reduce the time spent searching for information and what still needs to be improved.
If you are considering RAG for your company, the materials your colleagues already use to answer questions are a good place to start. By testing the system with real questions, you can quickly determine whether RAG could add value in your specific use case and which features the pilot should include.
If you have an idea for using RAG but are unsure how to approach it, we can help you assess its value and technical feasibility. Together, we will identify the right problem, design the solution, and turn your idea into a functional application that works with your company information.
Get in touch, and let’s explore the potential of your idea.
Kontakt
