How to use retrieval-augmented generation (RAG) effectively 

Companies across industries can use retrieval-augmented generation (RAG) to build powerful, reliable, and personalized AI features in their products.

We’ll break these use cases down by highlighting how real-world companies use this approach to power their AI features and products. 

But first, let’s align on the RAG workflow.

How RAG works

Put simply, RAG is a process that allows large language models (LLMs) to fetch and use context to generate more accurate and helpful outputs.

The RAG workflow consists of a few steps:

• A specific query—potentially from a user—gets embedded, or turned into a vector, and added to a vector database via an embedding algorithm

• All of the embeddings that are semantically similar to the embedded query are identified in the vector database and, along with the embedded query, get fed to an LLM

• The LLM uses all of these embeddings to generate an output. In many cases, the outputs include links to sources in case the user wants to learn more

RAG use cases

Here are just a few impactful ways that companies currently use RAG to build differentiated AI products and features.

Assembly

Assembly, an employee recognition software, offers an enterprise AI search solution (dubbed “DoraAI”) that’s built on RAG.

Essentially, once a user asks DoraAI a question (e.g., what’s the company’s PTO policy), the query gets embedded. 

Semantically-similar embeddings get identified (e.g., the company’s PTO policy documents), and both the embedded query and the identified embeddings get fed to the LLM Assembly uses, which generates a response for the user (e.g., “The company’s PTO policy is…”).

Here’s more on how Assembly’s enterprise AI search works and how they power it via Merge:

Juicebox

Juicebox is an AI-powered platform that helps recruiters identify and connect with high-fit candidates.

RAG enables a key workflow in their product: Once a user selects a job they want to find candidates for within Juicebox, the platform will embed the job description for that role and search for semantically-similar candidate profiles in a vector database. 

The LLM can then use the retrieved profiles and the embedded job description to refine, rank, and enrich the results—surfacing thousands of ideal candidates for the role.

Here’s more from Juicebox’s CEO and co-founder:

Ema

Ema is a Universal AI Employee that lets customers deploy AI agents to perform all kinds of tasks across the business—from troubleshooting and resolving customer issues to helping employees access company information and perform actions based on that information.

For example, Ema can help sales reps use their Proposal Writer AI Agent to create proposals for prospective customers.

The query could be whatever inputs the rep gives the agent. The agent can then go on to identify semantically-similar embeddings—such as examples of previous proposals drafted for similar prospects—in a vector database. 

From there, the agent can use the relevant documents as well as the initial query to generate a highly-relevant and accurate proposal.

How to implement RAG effectively

While the best practices for RAG can vary based on your use cases, there are generally a few worth following across the board.

Normalize the majority of your data before embedding it

Normalized data, which is data that’s standardized and transformed into a consistent format across multiple systems or platforms, is critical for RAG.

The process of normalizing data lets you remove sensitive and duplicate information. This, in turn, enables the LLM that uses the data to avoid generating outputs with potentially compromising and/or redundant information. 

Just as important, since normalized data doesn’t have unnecessary information and is consistently formatted the same, an embedding algorithm is more likely to embed it accurately. 

This means the embedded query can successfully identify the most relevant embeddings, which allows the LLM that receives these embeddings to then generate precise outputs.

Use raw data for edge cases

Your customers might use unique types of data to support their specific business processes.

It wouldn’t make sense to normalize the data that meets this criteria, but you may want to incorporate it into your RAG workflow when it’s relevant to a RAG use case you support for a customer. 

In short, it’s worth paying close attention to your customers’ data sets so that you can identify and account for these scenarios in your RAG implementation.

Leverage a unified API platform 

A unified API platform allows you to add hundreds of integrations to your product through a single integration build. These integrations also span several software categories—from file storage to CRM to ticketing to ERP solutions.

The platform can also normalize customer data automatically through its defined data models and access raw data from customers’ applications as needed.

Taken together, your platform can support any RAG use case effectively.

{{this-blog-only-cta}}