If pricing makes you nervous, you build less.

We’ve seen that first‑hand. When your bill swings with every spike in documents or searches, teams hesitate to ship, and finance can’t forecast.

Our philosophy towards building Meilisearch – from our [dual licensing model](https://www.meilisearch.com/blog/enterprise-license) to how we price – has always centered on giving our customers maximum flexibility.

That’s why we’re introducing resource-based pricing: a developer-first model that stabilizes spend, aligns pricing with actual infrastructure, and provides clear controls over your performance and infrastructure.

## Why we’re adding another pricing model

For a while, usage‑based pricing looked like the obvious path. It matched the serverless zeitgeist and mapped neatly to “documents” and “searches.” But reality didn’t fit the abstraction:

- **Unpredictability drove churn**. Teams with a large amount of data and relatively few searches were penalized. Bills rose in ways that were hard to anticipate.
- **Identical “units” did not have identical costs**. A million tiny product records and a hundred thousand large PDFs do not consume the same resources – yet usage pricing treats them the same.
- **We were over‑optimizing for plan limits**. Internally, we found ourselves hand‑tuning infrastructure to fit plan boxes rather than letting customers scale the resources they needed.

Customers (especially developers and CTOs) told us what they wanted instead: price stability, performance control, and transparency on what drives cost.

## What “resource‑based” means

With resource‑based pricing, you select dedicated compute for your project – and you pay **only** for those resources.

![Resource-based pricing](https://unable-actionable-car.media.strapiapp.com/resource_based_pricing_414c8e98aa.png)

- **Compute**: choose the CPU and RAM you want. Those resources are dedicated to your project via predefined tiers (e.g., XS through 4XL self‑serve; larger tiers via Sales).
- **Storage**: storage auto‑scales and isn’t customizable at the moment.
- **No caps on “documents” or “searches.”** Your resources define your limits and performance – scale them up or down as your needs change.

You can upgrade resources self‑serve. At the moment, downgrades are available via support, but very soon you’ll be able to do them from the Cloud interface.

Billing is at the team level. You can run mixed billing in the same invoice (usage‑based projects alongside resource‑based ones), and you can upgrade/downgrade resources as you go (downgrades via support for the moment, as mentioned above).

This mechanism is the same for both self-serve and enterprise. The Enterprise level then adds SLAs, SSO, support/SLOs, volume discounts, and optional add‑ons – not different machines.

Why this is better for teams needing to scale:

- **Predictability**: stabilize spend as you grow – no more surprise overages tied to abstract units.
- **Control**: developers can dial resources to match performance goals – whether that’s sub‑second search or faster indexing.
- **Transparency**: your bill tracks real cost drivers (CPU, RAM, storage), making budgeting (and explaining costs) far easier.

It also improves activation. If indexing or search feels slow, you can bump resources for a few cents per hour to validate your use case, then tune back once you’ve optimized.

## The developer experience: monitor, tune, repeat

Alongside the pricing model, we’re rolling out a new **Infrastructure** tab in the [Cloud UI](https://cloud.meilisearch.com/) so you can see what’s happening under the hood:

![The new Infrastructure tab allows you to monitor resources](https://unable-actionable-car.media.strapiapp.com/infrastructure_tab_278a604bfd.png)

The new interface allows you to monitor CPU, memory, and storage usage live and take action proactively to avoid performance issues.

Coming soon are also alerts and recommendations. You’ll be able to set up proactive alerts when resource usage reaches certain thresholds (e.g., 80/90/95%) to avoid outages and slow performance.

Wherever possible, we’ll suggest optimizations first (configuration and data‑model tips), then scaling guidance if you still need more headroom.

## How does this fit with usage‑based (we’re keeping both)

This isn’t an either/or ideology. It’s a toolbox:

- Start with usage‑based if you want near‑zero friction and the simplest mental model for a quick evaluation.
- Switch to resource‑based when predictable spend and performance control become more important than per‑unit simplicity.

We’re not forcing migrations. Existing plans are grandfathered. Going forward, when you create a project, you’ll be able to choose usage‑based or resource‑based in the flow – and mix them across a team as your portfolio grows.

## Add‑ons and future flexibility

Some capabilities sit above pricing mode:

- Team‑level add‑ons: SSO, SOC 2, analytics, merchandising
- AI services as first‑class add‑ons on a single Meilisearch bill: embeddings (by size), LLM tokens, re‑ranking models
- Horizontal scale via sharding (multiple machines per project) for advanced workloads

Bandwidth billing and alerting are planned future improvements.

## What to expect at launch

- Fully functional backend: applicable to any new or existing project; resource upgrades are self‑serve; downgrades via support
- Team‑level billing with mixed cycles: usage‑based and resource‑based projects on one invoice
- New Infrastructure tab in the dashboard to monitor CPU, memory, and storage
- Self‑serve and enterprise both supported; enterprise layers on support and governance – not different pricing mechanics

We are confident this new pricing model will lead to lower churn and faster activation because customers can plan with confidence and validate performance without friction.

## Choosing the right model (quick guide)

Choose usage‑based if:

- Business stakeholders lead the evaluation
- Traffic is modest and bursty
- You want the simplest on‑ramp

Choose resource‑based if:

- You’re a developer/CTO who needs predictable spend
- You want explicit control over performance via CPU/RAM
- Your data shape doesn’t map cleanly to “documents/searches”

## How to try it

- New projects: pick “Resource‑based” when creating a project.
- Existing projects: talk to our support team to switch to resource‑based pricing.
- Then, visit the Infrastructure tab to monitor resource usage and adjust CPU/RAM as needed. If you want help optimizing before scaling up, talk to us – we’re happy to review settings and tradeoffs with you.

<Callout intent="note">

Free trials are only offered for usage-based plans, if you want to start a resource-based project, you’ll need to purchase a subscription from the get-go. Of course, you can also start a trial on usage-based to verfy that Meilisearch is the right fit for you and then talk to Support to get migrated to a resource-based plan.

</Callout>

## Our goal

Pricing should reflect reality, not get in your way. Resource‑based pricing puts control back in developers’ hands and makes your bill as predictable as your architecture.

I’m excited to see what you build when pricing is finally one of the simplest aspects of running a search engine.

Scale with confidence: Introducing resource‑based pricing

We're kicking off [Launch Week](https://www.meilisearch.com/launch-week-q3-2025) with something we've been excited to share: Meilisearch Chat, a new way to make your data conversational.

For the first time, you can turn your existing Meilisearch index into an AI-powered chat experience – without complex infrastructure, separate vector databases, or months of custom development.

## Search is becoming a conversation

Your users and buyers don't want to search anymore – they want to ask. They expect apps, documentation, and internal tools to understand natural language and give them direct answers, not endless lists of results.

Until now, that's been really hard to deliver. Traditional conversational AI required juggling multiple LLM calls, vector databases, reranking pipelines, and endless tuning. The complexity alone made it expensive and slow, often taking months just to get a working prototype.

Meilisearch Chat changes all of that.

## One API call, complete conversational AI

The new `/chat` brings AI-native search to your Meilisearch data through a single API endpoint. It handles the entire Retrieval-Augmented Generation (RAG) workflow, from understanding the query to generating the response, automatically.

Here's what you get:

**Conversational AI for search** – Users ask natural-language questions and get precise, contextual answers from your Meilisearch index.

**Built-in RAG capabilities** – Retrieves relevant information, contextualizes it in your factual data, and generates accurate responses. No extra vector database, reranker, or separate AI service required.

**Native integration** – Runs directly on your Meilisearch instance with a single, OpenAI-compatible endpoint.

Less infrastructure. Faster implementation. No compromise on quality or accuracy.

## Why this matters for your business

**Get to market faster**

Meilisearch Chat turns months of AI integration work into days. You can prototype conversational experiences immediately, test them with real users, and iterate quickly without needing a dedicated AI team.

**Lower costs, less complexity**

We've streamlined the entire RAG pipeline, so you don't need separate LLM orchestration, reranking logic, or vector storage. Fewer moving parts means lower maintenance costs and less things that break.

**Better user experiences**

Whether it's customer support or internal knowledge bases, your users get direct, reliable answers in context – not just a list of links. That means faster resolution times, higher engagement, and happier users.

## Built for real-world use cases

**Retail & e-commerce** – Help shoppers find exactly what they need through natural conversation.

**Customer support** – Let users ask questions in plain language and get answers directly from your documentation or help center.

**Internal knowledge bases** – Turn your company documentation into a chat assistant that answers questions instantly.

**Developer docs** – Add conversational search to your API or SDK documentation so developers can find what they need faster.

## Simple to implement, easy to scale

If you're already using Meilisearch, adding conversational capabilities is straightforward. The `/chat` endpoint is [OpenAI-compatible](https://www.meilisearch.com/docs/learn/chat/getting_started_with_chat#configure-a-chat-completions-workspace), which means it works with your existing LLM setup and fits right into your current workflows.

You can start with your existing indexes – no need to rebuild or re-embed your data.

## A new approach to AI-powered search

With Meilisearch Chat, we're making it simpler to bring conversational AI to your data. Instead of managing multiple services and complex pipelines, you get a complete conversational search experience through one endpoint.

This isn't just another feature – it's a new foundation for more natural, human-like interactions with your data.

**Ready to make your search conversational? Explore the [docs](https://www.meilisearch.com/docs/learn/chat/getting_started_with_chat) for how to get started and [try /chat today](https://www.meilisearch.com/cloud)**

Introducing Meilisearch Chat: conversational search for your data

When building AI systems, one of the most significant decisions to make is how your model accesses and uses knowledge. Do you need real-time data every time, or can you rely on pre-stored, cached information? 

This is where the difference between retrieval-augmented generation (RAG) and cache-augmented generation (CAG) lies: 

* RAG retrieves live and updated data, making it ideal for situations where changes occur quickly and accuracy is crucial.
* CAG relies on cached information, delivering answers faster and more cheaply. It works well for repeated or predictable questions.
* RAG provides fresh, reliable responses and can handle various queries. However, it can be slow, expensive, and requires a solid infrastructure to run smoothly.
* CAG is quick, efficient, and easy to scale, but it can get outdated. It is also limited to what is already stored and requires regular updates to stay useful.

Choosing between the two comes down to considering your users: 

* Are the questions predictable or wide-ranging? 
* How up-to-date does the information need to be?
* What is your budget and infrastructure like?

You do not always have to pick just one. Hybrid approaches can combine both RAG and CAG. They allow the speed of cached answers while still retrieving live data when needed.

By the end of this article, you will understand how RAG and CAG work and which one better suits your projects.

## What is RAG (Retrieval-Augmented Generation)?

[RAG is a technique](https://www.meilisearch.com/blog/what-is-rag) that enhances the capabilities of generative AI by using external knowledge to provide more accurate responses to user queries. 

Rather than solely relying on the datasets the AI model was trained on, RAG searches a vector database or docs, retrieves the most relevant information, and then uses it to generate a response. This makes the answers more accurate and up-to-date. 

## What is CAG (Cache-Augmented Generation)?

CAG builds responses around the specific context window you provide. 

Unlike RAG, which searches external databases for information, CAG uses past interactions to shape its answers. This makes its output feel more relevant and personalized. 

In short, CAG makes AI less generic and more tuned to your needs by getting the proper context before generating a response.

## How does RAG work in AI models?

RAG combines two steps: retrieving information and generating a response. 

When you ask a question, the model first searches through an external knowledge base to find the most relevant documents. This retrieved information acts like notes or references that the large language model (LLM) keeps on hand for response generation. 

![A graphic depicting how RAG works.](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_8e6de82c99.png)

When generating its response, the LLM uses both internal knowledge obtained during training and the newly retrieved relevant context to return a response. It makes the response sound natural and grounded in real data. 

RAG co/mbines the best of both search engines and language models, ensuring the AI system is less prone to hallucinations.

Now, let’s take a closer look at CAG and how it works.

## How does CAG function in AI systems?

CAG works by providing AI with the necessary background information to generate an accurate response. When you interact with a CAG-enabled system, it does not rely on general knowledge alone. 

Instead, it takes note of the context you provide, such as your past conversations or personal preferences. This information is cached temporarily using a key-value (KV) cache and can be referenced while generating the responses.

![How CAG Works.png](https://unable-actionable-car.media.strapiapp.com/How_CAG_Works_8f5381b703.png)

During the generation process, the LLM combines this cached context with its pre-trained knowledge to produce personalized and relevant answers. 

Fundamentally, this limits the possibility of generating generic responses. CAG also continuously updates its context and determines the most relevant piece, ensuring the conversation stays aligned with your needs. 

## What is the main difference between RAG and CAG?

The main difference between these two [systems](https://www.meilisearch.com/blog/rag-types) is how they handle information when generating responses. 

RAG obtains external resources for knowledge augmentation, whereas CAG relies on previously stored data. 

Below is a quick preview of their key differences: 

![RAG vs CAG.png](https://unable-actionable-car.media.strapiapp.com/RAG_vs_CAG_cbdc135b87.png)

* **Data source:** RAG retrieves fresh information from external databases or search engines whenever a query is made. CAG works with pre-cached data that is already stored and ready for reuse.
* **Latency & performance:** RAG tends to have higher latency, as it requires a retrieval step every time, whereas CAG achieves low latency through preloading and reuse.
* **Cost efficiency:** With RAG, costs are typically higher due to the constant need to retrieve and process data. CAG, on the other hand, is more cost-effective since it eliminates repeated queries by reusing stored data.
* **Use case fit:** RAG is best suited for scenarios where real-time accuracy is essential. CAG is more useful for repetitive tasks where the same knowledge is applied often.
* **Scalability & maintenance:** RAG can be harder to maintain because it relies heavily on live updates and connections to external sources. CAG is easier to scale since cached data can be refreshed periodically and managed with less effort. 

Now, let’s look at the pros and cons of both RAG and CAG.

## What are the pros and cons of RAG?

RAG is beneficial for situations where accuracy is crucial, but it also has some limitations. Here are some upsides and downsides of RAG:

**Advantages**

* **Provides fresh information through real-time retrieval:** It can pull in the most up-to-date information, which is particularly beneficial when details change a lot.
* **Has a broad knowledge base:** Because it connects to external sources, it has access to a broader pool of knowledge than an AI model on its own.
* **Reliable for fact-checking:** It is beneficial for fact-checking or knowledge tasks where precision is key.

**Disadvantages**

* **Slower performance:** Since RAG must fetch new data, it has a higher inference time and responds more slowly.
* **Higher costs:** The constant retrieval of information makes it more expensive to operate.
* **Maintenance challenges:** RAG relies on external systems, so the results can be inaccurate if a source goes down or changes its data.
* **Complex setup:** Setting RAG up and keeping it running smoothly can be technically demanding.


## What are the pros and cons of CAG?

CAG may be faster and cheaper, but it does not come without downsides. Here are some advantages and disadvantages of CAG:

**Advantages**

* **Faster responses:** Since the data is pre-cached, there is no need to fetch information repeatedly, resulting in faster reactions.
* **Cost-effective:** CAG is more cost-effective and efficient because cached material reduces the need for repeated processing.
* **Efficient for repeat tasks:** CAG is ideal for repetitive queries, where the same information is frequently required.
* **Easier to scale:** It is easier to scale and maintain, as updating the cache can be done periodically with minimal hassle.

**Disadvantages**

* **Needs refresh cycles:** To remain useful, the caches used in CAG require periodic reprocessing and refreshment.
* **Risk of outdated information:** Cached data can go stale, meaning it might miss recent changes or updates.
* **Limited extended context:** The CAG is limited in terms of what is already stored, which means the scope of answers is narrower.
* **Potential bias:** If the cached content is biased or incomplete, the responses will reflect those flaws.

Wondering when it is best to use RAG over CAG? Let’s discuss the best-fit scenarios for both.

## When should you use RAG instead of CAG?

RAG is the better choice when your [retriever](https://www.meilisearch.com/blog/rag-tools) must fetch new information from an external source for every user’s query. 

For instance, in financial market updates, things move fast. Caching may not be helpful, as you do not want your AI system to return stale information. In such a case, RAG is an excellent approach. 

It can also be used in fact-checking scenarios or high-stakes research projects where credibility depends on having the most current and comprehensive information available. 

Another good use case of RAG over CAG is in scenarios where you do not expect repeated questions from users. It could also be used when you cannot precisely predict what users will ask. 

In these cases, caching will not be particularly helpful because the LLM must retrieve new information from the knowledge sources each time. 

In short, use RAG when your goal is to get current and accurate results, even if it takes a little longer or costs more.

## When is CAG preferable to RAG?

Use CAG when optimization for speed and low latency is key. By reusing embeddings and cached context, CAG reduces costs and improves response times for predictable scenarios.

This is particularly useful in situations such as customer support chatbots, where people frequently ask the same questions repeatedly. Answers to questions such as ‘How do I change my password?’ or ‘What is the refund policy?’ rarely change, so caching saves time and cost without sacrificing accuracy.

CAG also works well for internal knowledge bases where employees frequently need the same set of guidelines or instructions. 

Another great fit for CAG is large-scale deployments where keeping retrieval costs low is essential for scalability. 

While CAG may not always reflect the latest updates, its strengths lie in consistency and speed. So, if the information you are dealing with does not change often, then CAG is a practical choice.

## How do you choose between RAG and CAG for your project?

Choosing between RAG and CAG really depends on what your project needs most in terms of freshness, speed, or efficiency. 

Here are some key criteria to consider:

* **Nature of queries:** If your system faces unpredictable queries, RAG is stronger since it can reach out for fresh information. If the questions are repetitive (for example, common FAQs) then CAG will serve you better.
* **Budget considerations:** RAG is more costly because every query must undergo the retrieval phase. On the other hand, CAG is cheaper because the cached data is reused.
* **Performance requirements:** RAG can be slower, but it gives you depth and updated context. CAG is faster because it does not need to fetch new data each time it is executed.
* **Freshness of information required:** When working on tasks that require real-time information updates, such as financial data, it is recommended to use RAG. For stable knowledge, cached responses are more than enough.
* **Growth potential:** Scaling RAG can become complicated due to constant retrievals. Scaling CAG is easier, since updating caches is simpler to manage.
* **Handling varying outputs:** CAG keeps things quick and efficient when you do not need much flexibility, but when accuracy is non-negotiable, using RAG is the better option.
* **Technical setup:** RAG needs a stronger infrastructure to maintain live connections. CAG is easier to deploy due to fewer moving parts.

## Can RAG and CAG be combined?

Yes, you can combine RAG and CAG. In fact, combining RAG and CAG often gives you the best of both worlds. These are called [hybrid approaches](https://www.meilisearch.com/blog/rag-techniques). 

In such RAG pipelines, KV cache data is served first, and if it is missing or inadequate, the system triggers the retrieval step.

RAG and CAG can be combined in different ways, depending on your project’s goals. For example, you can build a:

* **Fallback hybrid:** The system defaults to cached responses for speed, but if the cache does not have a good answer, it switches to RAG for live retrieval.
* **Selective hybrid:** Certain types of queries are always routed to RAG (e.g., time-sensitive information) while routine or repetitive queries go straight to CAG.
* **Blended hybrid:** Both RAG and CAG are used together. Cached data provides a quick first draft, and RAG fills the gaps with fresh context if needed.
* **Tiered hybrid:** Responses start with cache for instant speed, then RAG silently updates or corrects the answer in the background so the answer to the query is fresher.

The benefits of a hybrid system are clear: faster responses, reduced costs, and still having the option to retrieve real-time data when needed. 

However, combining both also brings challenges. You need to decide on the best way to determine when to use the cache and when to fetch new information. Managing two approaches together can also add complexity to the system’s infrastructure and maintenance.

## RAG vs CAG: Choosing the right path for smarter, faster AI

When it comes to building more intelligent AI systems, RAG gives you freshness and accuracy by retrieving live data, while CAG provides you with speed and efficiency with its cached responses. 

The real power lies in knowing when to use each one or even combine them in a hybrid system. 

By balancing your API design, workflows, and knowledge integration needs, you can create LLMs that deliver optimized, real-world performance.

RAG vs. CAG: The Smarter Choice for Your AI Stack

Retrieval-augmented generation (RAG) has rapidly become one of the most effective methods for building AI systems. While most apps lean on Python, Ruby offers a lightweight path to implementing RAG pipelines without the need for large frameworks.

This guide will walk you through the essentials, from understanding what RAG means to building a working Ruby app.

You’ll discover: 

* What RAG is and how it fits into Ruby projects without heavy frameworks.
* Why developers pick Ruby for flexible, lightweight RAG setups.
* Libraries and services that make retrieval and generation work smoothly.
* Clear steps to build a working RAG app using plain Ruby.
* The difference between Ruby’s ecosystem and performance with Python for RAG development.
* How tools like Meilisearch help you build fast and reliable retrieval layers.

By the end, you’ll have everything you need to develop a functional RAG pipeline in Ruby from scratch.

## What is a RAG application in Ruby?

A RAG application in Ruby combines retrieval, augmentation, and generation to create smart, context-aware LLM responses. For details on how it works, you can take a peek at[ our guide on what RAG is](https://www.meilisearch.com/blog/what-is-rag). 

In essence, RAG retrieves relevant information from a vector knowledge base, augments a prompt with that data, and then sends it to a large language model (LLM) to generate accurate answers.

![How RAG Works.png](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_7352c5302f.png)

A frequent question is whether you can implement RAG without relying on heavy frameworks. Yes, you can. 

Ruby developers can rely on embeddings, search engines, and API calls to connect with LLM services for seamless integration. As a result, the AI-powered tools built through Ruby are incredibly lightweight. 

The following section explains why Ruby is a suitable option for using RAG.

## Why use RAG with Ruby?

In a RAG system, Ruby’s clean syntax and expressive language makes retrieval and embedding management relatively easy. Plus, you can connect to LLM APIs without any heavy frameworks. 

While the focus of this guide isn’t on Rails, generally, many teams already run production systems on Ruby or Rails. Adding RAG capabilities on top of that lets them integrate vector search and generation directly into existing apps without migrating their stack. 

Ruby’s ecosystem, although not as mature as Python's, simplifies API calls and background job handling, allowing developers to build lightweight scripts that optimize retrieval, augmentation, and generation.

Next, let’s look at the essential tools you’ll need to build a RAG application in Ruby.

## What tools are needed for RAG in Ruby?

To build a RAG application in Ruby, you need a combination of libraries and APIs that cover retrieval, embedding generation, and LLM integration. 

While the Ruby ecosystem is still growing, several reliable options are available to set up a complete RAG pipeline, many of which overlap with those discussed in [our RAG tools comparison](https://www.meilisearch.com/blog/rag-tools). 

Let’s look at the best ones below:

* **Langchainrb:** A Ruby library that aids in building retrieval and generation workflows through building blocks. It excels at supporting document loading, embedding generation, and basic orchestration.
* **Qdrant-ruby:** A popular open-source vector database as well as a Ruby client for Qdrant. It allows you to store and search embeddings efficiently from Ruby code.
* **Hugging Face or Transformer APIs:** These APIs can be called from Ruby to generate embeddings or run models remotely. They’re useful when you want to avoid maintaining your own ML infrastructure.
* **Other vector store gems:** Some developers use custom wrappers for databases, like PostgreSQL with pgvector, or they integrate with Meilisearch for hybrid keyword and [vector search](https://www.meilisearch.com/blog/what-is-vector-search).
* **LLM API gems:** These are gems for OpenAI, Anthropic, or other model providers that allow you to generate responses directly from Ruby.

There you have it: a set of tools associated with Ruby that provide the foundation for handling data retrieval and connecting to powerful language models.


## Step-by-step guide to building a RAG app in Ruby

This section is your walkthrough of a five-step process that covers data preparation, embedding generation, retrieval setup with Meilisearch, connecting to an LLM, and building a minimal interface to tie it all together.


### 1. Load and prepare your data

The first step is to bring text into your Ruby environment in a structured way. This could be from local files, APIs, or databases. 

You will need to clean the text, split it into logical chunks, and attach metadata (such as titles or URLs) so that retrieval later on is meaningful.


```ruby
# Example: Loading and cleaning a text file
file_path = "docs/guide.txt"
content = File.read(file_path).strip
chunks = content.split(/\n\n+/) # split into paragraphs
docs = chunks.map.with_index do |chunk, i|
  { id: i, text: chunk, metadata: { source: file_path } }
end
```


This preparation is the foundation that ensures your retriever can quickly find relevant information without having to parse messy inputs later.


### 2. Generate embeddings and store them

Once the documents are ready, you’re going to convert each chunk into a numerical embedding. You can use an API like OpenAI or Hugging Face for this purpose. 

Next, you will store the resulting vectors alongside the original text in a lightweight database or pair them with Meilisearch for hybrid retrieval.


```ruby
require 'net/http'
require 'json'

def embed(text)
  uri = URI("https://api.openai.com/v1/embeddings")
  headers = { "Content-Type" => "application/json", "Authorization" => "Bearer #{ENV['OPENAI_API_KEY']}" }
  body = { input: text, model: "text-embedding-3-small" }.to_json
  JSON.parse(Net::HTTP.post(uri, body, headers).body)["data"].first["embedding"]
end

embeddings = docs.map { |d| d.merge(embedding: embed(d[:text])) }
```


This gives your pipeline a semantic backbone for accurate retrieval.


### 3. Implement a retriever with Meilisearch

Now, it’s time to index your documents in Meilisearch. Why? To enable fast keyword and hybrid search. Feel free to store metadata alongside text for filtering as well. This gives you typo tolerance, relevance ranking, and blazing-fast queries.


```ruby
require 'meilisearch'

client = MeiliSearch::Client.new('http://127.0.0.1:7700', 'masterKey')
index = client.create_index('documents', { primaryKey: 'id' })
index.add_documents(docs)

# Query example
results = index.search('embedding generation')["hits"]
```


Developers often underestimate how significantly Meilisearch outperforms manual filtering or SQL full-text search in text retrieval. A clean index makes every later step more reliable.


### 4. Pass retrieved context to an LLM

Now that you have the retrieved context, let’s move it on to the LLM so the answers are actually grounded in data. You’ll build a prompt that includes the essentials, such as user query, brief instructions, and a trimmed context window. 

You can avoid the bloat by focusing on only the most relevant snippets. If you really want to show where the facts came from, just use citation markers. 


```ruby
require "openai"

class AnswerGenerator
  def initialize(model: ENV.fetch("OPENAI_MODEL", "gpt-4o-mini"))
    @client = OpenAI::Client.new(access_token: ENV.fetch("OPENAI_API_KEY"))
    @model  = model
  end

  def answer(query:, hits:)
    context = build_context(hits, max_chars: 6_000) # keep under model limits
    messages = [
      { role: "system", content: "You are a helpful assistant. Answer using only the provided context. Cite sources like [doc:index]. If the answer is not in the context, say you do not know." },
      { role: "user",   content: "Query: #{query}\n\nContext:\n#{context}\n\nAnswer:" }
    ]

    @client.chat(parameters: { model: @model, messages: messages, temperature: 0.2 })
           .dig("choices", 0, "message", "content")
  end

  private

  def build_context(hits, max_chars:)
    buf = +""
    hits.each_with_index do |hit, i|
      snippet = hit.dig("_formatted", "content") || hit["content"] || hit["snippet"] || ""
      block = "[doc:#{i+1}] #{snippet}\n"
      break if buf.size + block.size > max_chars
      buf << block
    end
    buf
  end
end
```


Several factors enhance the retrieval-to-generation step. 

Keep the temperature low so the model sticks to facts. If the context doesn’t support an answer, it should say so clearly. Trim or rerank longer contexts to stay within token limits. 

And don’t forget to log document IDs for easy traceability and later evaluation.


### 5. Build a CLI or service interface

Here’s a tidbit you’ll enjoy: wrap retrieval and generation in a small interface so you and your teammates can try the flow quickly. 

A CLI keeps things dependency-light and works well for scripts, cron jobs, and demos. Add retries and basic timeouts to handle flaky networks or rate limits.


```ruby
require "json"
require "timeout"
require_relative "indexer"          # your Meili client and search method
require_relative "answer_generator" # the class above

class RagCli
  def initialize
    @indexer  = Indexer.new(index: ENV.fetch("MEILI_INDEX", "docs"))
    @llm      = AnswerGenerator.new
  end

  def run
    puts "RAG on Ruby. Type a question, or 'exit'."
    loop do
      print "> "
      q = STDIN.gets&.strip
      break if q.nil? || q.downcase == "exit"
      next if q.empty?

      hits = safe { @indexer.search(q, limit: 5) } || []
      if hits.empty?
        puts "No context found."
        next
      end

      response = safe { @llm.answer(query: q, hits: hits) }
      puts "\n#{response}\n\n"
      puts "Sources:"
      hits.each_with_index do |h, i|
        id = h["id"] || h.dig("_formatted", "id") || "unknown"
        puts "  [doc:#{i+1}] #{id}"
      end
      puts
    end
  end

  private

  def safe(max_retries: 3)
    tries = 0
    begin
      Timeout.timeout(30) { return yield }
    rescue => e
      tries += 1
      sleep(0.5 * tries)
      retry if tries < max_retries
      warn "Error: #{e.class} #{e.message}"
      nil
    end
  end
end

RagCli.new.run if $PROGRAM_NAME == __FILE__
```


A few production checks go a long way. Ensure API keys are validated upfront and with clear errors when any information is missing. If your provider allows it, stream tokens so users see answers as they’re generated. 

Log the essentials, such as queries, selected document IDs, answers, and citations, for easy debugging and evaluation later. Keep prompts within safe length limits and sanitize inputs to block prompt injection attempts. 

And if you’re exposing the app over HTTP, run it through a small Rack or Sinatra service with sensible timeouts and rate limits to maintain stability.

This completes the core loop: retrieve with Meilisearch, ground the answer with your context, and ship an interface that is easy to run and extend.

## How is RAG in Ruby different from Python?

While Python dominates the RAG ecosystem, Ruby has its own strengths for developers who prefer a leaner, language-level approach. 

The key differences lie in the ecosystem, available tools, and performance characteristics.

![Ruby vs. Python for RAG.png](https://unable-actionable-car.media.strapiapp.com/Ruby_vs_Python_for_RAG_4790deaa9a.png)

Ruby focuses on simplicity and direct integrations rather than building entire RAG stacks internally. 

Python offers more built-in support for machine learning and orchestration, but can feel heavier for smaller apps. 

The right choice depends on your goals, scale, and existing stack.

## Unlocking the power of RAG with Ruby

RAG unlocks new possibilities for Ruby developers who want intelligent, context-aware applications without shifting their entire stack. 

For those working with Rails, our [guide to building RAG apps on Rails](https://www.meilisearch.com/blog/build-rag-apps-on-rails) shows how to extend these ideas further. 

By combining Ruby’s flexibility with the right tools, you can build retrieval and generation pipelines that are both lightweight and powerful.

How to Build RAG Applications Using Ruby: A Developer’s Guide

Today, we’re excited to share that Meilisearch is [live on AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-7ougkqatzyr7i). You can now get Meilisearch with one click directly through your AWS account!

This builds on our existing partnership with AWS. Earlier this year we even showcased Meilisearch at VivaTech, and since then have been expanding our collaboration to make search with AWS even easier!

## Why this matters

- You can use your AWS budget and keep one consolidated bill.
- Move faster with standardized AWS terms and streamlined procurement.
- Ship confidently with enterprise-grade options: SLAs, high availability, multi‑region replication, and deployment in your preferred AWS region.

## What you get with Meilisearch

Meilisearch is developer-first, comes with a simple API, and allows you to leverage search instantly with your existing AWS workflow. We stand out with the following features:

- Unified search, one API: full‑text + vectors + hybrid retrieval.
- Great relevance out of the box: typo tolerance, filters/facets, synonyms, ranking rules.
- Built for application search: ultra‑low latency, instant indexing, and clean SDKs.
- Designed for developers: quick integration, straightforward tuning, and clear docs.

## When to choose Meilisearch vs. OpenSearch

AWS does have their own search offering with the OpenSearch service, but Meilisearch can be the right option when you need:

- User‑facing product/docs/in‑app search with extremely low latency.
- Minimal DevOps and fast onboarding with consistent low‑latency responses.
- Simple, powerful controls for ranking rules, facets, synonyms, and hybrid (keyword + vector) search.

On the other hand, OpenSearch is recommended if you need:

- Log analytics, observability, or heavy aggregations across massive datasets.
- Deep Elasticsearch/OpenSearch DSL compatibility for analytics-style workloads.

**Rule of thumb**: Meilisearch is purpose-built for high-quality, user-facing discovery in applications. OpenSearch fits analytics‑heavy and observability scenarios.

## How to get started on AWS Marketplace

1. [Open the listing and choose a plan](https://aws.amazon.com/marketplace/pp/prodview-7ougkqatzyr7i) (Growth or Dedicated) and a contract term 
2. Subscribe via AWS Marketplace
3. Create your Meilisearch project and generate an API key.
4. Index your data using the REST API or official SDKs (JS, Python, Go, PHP, Ruby, and more).
5. Configure relevance: ranking rules, synonyms, stop‑words, and facets.
6. Plug in your UI and iterate with monitoring and tuning as needed.

## Pricing and procurement notes

- Contract‑based subscription with usage‑based options to help avoid overprovisioning.
- Additional AWS infrastructure charges may apply (see AWS Pricing Calculator as needed).
- Need something tailored? Talk to sales for a customized offer.

## Security, compliance, and support

- AWS Marketplace vetting and standardized procurement processes.
- Enterprise support available: SLAs, Technical Account Management, and shared Slack/Teams channels (details defined per contract).
- See the listing and documentation for region availability, high availability, and replication.


## FAQs

- Can I use my AWS commit? Yes—purchases flow through AWS Marketplace.
- Do I have to manage infrastructure? Meilisearch Cloud handles operations for you.
- How do migrations work from Elasticsearch/OpenSearch/Algolia? Export your data, map fields, and re‑index; we provide guidance and best practices.
- How do I size my deployment? Start with our sizing guidance and talk to sales for a customized recommendation.

## Try Meilisearch on AWS

[Get Meilisearch on AWS Marketplace today](https://aws.amazon.com/marketplace/pp/prodview-7ougkqatzyr7i?sr=0-2): 

Have questions or unique requirements? Talk to sales for a customized offer.

Meilisearch is now available on AWS Marketplace!

How to Build RAG Applications on Rails: Step-by-Step Guide

Here's a recap of everything that happened this month!

---

## 💬 New and improved Geosearch

We introduced a new geo backend to store geojson and filter on polygon in v1.22.0 🐦‍🔥

This make the ```_geojson``` field filterable, when mean you can then send your documents with a ``` _geojson``` field filled with a valid geojson

You can also filter your documents with the new``` _geoPolygon``` filter - meaning you ** draw any polgyon** and get instant search!
## Improved vector storage

We Introduced a new vector store backend for better performance, especially if using the binary quantization. You need to enable the new ```vectorStoreSetting``` experimental feature and change the vectorSetting index setting to "experimental" for the indexes where you want to try the new vector store!

This should give you much more improved performance!
## Now supporting Persian language

With the latest update to [charabia](https://github.com/meilisearch/charabia), we now have Persian language support as well!
## Other news

- Check out the [Experimental voice demo](https://voice.meilisearch.com/) we launched earlier this month
- [We're growing our team](https://www.meilisearch.com/careers) and are looking for talented senior software engineers
 - October 13th will be the start of [launch week](https://www.meilisearch.com/launch-week-q3-2025), where we'll be discussing our newest features!

---

Receive this updates in your inbox by subscribing to our [monthly newsletter](https://dub.sh/meili-newsletter?utm_campaign=newsletter&utm_source=blog), or participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions). 

For anything else, join our developers community on [Discord](https://dub.sh/meili-discord?utm_campaign=newsletter&utm_source=blog&utm_medium=july-25-updates).

Meilisearch September Updates

RAG (retrieval-augmented generation) evaluation is the process of measuring how well a system actually retrieves relevant information to generate accurate answers for the users. It helps improve retrieval logic as well as the response quality.

RAG evaluation is crucial for identifying gaps in the system’s performance and reliability across various use cases.

The most common methods associated with RAG evaluation are human evaluation, reference answer scoring, and retrieval relevance checks.

In addition to this, the most widely used retrieval metrics are Exact Match, F1 score, Recall, Precision, BLEU, and ROUGE. The tools you’ll see used most often in this domain include Meilisearch, LangChain, RAGatouille, and Haystack. 

Now that the stage is set, let’s have a deeper look at what RAG evaluation is and the methods involved.

## What is RAG evaluation?

RAG evaluation is the process that determines how effectively a [RAG system](https://www.meilisearch.com/blog/what-is-rag) retrieves information and then uses that information to generate accurate answers for users. 

You might have noticed that RAG evaluation covers both retrieval and generation. This is to provide a clear view of the system's overall correctness. 

Once you have that, it becomes easier to identify weaknesses, test improvements, and benchmark different models or approaches. 

## Why is RAG evaluation important?

RAG evaluation is important because it helps determine whether your system is actually delivering responses that are relevant and helpful to users. 

Without this kind of evaluation, it's difficult to know if your pipeline is improving or drifting off course.

RAG evaluation can help teams identify and address issues early. This can include spotting hallucinations, irrelevant retrieval, and complex, inaccurate answers: 

* **Reduces hallucination risk:** Evaluation highlights the generation based on weak retrieval, allowing you to correct it immediately.
* **Improves user trust:** This evaluation helps make your RAG system fact-based and transparent, improving user confidence.
* **Enables faster iteration:** With regular evaluation, you can measure impact across changes and optimize performance quickly.

Next, let’s look at the methods used in RAG evaluation. 

## What methods are used in RAG evaluation?

The different methods used in RAG evaluation are shown in the image below:

![RAG Evaluation Methodologies.png](https://unable-actionable-car.media.strapiapp.com/RAG_Evaluation_Methodologies_28942e88c7.png)

These approaches evaluate both the retriever and generator, ensuring no compromise on high-quality outputs at any step.


### End-to-end evaluation

First up is the end-to-end evaluation. It focuses on the overall output of the RAG system. 

The factors under consideration are the usefulness, accuracy, and fluency of the generated response without breaking down the pipeline components. 

While it provides a quick way to compare systems, it doesn’t reveal which part of the pipeline – if any – actually failed.

### Retriever evaluation

Retrieval evaluation, as its name suggests, focuses on whether the retrieval system brings back relevant documents for the query. 

The evaluators can be human or automated. Either way, their function is to judge the quality of the retrieved context using a relevance score. 

You can pinpoint the weakness that led to poor answers downstream. 

### Grounding evaluation

Grounding eval checks if the generated response remains faithful to the retrieved content. 

How do we find that out? The output is compared with the supporting context to check factual accuracy. Naturally, this helps you spot and fix potential hallucinations.


## What are the key RAG evaluation metrics?

RAG evaluation metrics measure the quality of the retrieved documents and the accuracy of the generated response. Developers can use this information to identify where the pipeline is failing and make the necessary corrections.

Some of the key RAG evaluation metrics to know are the following:

* **Precision and recall:** Precision measures the percentage of retrieved docs that are actually relevant, while recall assesses the percentage of relevant documents that were retrieved. You need both to achieve a balance between accuracy and coverage in information retrieval. 

![Precision and recall.png](https://unable-actionable-car.media.strapiapp.com/Precision_and_recall_13f62b830c.png)

* **Groundedness:** This metric evaluates whether the generated output is factually consistent with the retrieved documents. A high groundedness score means the model is less likely to hallucinate.
* **Faithfulness:** Faithfulness measures how well the generated text sticks to the facts in the source material. Unlike groundedness, faithfulness seeks to identify distortions or unsupported claims.
* **Answer relevance:** Is the final answer even relevant to the user’s query? This is determined through answer relevance that relies on human ratings or automated scoring methods.
* **Fluency:** Fluency assesses how natural and readable the generated response is. After all, the target user is human. While secondary to accuracy, fluency impacts user trust and the overall experience.

Together, these metrics serve as essential benchmarks for any RAG applications, providing a complete and honest picture of the RAG system.

## What are the challenges in RAG evaluation?

RAG evaluation has a few unique challenges that we’ll list here:

1. **No universal benchmarks:** Unlike classic machine learning, there is no universal benchmark for RAG evaluation that applies to every [type of RAG system](https://www.meilisearch.com/blog/rag-types), as they all differ in architecture.
2. **Human evaluation costs too much:** Groundedness and relevance often require human judgment, which is expensive and hard to scale.
3. **Dynamic data and drift:** The frequent interaction of RAG pipelines with evolving knowledge bases means that evaluation results soon become outdated.
4. **Trade-offs between metrics:** There’s always a trade-off between the metrics. For instance, optimizing for fluency might reduce faithfulness. Similarly, improving recall could harm precision. 

## How is RAG evaluation implemented?

Different tools, frameworks, and services are responsible for implementing RAG evaluation. The evaluation process is mostly a combination of human review, automated evaluation, and continuous monitoring. 

Most teams prefer to conduct offline testing first and then move on to live monitoring once the pipeline is deployed. 

### Popular RAG evaluation frameworks

Popular open-source frameworks, such as Ragas, TruLens, and DeepEval, are a major help when it comes to automated testing. 

These frameworks use automated tests to help measure retrieval precision and factual consistency, and monitor hallucination rates. Through the LLM-as-a-judge grading they provide, you can easily identify weaknesses and correct them within the system.

For an easily digestible overview of retrieval layers and orchestration platforms, feel free to check out our guide on[ the top RAG tools](https://www.meilisearch.com/blog/rag-tools). 

### Synthetic test datasets

Another common method is creating synthetic evaluation datasets that replicate real-world query patterns. 

How does this help? It allows developers to benchmark retrieval quality and groundedness under a controlled set of conditions. You get to identify coverage gaps before the pipeline enters the production stage.

### Human-in-the-loop validation

There’s no alternative to human insight. Even with strong automation, human evaluation remains critical. 

Subject matter experts review sampled queries to verify if the retrieved content is relevant and the generated answers are accurate. It provides a nuanced view of the system that metrics alone cannot determine.

### Continuous monitoring in production

As a developer, you already know that evaluation does not stop once a system is live. Production monitoring tracks key metrics, including latency, error rates, and feedback loops. 

A reliable retrieval layer, such as Meilisearch, supports this process by consistently delivering fast, high-quality context that evaluation frameworks depend on.

Together, these approaches form a cycle of pre-deployment testing, human review, and real-time monitoring. As a result, the RAG pipelines remain accurate, efficient, and trustworthy over time.

## What are the best practices in RAG evaluation?

Best practices give you the most efficient way to ensure that pipelines are tested thoroughly and fairly. Early weakness detection, hallucination spotting, and maintaining reliability are all part of these best practices.

![Best Practices in RAG Evaluation.png](https://unable-actionable-car.media.strapiapp.com/Best_Practices_in_RAG_Evaluation_06aaf08217.png)

Some of the key best practices include:

* **Use more than one test set:** Evaluate your system with multiple data types and domains, rather than relying solely on a single benchmark. This exposes weaknesses that would otherwise go unnoticed.
* **Combine human and automated checks:** Humans provide context-sensitive judgments, while automated metrics offer speed and scale. A balance between both is your best bet.
* **Test the robustness of the system:** Introduce adversarial or noisy inputs to see how well the system maintains accuracy under less-than-ideal conditions.
* **Track longitudinal performance:** Don’t stop evaluating pipelines. Continue over a staggered time period. This helps identify model drift and measure whether improvements stick.
* **Target user expectations:** Design evaluations that mirror how end-users will interact with the system. Metrics should reflect usefulness, not just technical accuracy.

These practices make RAG evaluation a reliable part of development, ensuring systems remain stable and trustworthy at scale. To explore techniques that can further improve pipeline performance, take a look at our article on[ advanced RAG techniques](https://www.meilisearch.com/blog/rag-techniques).

## How long does it take to evaluate a RAG pipeline?

The answer to this question lies in the variables. Smaller prototypes can be evaluated within hours, while production-scale pipelines require days of rigorous evaluation. 

The duration also depends on factors such as dataset chunk size, evaluation methods, and the number of metrics being tracked. 

For example, using human annotators for quality checks is slower but provides better, more nuanced insights than automated benchmarks. 

The complexity of the pipeline also affects the timeline.

## Are there any papers on RAG evaluation?

Yes, there are several notable papers on RAG evaluation, and most of them can be found on arXiv. Here are some recent ones from 2024 and 2025:

* ‘Evaluation of Retrieval-Augmented Generation: A Survey’ (2024) by Hao Yu et al.
* ‘Evaluating Retrieval Quality in Retrieval-Augmented Generation’ (2024) by A. Salemi et al. 
* ‘VERA: Validation and Evaluation of Retrieval-Augmented Systems’ (August 2024) by Tianyu Ding et al.
* ‘Unanswerability Evaluation for Retrieval-Augmented Generation’ (December 2024) by Xiangyu Peng et al.
* ‘Retrieval Augmented Generation Evaluation in the Era of Large Language Models’ (April 2025) by Aoran Gan et al.
* ‘The Viability of Crowdsourcing for RAG Evaluation’ (April 2025) by Lukas Gienapp et al. 

These papers provide a rich foundation for anyone wanting to understand or benchmark RAG evaluation methods, metrics, and emerging best practices.

## The future belongs to teams that master RAG evaluation

RAG evaluation is not just a technical exercise – it is the feedback loop that keeps RAG systems honest, reliable, and ultimately useful. 

In practice, evaluation becomes the difference between a system that only looks impressive in demos and one that consistently delivers value in production.

As data sources shift, user expectations evolve, and generative models grow more complex, the ability to adapt evaluation strategies will separate cutting-edge teams from those left chasing errors downstream. 

RAG evaluation: Metrics, methodologies, best practices & more

Modular RAG refers to a method of building [retrieval-augmented generation (RAG) systems](https://www.meilisearch.com/blog/what-is-rag) where the entire setup is divided into smaller and independent components. This makes designing, customizing, or upgrading the pieces easier.

The key components of modular RAG are the retriever, which finds useful information, the generator, which turns the information into responses, and additional modules such as rerankers, memory, or vector databases, which improve the accuracy of the artificial intelligence system. Each module handles a specific function that contributes to the overall quality.

Modular RAG works by retrieving relevant context from external knowledge sources, passing it through a generative model, and allowing different modules to handle tasks such as filtering, summarization, or reasoning, depending on the task's requirements.

The key advantages of modular RAG are increased flexibility, easier debugging, and better scalability. The key disadvantages of modular RAG are added complexity in system design and potential challenges in integrating the different modules.

You can implement modular RAG by combining retrievers and LLMs into a RAG pipeline that fits your use case.

This article will discuss everything you should know about the modular RAG framework.

## What is modular RAG?

Modular RAG is a way of building RAG systems that treats each part of the process as separate but connected modules. Instead of having one rigid design, you get to choose the system build that works best.

In modular RAG, you can create a pipeline that adapts to your needs, be it faster retrieval or more accurate answers. Because you are not forced to use the same retrieval method, vector database, or language model every time, you get finer control over how information is retrieved, processed, and turned into answers. 

For example, you might love how one company's embedding model works but dislike its retrieval system. With modular RAG, you can retain the embeddings and change to a different retrieval process. You can even experiment with multiple language models to see the best-performing one for your use case.

Modular RAG is subdivided into three different components:

* The retriever that gathers context
* The generator that creates responses
* Additional modules, such as filters, memory, or rerankers, that refine the output to suit the user’s needs

Now, let’s see the working mechanism behind modular RAG.

## How does modular RAG work?

Modular RAG works by breaking the [RAG process](https://www.meilisearch.com/blog/how-to-build-rag) into separate components, with each component handling a specific task. 

![image5.png](https://unable-actionable-car.media.strapiapp.com/image5_6b10b78e7f.png)

These components can include:

* **Retriever:** The retriever searches a knowledge base or [vector database](https://www.meilisearch.com/blog/what-is-a-vector-database) for relevant information.
* **Reranker or filter:** This is optional but recommended. Before sending the results forward, a reranker or filter can reorder, clean, or narrow down the retrieved data so that only the most useful content is passed along to the next component.
* **Generator:** The generator (typically a large language model such as OpenAI GPT) takes the refined context and uses it to produce a response.
* **Additional modules:** Depending on the use case, additional modules such as memory modules or summarizers can be slotted in to improve accuracy or break down complex queries.
* **Output delivery:** Finally, the GenAI system returns an accurate response that is grounded in the retrieved information.

Let’s see some examples of modular RAG.

## What is an example of modular RAG?

An example of modular RAG in action can be an online shopping assistant. 

Let’s say you search for ‘running shoes under $100.’ The router first decides the best way to handle that query: either through keyword search or by semantic similarity search. 

The retriever then digs through the store’s product database. 

Next, the processing modules clean up the results, filter out any irrelevant items, and then rank the best fits. 

The generator takes that context and produces a useful reply such as, ‘Here are three running shoes under $100 that match your style.’

If the store later wants to add a recommendation setup or customer reviews as part of its process workflow, it can be done without changing the overall setup.

## What are the advantages of modular RAG?

The key advantages of using modular RAG over traditional and one-piece [RAG types](https://www.meilisearch.com/blog/rag-types) are the following:

* **Easy maintenance:** You can fix, upgrade, or replace a single part without worrying that the entire system will break.
* **Faster development:** Different teams can work on separate modules in parallel, allowing for faster deployment and optimization of new features.
* **Easy customization:** Different industries have different needs for modular RAG. A legal assistant may add citation tracking, while an e-commerce chatbot may use recommendation modules. Modular RAG makes it easy to tailor the modules to your needs.
* **Scalable performance:** Modules that handle heavier workflow can be given extra resources, while lighter ones can run with less. 
* **Reliability:** Since the AI system is divided into separate components, weak spots can be easily spotted and fixed, making it more reliable over time.

## What are the disadvantages of modular RAG?

The disadvantages of implementing modular RAG are the following:

* **Higher setup costs:** More modules mean more infrastructure and skilled people are needed to run them, which drives costs up. 
* **Complexity overload:** Managing several modules at once is not easy. There are more parts to manage individually and collectively. Troubleshooting the modules can also involve complex procedures, particularly in identifying the root cause of the issues.
* **Learning curve:** Teams using modular RAG must understand how each module works and fits together, which can slow down onboarding and often requires knowledge of data science and NLP.

## What is the architecture of modular RAG?

The architecture of modular RAG is built around breaking the whole system into smaller parts that work together. Here is how the typical architecture looks:

![image1.png](https://unable-actionable-car.media.strapiapp.com/image1_c115ed35eb.png)

* **Client query interface:** This is where the user’s query first comes in. 
* **Retrieval module:** The query from the client query interface is sent to the retrieval module. This module includes a vector search store (used for semantic search), a document store (used for traditional text search), and a cache layer (used to quickly retrieve recent or repeated queries). Together, they act like a library housing the important information.
* **Reasoning module:** Here, the context processor organizes what was retrieved, the inference engine helps the system reason through it, and the state manager keeps track of any history or ongoing context.
* **Generation module:** This is the last phase. The LLM generates a draft response, the template engine structures it neatly, and the output validator checks it for accuracy before delivering the final response to the user.

The architecture of modular RAG makes it easy to use for domain-specific use cases, such as customer support or enterprise knowledge graphs.

## How can modular RAG be implemented?

Modular RAG can be implemented by combining several independent pillars. Each pillar addresses a different aspect of the system and can work on its own or alongside the others. 

Let’s look at these pillars:  

### 1. Data pipeline

The data pipeline handles how datasets are ingested and processed. You can create multiple data pipelines based on the type of content. 

For example, an ETL pipeline that implements modular RAG for customer support documents and another that handles technical manuals. Each use case has its processing pipeline that knows how to handle that specific content. 

Techniques such as chunking and carefully selecting the chunk size can significantly improve retrieval quality.

### 2. Microservices 

The system is divided into independent services that communicate with each other via API calls. Your ‘retrieval’ service can handle document searches, while the ‘reasoning’ service processes the logic, and your ‘generation’ service crafts the final response.

[Meilisearch](https://www.meilisearch.com/docs) can be used as the retrieval service because it can quickly index and retrieve documents without heavy infrastructure overhead.

### 3. Routing layer

The routing layer directs queries based on rules or classifiers and decides which module or pipeline should handle user queries using retrieval strategies or classifiers. You can use simple keyword matching for simple queries, but machine learning classifiers are better for complex queries. 

### 4. Containers for technical implementation

Containers for technical implementation manage how services are packaged and deployed. They make it easy to scale specific modules up or down, swap one out if needed, or update a service without breaking the whole setup.

### 5. Monitoring and orchestration tools

Monitoring and orchestration tools ensure smooth coordination, error handling, and balancing of generation processes. They track how each module performs, catch errors before users find them, and coordinate complex queries that need multiple modules to work together.

## How can modular RAG be evaluated?

Evaluating modular RAG requires tracking metrics across modules:  

* Information retrieval quality via precision and recall.  
* Post-retrieval validation to ensure the correct retrieved documents.  
* End-to-end evaluation of speed and quality, compared against baseline pipelines.

The most comprehensive setup is a ‘sandboxed RAG playground,’ which includes static data, swappable modules, and an LLM to evaluate answers across different configurations.

You want to measure both individual module performance and overall system effectiveness. 

For individual modules, you can track metrics like retrieval accuracy, response time, and resource usage. Each specialized module should behave like an expert in its field, meaning your legal module should confidently handle legal queries, while the technical one should handle engineering problems perfectly.

End-to-end [RAG evaluation](https://www.meilisearch.com/blog/rag-evaluation) matters just as much as module benchmarks. Run complex queries and observe how well the modules work with each other. Check if the system returns an accurate answer. 

In addition, compare the overall quality of information retrieval against a single-pipeline baseline to determine if modular RAG is an improvement.

Another consideration is comparing latency and quality trade-offs. Modular systems often introduce increased latency due to the coordination overhead, so you will need to balance response speed with answer quality.

Real-world testing with iterative feedback loops and augmentation scenarios is great for evaluating modular RAG. It is better to use actual user queries on your system than synthetic test cases. 

Also, pay attention to edge cases where routing gets tricky or where the modules need to collaborate.

## How does modular RAG compare to naive RAG?

Modular RAG uses multiple specialized components that capture domain-specific details and deliver more accurate results, while naive RAG relies on a single retrieval system that may overlook nuances. 

Modular RAG is like a team of specialists, while naive RAG is more like relying on a generalist.

Compared to naive RAG, modular RAG is more powerful for complex queries, though it requires more effort to build and maintain. 

Naive RAG, on the other hand, is quick and straightforward. Your query goes into one retrieval step, documents come back, and an answer is generated.

## How does modular RAG compare to advanced RAG?

Modular RAG can be seen as an evolution of advanced RAG techniques, taking the same core idea but pushing it further by emphasizing modularity, scalability, and flexibility. 

Advanced RAG represents an improvement over naive RAG by incorporating smarter algorithms, enhanced reranking, or multiple retrieval steps. But it is still often built as one big pipeline. 

Modular RAG breaks this apart into smaller, specialized modules that can be swapped, upgraded, or scaled independently.

Both approaches are powerful, but modular RAG shines when you want control, scalability, and the freedom to adjust as your needs evolve.

## Are there papers on modular RAG?

Yes, there are different papers on modular RAG:

1. ‘Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks,’ published July 2024 by Yunfan Gao, Yun Xiong, Meng Wang, and Haofen Wang (arXiv). It introduces the modular RAG paradigm and lays out a flexible, multi-module architecture that evolves beyond the classic linear RAG flow. 
2. ‘Retrieval-Augmented Generation for Large Language Models: A Survey’ by Gao *et al.,* published December 2023 on arXiv. It covers the broader RAG evolution, including the various modular approaches.
3. ‘RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing’ by Hu and Lu, published April 2024 on arXiv. It examines modular components and their interactions. 
4. ‘A Survey on RAG Meets LLMs: Towards Retrieval-Augmented Large Language Models’ by Ding *et al.,* published May 2024 on arXiv. It explores modular architectures and training strategies.


## Start building intelligent RAG systems with Meilisearch

If you are looking to build smarter RAG systems, Meilisearch is your go-to tool. 

It is easy to set up, meaning you can focus more on building intelligent retrieval workflows rather than battling with designing RAG infrastructure. 

Meilisearch also shines at delivering highly relevant documents with low latency, making it ideal for feeding accurate context into your generative AI models.

Meilisearch latest news and company updates