When working with large language models, achieving accurate, relevant results depends on several factors. 

Two of them are Retrieval-Augmented Generation (RAG) and prompt engineering. 

RAG helps generate better answers by allowing the generative AI model to retrieve information from external knowledge sources, keeping responses fact-based and up to date. 

Prompt engineering, on the other hand, focuses on how you communicate with the AI to produce clear and context-aware answers. 

* RAG adds new knowledge without retraining, whereas prompt engineering improves the model’s use of its existing knowledge.
* RAG works well for research, legal, healthcare, and customer support; prompt engineering is well-suited for summarization, creative writing, Q&A, and data formatting.
* RAG is complex and can be slower, while prompt engineering is cheaper but requires skill.
* RAG requires more setup and maintenance, but prompt engineering is faster and cost-effective.

Choosing the right approach depends on what you want: accuracy, freshness, or speed. Often, combining both gives the best results. Let’s dig deeper into this. 

## What is prompt engineering?

Prompt engineering is the art of crafting clear instructions for large language models (LLMs) to achieve the best possible results. 

Think of it like learning to ask the right questions. A vague prompt might produce a generic answer, while a carefully worded one can give you precisely what you need. 

Prompt engineering is about understanding how the gen AI model interprets instructions and using that insight to guide it toward your desired output. 

## What is RAG (retrieval-augmented generation)?

[RAG](https://www.meilisearch.com/blog/what-is-rag) is a technique that improves LLM outputs by providing them with access to external resources. This could include databases, documents, and APIs. Instead of relying solely on what they were trained on, RAG allows AI models to access additional information when generating answers. 

For instance, we can give an AI model both memory and internet search skills, so that it knows today’s date and searches the internet for the most recent data.

## How does prompt engineering work?

Prompt engineering is about learning how to interact with an artificial intelligence model in a way that generates the best possible response.

As a rule of thumb, it is good practice to provide the LLM with enough context to answer your questions. LLMs are trained to follow user instructions, and they do so within the context they have. 

For example, if you ask an LLM to act like a hip-hop artist, its response will sound like one. Ask it to write like a Nobel Prize–winning author, and it will adapt its responses accordingly.

Finding the right prompt involves experimentation. If the answer from an initial prompt is not accurate, you can refine your prompt to provide more context and make it clearer. You may also check for noise or contradictory statements and eliminate them. 

For example, if you do not have a background in science, instead of asking the model to “Explain photosynthesis,” you could say, “You have over a decade of experience teaching biology. Explain photosynthesis like you are teaching someone in eleventh grade.” The goal is to give the AI enough context and direction to produce relevant responses.

## How does RAG work?

RAG works by combining retrieval and generation so that an LLM can answer using external facts. 

In the retrieval phase, RAG searches external sources to find the most relevant information, and in the generation phase, the model uses that retrieved data to craft an accurate response. 

![How RAG Works.png](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_e26007ec0c.png)

Here is how the process goes, step-by-step:

1. A user types in a search query. 
2. A retriever searches a document store or database for information that matches the query. 
3. The information is ranked, and the best is selected to form a context. 
4. The original question and the context are sent to the LLM. 
5. The model generates a response grounded in the retrieved information. 
6. Finally, a verifier or re-ranker checks the model's outputs before they are returned. This is optional.

## What are the main differences between RAG and prompt engineering?

The main difference between RAG and prompt engineering is how they help AI models produce better answers. 

Different [RAG types](https://www.meilisearch.com/blog/rag-types) improve accuracy by allowing the model to retrieve real-time data before generating a response. 

Prompt engineering focuses on phrasing and optimizing questions to get the best result from the model’s existing knowledge. 

To sum it up, RAG gets its information externally, while prompt engineering uses information already within the AI model.

![RAG vs. Prompt Engineering.png](https://unable-actionable-car.media.strapiapp.com/RAG_vs_Prompt_Engineering_a00885536d.png)

Now, let’s discuss the benefits and challenges of using RAG and prompt engineering.

## What are the advantages of prompt engineering?

Prompt engineering can help users get the most out of AI tools without requiring additional data. 

Here are some of the advantages:

* **Better control over responses:** You can guide the AI model to produce more relevant outputs by learning how to phrase questions effectively. For example, a marketer can ask ChatGPT to ‘act like an expert marketer in a Fortune 500 company and write an email to promote a new product,’ and receive a good draft email immediately.
* **Works across different tasks:** Prompt engineering can be applied across various sectors. It is helpful for specific tasks such as coding, summarizing reports, writing essays, and even brainstorming ideas. 
* **Saves time and effort:** Making your input prompts clearer reduces the need for multiple iterations. Instead of constantly reworking the answer, a well-crafted prompt helps the AI deliver the desired result in a single go.
* **Encourages creativity and personalization:** Prompt engineering gives users the freedom to set their own tone or style. This makes it an excellent tool for writers, designers, or educators who desire unique results.


## What are the advantages of RAG?

RAG has several benefits that make AI responses more helpful in real-life situations.

* **Improves accuracy:** RAG reduces the risk of errors by pulling information only from verified external sources.
* **Access to up-to-date information:** RAG can retrieve current information from live databases or documents. This makes it perfect for fast-changing topics, such as the news.
* **Better context understanding:** RAG provides the AI model with additional background information before generating an answer, helping it better understand the question and respond more accurately.
* **Flexibility across domains:** Because it can connect to various data sources, RAG can easily adapt to resource-intensive, specialized fields like healthcare, agriculture, or finance while still delivering insights tailored to each domain.
* **Reduced AI model training costs:** Since RAG relies on retrieval rather than constant retraining, organizations can update knowledge from data sources rather than rebuilding the entire model for each process.

## What are common challenges with prompt engineering?

Prompt engineering is a powerful skill, but it is not always straightforward. Generating the perfect response from AI often takes creativity and practice. 

Here are some common challenges:

* **Inconsistency in results:** Just a slight change in wording can completely alter the AI’s answer. Also, because AI is non-deterministic, what works once may not work the same way next time, making it hard to achieve consistent responses.
* **A need for expertise:** Crafting a strong prompt requires some level of domain knowledge and an understanding of how the AI model interprets instructions. Newbies may give generic prompts, leading to generic AI responses.
* **Can be time-consuming:** Refining your prompts through trial and error can take a long time, especially when dealing with complex tasks. If not properly managed, this can slow down your workflow or even reduce your productivity.
* **Model limitations:** AI systems only know the training data on which they were built. Therefore, even the best prompt cannot fix a model whose knowledge base is lacking. Such situations lead to hallucinations or outdated responses.

## What are the common challenges with RAG?

Some challenges can make RAG implementation difficult, even though it greatly improves accuracy and relevance. 

Here are some of them:

* **System complexity:** RAG can be complex to set up and maintain, especially for non-technical teams. It requires [connecting a retriever and a database to the LLM](https://www.meilisearch.com/blog/how-do-you-search-in-a-database-with-llm).
* **Integration overhead:** Integrating a RAG system into an existing application or workflow often requires extra resources. You have to set up the RAG infrastructure while maintaining the external data sources at the same time.
* **Latency and speed issues:** Balancing accuracy with performance can be challenging with RAG. Since it searches and retrieves data before generating a response, it can be slower than the traditional models, especially when handling large datasets. 
* **Information retrieval error:** Sometimes a RAG system retrieves the wrong documents, leading to answers that sound confident but are based on inaccurate data. This makes proper data indexing and filtering critical when setting up the RAG system.

## What are examples of prompt engineering use cases?

Prompt engineering can be applied in many real-world situations to make AI tools more valuable and efficient: 

* **Question answering:** Businesses use prompt engineering to make chatbots and help desks smarter. With the right prompt, an AI agent can provide clear and accurate responses to customer or employee questions.
* **Text summarization:** It is excellent for turning long text into short but meaningful takeaways. For instance, you can say, ‘Summarize this report in three simple points,’ and save hours of reading time.
* **Data conversion:** Prompt engineering can help you quickly clean or restructure information. For example, changing your raw notes into a professional report.
* **Creative writing:** Writers use prompts to get innovative ideas, explore storylines, or overcome creative block.

## What are examples of RAG applications?

[RAG tools](https://www.meilisearch.com/blog/rag-tools) are powering many of the apps we use today. Here are a few ways RAG is being used in the real world:

* **Customer support systems:** Many customer service chatbots now use RAG to fetch real answers from company manuals. So when you ask your bank’s chatbot about your account details, it gives you the exact information instead of a generic reply.
* **Legal research:** Law firms use RAG to review thousands of case files and contracts in seconds. A lawyer can ask about precedents for a case and instantly get relevant examples instead of manually searching the archives.
* **Healthcare and medical support:** Doctors and researchers use RAG to find the latest studies, treatment guidelines, and patient data. This enables them to make more informed medical decisions.
* **E-commerce product comparison:** RAG is used by e-commerce companies to compare products. It retrieves product specs, reviews, or prices from the database and generates personalized recommendations.

## Can RAG and prompt engineering be combined?

Yes, RAG and prompt engineering work better together. 

For example, when someone asks, ‘Summarize this customer’s issue in three bullet points,’ a RAG setup might pull a customer’s record from the database, and a good prompt allows the LLM to summarize only the issues returned. 

In legal work, RAG can find relevant cases, which are then combined with a prompt that says, ‘Compare these rulings and highlight the key differences.’ This provides a concise, actionable comparison rather than a long list that might be difficult to interpret. 

In healthcare, RAG retrieves patient history and recent medical studies, while prompts ensure the results follow clinical reporting standards.

RAG handles the ‘what’ (or the facts), and prompt engineering handles the ‘how’ (or the presentation). When used together, they make answers both relevant and accurate.

## What are the costs of RAG vs. prompt engineering?

The costs of setting up RAG and prompt engineering are different. 

RAG can be expensive to implement because it requires additional infrastructure (such as a database, a retrieval system, and integration tools). There is also a maintenance cost for updating data sources, handling storage, and ensuring the retrieval system stays accurate and fast. 

Prompt engineering, on the other hand, is far cheaper because it mainly involves using your time and skill. You have to learn how to design effective prompts and also test them for consistent results. There is no need for new servers, APIs, or databases, making it suitable for quick, low-cost projects.

RAG requires more upfront investment but delivers higher accuracy, while prompt engineering is cost-effective and easily scalable for smaller applications.

Want to decide whether to use RAG or prompt engineering? Let’s see the factors you should consider when choosing.

## How do you choose between RAG and prompt engineering?

Choosing between RAG and prompt engineering depends on what you need from your system and the kind of resources you have. 

Here is a simple decision guide to use:

* **Check if you need fresh data:** RAG is the way to go if your answers must include the latest information. But if you are working with general knowledge that doesn't change often, prompt engineering should be sufficient.
* **Weigh your accuracy and reliability needs:** For fields like healthcare, law, or finance, where mistakes can be costly, RAG is safer because it cites real sources. Prompts alone cannot always guarantee accuracy. 
* **Consider your time, budget, and skills:** RAG takes more setup and maintenance, while prompt engineering requires creative thinking and practice. If you have limited resources, then go with prompt engineering.
* **Balancing performance and workflow:** RAG systems can be slightly slower since they search external data before responding, but the answers are usually more accurate. Prompt engineering is quicker and is perfect for lightweight applications where speed matters more than depth.

## How does RAG compare to fine-tuning?

The main difference between [RAG and fine-tuning](https://www.meilisearch.com/blog/fine-tuning-vs-rag) is that RAG pulls in fresh information from external sources when needed, while fine-tuning actually trains the model to learn that knowledge internally.

With fine-tuning, you take a pre-trained model and feed it domain-specific data (such as legal documents or customer records) so that it learns the patterns and language of that field. This makes it faster and consistent. 

The limitations are that it is expensive to train, requires lots of clean data, and must be redone whenever new information comes in.

RAG, on the other hand, does not retrain the model. Instead, it connects to a database or document store and retrieves the most relevant information whenever a question is asked. This ensures the answers are up to date and adaptable to new topics.

## How does prompt engineering compare to fine-tuning?

The main difference between prompt engineering and fine-tuning is that prompt engineering focuses on how you ask the model your question to get the best response, while fine-tuning changes what* *the model actually knows.

With prompt engineering, you are not retraining anything. You are instead crafting the proper instructions to guide the model’s behavior. It is fast, affordable, and flexible, making it perfect for tailoring outputs to specific needs. 

The catch is that results can vary greatly depending on how well the prompts were written.

Fine-tuning, on the other hand, takes things deeper. You train the model on new data so it permanently learns that knowledge. This gives you consistency and expertise but comes with higher costs, more setup time, and the need to continuously retrain to stay up to date.

## RAG vs. prompt engineering: and the winner is...

The reality is that both techniques shine in their own way. 

RAG is your go-to tool when accuracy and up-to-date information matter most, since it can retrieve from live data sources. Prompt engineering is the fastest, most flexible way to generate outputs without retraining any model.

If you are building something that needs freshness, go with RAG. For quick, creative control, prompt engineering takes the lead. 

RAG vs. prompt engineering: choosing the right approach for you

Algolia vs Elasticsearch (vs Meilisearch): Which search solution actually fits your needs in 2025?

GraphRAG vs. Vector RAG: Side-by-side comparison guide

Another [Launch Week](https://www.meilisearch.com/launch-week-q3-2025) is in the books. We shipped five major updates this week, designed to solve real problems our users told us they were struggling with. Each one of them makes your search experience more powerful, more flexible, and easier to work with. 

Here's what we built:
##  Meilisearch Chat*: we brought conversational AI to Meilisearch

Search is becoming a conversation. Your users no longer want to scroll through results - they want to ask a question and get an answer. But building conversational AI has meant juggling LLM calls, vector databases, reranking logic, and weeks of infrastructure work. 

Our new `/chat` endpoint brings complete RAG capabilities directly to your Meilisearch index. One API call handles everything: understanding the query, retrieving relevant data, and generating accurate responses. 

Key features:

- **Conversational AI for search:** your users can ask natural-language questions and get precise, contextual answers directly from your Meilisearch index.
- **Built-in RAG capabilities:** retrieves relevant information, contextualizes it, and generates accurate responses - no separate vector database, reranker, or AI service required.
- **Native integration:** runs directly on your Meilisearch instance as a single API endpoint.

_*This feature is in open beta._

[See the demo → ](https://supportbot.meilisearch.com/)

[Explore the docs → ](https://www.meilisearch.com/docs/learn/chat/getting_started_with_chat)

[Read the full announcement and start building with `/chat` →](https://www.meilisearch.com/blog/meilisearch-chat)

##  Advanced geosearch

Location-based search is powerful, but until now, Meilisearch only supported [basic shapes](https://www.meilisearch.com/blog/powering-bildhistoria-discovery) and points on the map. If your use case involved complex shapes, delivery zones, city boundaries, or irregular service areas, you had to work around it.

We've added full GeoJSON support with polygon and custom shape filtering. It allows to define a complex geographic boundary and search within it with precision. It's built on the same fast, filterable foundation you're used to, just way more flexible.

Key features:

- **Advanced shapes search:** search within polygons, multi-points, and complex shapes
- **Fine-grained filtering:** mix geo queries with text relevance and filters
- **Faster performance:** optimized for real-time map and delivery use cases

[See the demo → ](https://chantiers-paris.vercel.app/)

[Explore geosearch with polygons in the docs →](https://docs.meilisearch.com/)

##  Enhanced multi-modal: integrated image search

Image search has been possible in Meilisearch, but it required you to generate embeddings externally and inject them manually. For a lot of teams, that friction was enough to shelve the idea entirely.

Now it's built in. Provide images in base64 format, select an embedder (like CLIP), and Meilisearch handles the rest-embedding, indexing, and search. Support text-to-image queries, image-to-image queries, or both. Whether you're building [product catalogs](https://www.meilisearch.com/blog/Agora), [media libraries](https://www.meilisearch.com/blog/motionelements), or visual discovery tools, you can ship image search without the setup headache.

Key features:

- **Integrated support:** image-to-image and text-to-image search now built directly into Meilisearch
- **Multi-modal queries:** mix image, text, and embedding inputs seamlessly
- **Simple setup:** everything runs through a single API and index with flexible configuration

[See the demo → ](https://multi-modal.meilisearch.com/)

[Explore how multi-modal search works in our docs →](https://www.meilisearch.com/docs/learn/ai_powered_search/image_search_with_multimodal_embeddings)

##  Search personalization: results that adapt to each user

Not everyone searches the same way. A user shopping for running shoes shouldn't see the same results as someone looking for dress boots - but most search engines treat every query identically.

Search personalization lets you rerank results based on user context: preferences, behavior, past interactions. Powered by Cohere Rerank 3.5, it works with a simple `personalize` parameter. You provide the user context, Meilisearch adjusts the ranking, and your users get results that actually match their intent.

Key features:

- **Personalization API:** individualize search results based on behavior
- **Cohere 3.5 embeddings:** accurate, privacy-safe personalization
- **Lightweight integration:** add personalization without reindexing

[See the demo →](https://personalization.meilisearch.com/) 

[Talk to our team to get access to personalized search →](https://meet.meilisearch.com/meetings/cloud/presentation)

##  Resource-based pricing: predictable costs that scale with you

Usage-based pricing works when your workload is unpredictable. But as you scale, you want to know exactly what you're paying for: compute, memory, and storage, ideally, without surprise bills tied to search volume.

We've introduced resource-based pricing alongside our existing usage-based model. Choose your dedicated resources (CPU and RAM) from predefined tiers, and your costs stay predictable as you grow. You can see real-time resource usage, upgrade when you need to, and only pay for what you allocate. 

Key features:

- **Resource-based Pricing:** transparent, usage-based costs
- **Predictable scaling:** pay as you grow with your data
- **Simple migration:** no changes needed to your setup

[Learn about resource-based pricing  →](https://www.meilisearch.com/blog/resource-based-pricing) 

[Get started now →](https://www.meilisearch.com/cloud)

---

## What's next

These releases aren't just features, they're the foundation for where we're taking Meilisearch. Conversational AI, multi-modal search, and personalization: it's all pointing toward search that understands context, not just keywords.

If you're building with Meilisearch or thinking about it, now's a good time to dig in. We're here if you need help.

That's a wrap: Meilisearch Launch Week, October 13-17th 2025

If pricing makes you nervous, you build less.

We’ve seen that first‑hand. When your bill swings with every spike in documents or searches, teams hesitate to ship, and finance can’t forecast.

Our philosophy towards building Meilisearch – from our [dual licensing model](https://www.meilisearch.com/blog/enterprise-license) to how we price – has always centered on giving our customers maximum flexibility.

That’s why we’re introducing resource-based pricing: a developer-first model that stabilizes spend, aligns pricing with actual infrastructure, and provides clear controls over your performance and infrastructure.

## Why we’re adding another pricing model

For a while, usage‑based pricing looked like the obvious path. It matched the serverless zeitgeist and mapped neatly to “documents” and “searches.” But reality didn’t fit the abstraction:

- **Unpredictability drove churn**. Teams with a large amount of data and relatively few searches were penalized. Bills rose in ways that were hard to anticipate.
- **Identical “units” did not have identical costs**. A million tiny product records and a hundred thousand large PDFs do not consume the same resources – yet usage pricing treats them the same.
- **We were over‑optimizing for plan limits**. Internally, we found ourselves hand‑tuning infrastructure to fit plan boxes rather than letting customers scale the resources they needed.

Customers (especially developers and CTOs) told us what they wanted instead: price stability, performance control, and transparency on what drives cost.

## What “resource‑based” means

With resource‑based pricing, you select dedicated compute for your project – and you pay **only** for those resources.

![Resource-based pricing](https://unable-actionable-car.media.strapiapp.com/resource_based_pricing_414c8e98aa.png)

- **Compute**: choose the CPU and RAM you want. Those resources are dedicated to your project via predefined tiers (e.g., XS through 4XL self‑serve; larger tiers via Sales).
- **Storage**: storage auto‑scales and isn’t customizable at the moment.
- **No caps on “documents” or “searches.”** Your resources define your limits and performance – scale them up or down as your needs change.

You can upgrade resources self‑serve. At the moment, downgrades are available via support, but very soon you’ll be able to do them from the Cloud interface.

Billing is at the team level. You can run mixed billing in the same invoice (usage‑based projects alongside resource‑based ones), and you can upgrade/downgrade resources as you go (downgrades via support for the moment, as mentioned above).

This mechanism is the same for both self-serve and enterprise. The Enterprise level then adds SLAs, SSO, support/SLOs, volume discounts, and optional add‑ons – not different machines.

Why this is better for teams needing to scale:

- **Predictability**: stabilize spend as you grow – no more surprise overages tied to abstract units.
- **Control**: developers can dial resources to match performance goals – whether that’s sub‑second search or faster indexing.
- **Transparency**: your bill tracks real cost drivers (CPU, RAM, storage), making budgeting (and explaining costs) far easier.

It also improves activation. If indexing or search feels slow, you can bump resources for a few cents per hour to validate your use case, then tune back once you’ve optimized.

## The developer experience: monitor, tune, repeat

Alongside the pricing model, we’re rolling out a new **Infrastructure** tab in the [Cloud UI](https://cloud.meilisearch.com/) so you can see what’s happening under the hood:

![The new Infrastructure tab allows you to monitor resources](https://unable-actionable-car.media.strapiapp.com/infrastructure_tab_278a604bfd.png)

The new interface allows you to monitor CPU, memory, and storage usage live and take action proactively to avoid performance issues.

Coming soon are also alerts and recommendations. You’ll be able to set up proactive alerts when resource usage reaches certain thresholds (e.g., 80/90/95%) to avoid outages and slow performance.

Wherever possible, we’ll suggest optimizations first (configuration and data‑model tips), then scaling guidance if you still need more headroom.

## How does this fit with usage‑based (we’re keeping both)

This isn’t an either/or ideology. It’s a toolbox:

- Start with usage‑based if you want near‑zero friction and the simplest mental model for a quick evaluation.
- Switch to resource‑based when predictable spend and performance control become more important than per‑unit simplicity.

We’re not forcing migrations. Existing plans are grandfathered. Going forward, when you create a project, you’ll be able to choose usage‑based or resource‑based in the flow – and mix them across a team as your portfolio grows.

## Add‑ons and future flexibility

Some capabilities sit above pricing mode:

- Team‑level add‑ons: SSO, SOC 2, analytics, merchandising
- AI services as first‑class add‑ons on a single Meilisearch bill: embeddings (by size), LLM tokens, re‑ranking models
- Horizontal scale via sharding (multiple machines per project) for advanced workloads

Bandwidth billing and alerting are planned future improvements.

## What to expect at launch

- Fully functional backend: applicable to any new or existing project; resource upgrades are self‑serve; downgrades via support
- Team‑level billing with mixed cycles: usage‑based and resource‑based projects on one invoice
- New Infrastructure tab in the dashboard to monitor CPU, memory, and storage
- Self‑serve and enterprise both supported; enterprise layers on support and governance – not different pricing mechanics

We are confident this new pricing model will lead to lower churn and faster activation because customers can plan with confidence and validate performance without friction.

## Choosing the right model (quick guide)

Choose usage‑based if:

- Business stakeholders lead the evaluation
- Traffic is modest and bursty
- You want the simplest on‑ramp

Choose resource‑based if:

- You’re a developer/CTO who needs predictable spend
- You want explicit control over performance via CPU/RAM
- Your data shape doesn’t map cleanly to “documents/searches”

## How to try it

- New projects: pick “Resource‑based” when creating a project.
- Existing projects: talk to our support team to switch to resource‑based pricing.
- Then, visit the Infrastructure tab to monitor resource usage and adjust CPU/RAM as needed. If you want help optimizing before scaling up, talk to us – we’re happy to review settings and tradeoffs with you.

<Callout intent="note">

Free trials are only offered for usage-based plans, if you want to start a resource-based project, you’ll need to purchase a subscription from the get-go. Of course, you can also start a trial on usage-based to verfy that Meilisearch is the right fit for you and then talk to Support to get migrated to a resource-based plan.

</Callout>

## Our goal

Pricing should reflect reality, not get in your way. Resource‑based pricing puts control back in developers’ hands and makes your bill as predictable as your architecture.

I’m excited to see what you build when pricing is finally one of the simplest aspects of running a search engine.

Scale with confidence: Introducing resource‑based pricing

We're kicking off [Launch Week](https://www.meilisearch.com/launch-week-q3-2025) with something we've been excited to share: Meilisearch Chat, a new way to make your data conversational.

For the first time, you can turn your existing Meilisearch index into an AI-powered chat experience – without complex infrastructure, separate vector databases, or months of custom development.

## Search is becoming a conversation

Your users and buyers don't want to search anymore – they want to ask. They expect apps, documentation, and internal tools to understand natural language and give them direct answers, not endless lists of results.

Until now, that's been really hard to deliver. Traditional conversational AI required juggling multiple LLM calls, vector databases, reranking pipelines, and endless tuning. The complexity alone made it expensive and slow, often taking months just to get a working prototype.

Meilisearch Chat changes all of that.

## One API call, complete conversational AI

The new `/chat` brings AI-native search to your Meilisearch data through a single API endpoint. It handles the entire Retrieval-Augmented Generation (RAG) workflow, from understanding the query to generating the response, automatically.

Here's what you get:

**Conversational AI for search** – Users ask natural-language questions and get precise, contextual answers from your Meilisearch index.

**Built-in RAG capabilities** – Retrieves relevant information, contextualizes it in your factual data, and generates accurate responses. No extra vector database, reranker, or separate AI service required.

**Native integration** – Runs directly on your Meilisearch instance with a single, OpenAI-compatible endpoint.

Less infrastructure. Faster implementation. No compromise on quality or accuracy.

## Why this matters for your business

**Get to market faster**

Meilisearch Chat turns months of AI integration work into days. You can prototype conversational experiences immediately, test them with real users, and iterate quickly without needing a dedicated AI team.

**Lower costs, less complexity**

We've streamlined the entire RAG pipeline, so you don't need separate LLM orchestration, reranking logic, or vector storage. Fewer moving parts means lower maintenance costs and less things that break.

**Better user experiences**

Whether it's customer support or internal knowledge bases, your users get direct, reliable answers in context – not just a list of links. That means faster resolution times, higher engagement, and happier users.

## Built for real-world use cases

**Retail & e-commerce** – Help shoppers find exactly what they need through natural conversation.

**Customer support** – Let users ask questions in plain language and get answers directly from your documentation or help center.

**Internal knowledge bases** – Turn your company documentation into a chat assistant that answers questions instantly.

**Developer docs** – Add conversational search to your API or SDK documentation so developers can find what they need faster.

## Simple to implement, easy to scale

If you're already using Meilisearch, adding conversational capabilities is straightforward. The `/chat` endpoint is [OpenAI-compatible](https://www.meilisearch.com/docs/learn/chat/getting_started_with_chat#configure-a-chat-completions-workspace), which means it works with your existing LLM setup and fits right into your current workflows.

You can start with your existing indexes – no need to rebuild or re-embed your data.

## A new approach to AI-powered search

With Meilisearch Chat, we're making it simpler to bring conversational AI to your data. Instead of managing multiple services and complex pipelines, you get a complete conversational search experience through one endpoint.

This isn't just another feature – it's a new foundation for more natural, human-like interactions with your data.

**Ready to make your search conversational? Explore the [docs](https://www.meilisearch.com/docs/learn/chat/getting_started_with_chat) for how to get started and [try /chat today](https://www.meilisearch.com/cloud)**

Introducing Meilisearch Chat: conversational search for your data

When building AI systems, one of the most significant decisions to make is how your model accesses and uses knowledge. Do you need real-time data every time, or can you rely on pre-stored, cached information? 

This is where the difference between retrieval-augmented generation (RAG) and cache-augmented generation (CAG) lies: 

* RAG retrieves live and updated data, making it ideal for situations where changes occur quickly and accuracy is crucial.
* CAG relies on cached information, delivering answers faster and more cheaply. It works well for repeated or predictable questions.
* RAG provides fresh, reliable responses and can handle various queries. However, it can be slow, expensive, and requires a solid infrastructure to run smoothly.
* CAG is quick, efficient, and easy to scale, but it can get outdated. It is also limited to what is already stored and requires regular updates to stay useful.

Choosing between the two comes down to considering your users: 

* Are the questions predictable or wide-ranging? 
* How up-to-date does the information need to be?
* What is your budget and infrastructure like?

You do not always have to pick just one. Hybrid approaches can combine both RAG and CAG. They allow the speed of cached answers while still retrieving live data when needed.

By the end of this article, you will understand how RAG and CAG work and which one better suits your projects.

## What is RAG (Retrieval-Augmented Generation)?

[RAG is a technique](https://www.meilisearch.com/blog/what-is-rag) that enhances the capabilities of generative AI by using external knowledge to provide more accurate responses to user queries. 

Rather than solely relying on the datasets the AI model was trained on, RAG searches a vector database or docs, retrieves the most relevant information, and then uses it to generate a response. This makes the answers more accurate and up-to-date. 

## What is CAG (Cache-Augmented Generation)?

CAG builds responses around the specific context window you provide. 

Unlike RAG, which searches external databases for information, CAG uses past interactions to shape its answers. This makes its output feel more relevant and personalized. 

In short, CAG makes AI less generic and more tuned to your needs by getting the proper context before generating a response.

## How does RAG work in AI models?

RAG combines two steps: retrieving information and generating a response. 

When you ask a question, the model first searches through an external knowledge base to find the most relevant documents. This retrieved information acts like notes or references that the large language model (LLM) keeps on hand for response generation. 

![A graphic depicting how RAG works.](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_8e6de82c99.png)

When generating its response, the LLM uses both internal knowledge obtained during training and the newly retrieved relevant context to return a response. It makes the response sound natural and grounded in real data. 

RAG co/mbines the best of both search engines and language models, ensuring the AI system is less prone to hallucinations.

Now, let’s take a closer look at CAG and how it works.

## How does CAG function in AI systems?

CAG works by providing AI with the necessary background information to generate an accurate response. When you interact with a CAG-enabled system, it does not rely on general knowledge alone. 

Instead, it takes note of the context you provide, such as your past conversations or personal preferences. This information is cached temporarily using a key-value (KV) cache and can be referenced while generating the responses.

![How CAG Works.png](https://unable-actionable-car.media.strapiapp.com/How_CAG_Works_8f5381b703.png)

During the generation process, the LLM combines this cached context with its pre-trained knowledge to produce personalized and relevant answers. 

Fundamentally, this limits the possibility of generating generic responses. CAG also continuously updates its context and determines the most relevant piece, ensuring the conversation stays aligned with your needs. 

## What is the main difference between RAG and CAG?

The main difference between these two [systems](https://www.meilisearch.com/blog/rag-types) is how they handle information when generating responses. 

RAG obtains external resources for knowledge augmentation, whereas CAG relies on previously stored data. 

Below is a quick preview of their key differences: 

![RAG vs CAG.png](https://unable-actionable-car.media.strapiapp.com/RAG_vs_CAG_cbdc135b87.png)

* **Data source:** RAG retrieves fresh information from external databases or search engines whenever a query is made. CAG works with pre-cached data that is already stored and ready for reuse.
* **Latency & performance:** RAG tends to have higher latency, as it requires a retrieval step every time, whereas CAG achieves low latency through preloading and reuse.
* **Cost efficiency:** With RAG, costs are typically higher due to the constant need to retrieve and process data. CAG, on the other hand, is more cost-effective since it eliminates repeated queries by reusing stored data.
* **Use case fit:** RAG is best suited for scenarios where real-time accuracy is essential. CAG is more useful for repetitive tasks where the same knowledge is applied often.
* **Scalability & maintenance:** RAG can be harder to maintain because it relies heavily on live updates and connections to external sources. CAG is easier to scale since cached data can be refreshed periodically and managed with less effort. 

Now, let’s look at the pros and cons of both RAG and CAG.

## What are the pros and cons of RAG?

RAG is beneficial for situations where accuracy is crucial, but it also has some limitations. Here are some upsides and downsides of RAG:

**Advantages**

* **Provides fresh information through real-time retrieval:** It can pull in the most up-to-date information, which is particularly beneficial when details change a lot.
* **Has a broad knowledge base:** Because it connects to external sources, it has access to a broader pool of knowledge than an AI model on its own.
* **Reliable for fact-checking:** It is beneficial for fact-checking or knowledge tasks where precision is key.

**Disadvantages**

* **Slower performance:** Since RAG must fetch new data, it has a higher inference time and responds more slowly.
* **Higher costs:** The constant retrieval of information makes it more expensive to operate.
* **Maintenance challenges:** RAG relies on external systems, so the results can be inaccurate if a source goes down or changes its data.
* **Complex setup:** Setting RAG up and keeping it running smoothly can be technically demanding.


## What are the pros and cons of CAG?

CAG may be faster and cheaper, but it does not come without downsides. Here are some advantages and disadvantages of CAG:

**Advantages**

* **Faster responses:** Since the data is pre-cached, there is no need to fetch information repeatedly, resulting in faster reactions.
* **Cost-effective:** CAG is more cost-effective and efficient because cached material reduces the need for repeated processing.
* **Efficient for repeat tasks:** CAG is ideal for repetitive queries, where the same information is frequently required.
* **Easier to scale:** It is easier to scale and maintain, as updating the cache can be done periodically with minimal hassle.

**Disadvantages**

* **Needs refresh cycles:** To remain useful, the caches used in CAG require periodic reprocessing and refreshment.
* **Risk of outdated information:** Cached data can go stale, meaning it might miss recent changes or updates.
* **Limited extended context:** The CAG is limited in terms of what is already stored, which means the scope of answers is narrower.
* **Potential bias:** If the cached content is biased or incomplete, the responses will reflect those flaws.

Wondering when it is best to use RAG over CAG? Let’s discuss the best-fit scenarios for both.

## When should you use RAG instead of CAG?

RAG is the better choice when your [retriever](https://www.meilisearch.com/blog/rag-tools) must fetch new information from an external source for every user’s query. 

For instance, in financial market updates, things move fast. Caching may not be helpful, as you do not want your AI system to return stale information. In such a case, RAG is an excellent approach. 

It can also be used in fact-checking scenarios or high-stakes research projects where credibility depends on having the most current and comprehensive information available. 

Another good use case of RAG over CAG is in scenarios where you do not expect repeated questions from users. It could also be used when you cannot precisely predict what users will ask. 

In these cases, caching will not be particularly helpful because the LLM must retrieve new information from the knowledge sources each time. 

In short, use RAG when your goal is to get current and accurate results, even if it takes a little longer or costs more.

## When is CAG preferable to RAG?

Use CAG when optimization for speed and low latency is key. By reusing embeddings and cached context, CAG reduces costs and improves response times for predictable scenarios.

This is particularly useful in situations such as customer support chatbots, where people frequently ask the same questions repeatedly. Answers to questions such as ‘How do I change my password?’ or ‘What is the refund policy?’ rarely change, so caching saves time and cost without sacrificing accuracy.

CAG also works well for internal knowledge bases where employees frequently need the same set of guidelines or instructions. 

Another great fit for CAG is large-scale deployments where keeping retrieval costs low is essential for scalability. 

While CAG may not always reflect the latest updates, its strengths lie in consistency and speed. So, if the information you are dealing with does not change often, then CAG is a practical choice.

## How do you choose between RAG and CAG for your project?

Choosing between RAG and CAG really depends on what your project needs most in terms of freshness, speed, or efficiency. 

Here are some key criteria to consider:

* **Nature of queries:** If your system faces unpredictable queries, RAG is stronger since it can reach out for fresh information. If the questions are repetitive (for example, common FAQs) then CAG will serve you better.
* **Budget considerations:** RAG is more costly because every query must undergo the retrieval phase. On the other hand, CAG is cheaper because the cached data is reused.
* **Performance requirements:** RAG can be slower, but it gives you depth and updated context. CAG is faster because it does not need to fetch new data each time it is executed.
* **Freshness of information required:** When working on tasks that require real-time information updates, such as financial data, it is recommended to use RAG. For stable knowledge, cached responses are more than enough.
* **Growth potential:** Scaling RAG can become complicated due to constant retrievals. Scaling CAG is easier, since updating caches is simpler to manage.
* **Handling varying outputs:** CAG keeps things quick and efficient when you do not need much flexibility, but when accuracy is non-negotiable, using RAG is the better option.
* **Technical setup:** RAG needs a stronger infrastructure to maintain live connections. CAG is easier to deploy due to fewer moving parts.

## Can RAG and CAG be combined?

Yes, you can combine RAG and CAG. In fact, combining RAG and CAG often gives you the best of both worlds. These are called [hybrid approaches](https://www.meilisearch.com/blog/rag-techniques). 

In such RAG pipelines, KV cache data is served first, and if it is missing or inadequate, the system triggers the retrieval step.

RAG and CAG can be combined in different ways, depending on your project’s goals. For example, you can build a:

* **Fallback hybrid:** The system defaults to cached responses for speed, but if the cache does not have a good answer, it switches to RAG for live retrieval.
* **Selective hybrid:** Certain types of queries are always routed to RAG (e.g., time-sensitive information) while routine or repetitive queries go straight to CAG.
* **Blended hybrid:** Both RAG and CAG are used together. Cached data provides a quick first draft, and RAG fills the gaps with fresh context if needed.
* **Tiered hybrid:** Responses start with cache for instant speed, then RAG silently updates or corrects the answer in the background so the answer to the query is fresher.

The benefits of a hybrid system are clear: faster responses, reduced costs, and still having the option to retrieve real-time data when needed. 

However, combining both also brings challenges. You need to decide on the best way to determine when to use the cache and when to fetch new information. Managing two approaches together can also add complexity to the system’s infrastructure and maintenance.

## RAG vs CAG: Choosing the right path for smarter, faster AI

When it comes to building more intelligent AI systems, RAG gives you freshness and accuracy by retrieving live data, while CAG provides you with speed and efficiency with its cached responses. 

The real power lies in knowing when to use each one or even combine them in a hybrid system. 

By balancing your API design, workflows, and knowledge integration needs, you can create LLMs that deliver optimized, real-world performance.

RAG vs. CAG: The Smarter Choice for Your AI Stack

Retrieval-augmented generation (RAG) has rapidly become one of the most effective methods for building AI systems. While most apps lean on Python, Ruby offers a lightweight path to implementing RAG pipelines without the need for large frameworks.

This guide will walk you through the essentials, from understanding what RAG means to building a working Ruby app.

You’ll discover: 

* What RAG is and how it fits into Ruby projects without heavy frameworks.
* Why developers pick Ruby for flexible, lightweight RAG setups.
* Libraries and services that make retrieval and generation work smoothly.
* Clear steps to build a working RAG app using plain Ruby.
* The difference between Ruby’s ecosystem and performance with Python for RAG development.
* How tools like Meilisearch help you build fast and reliable retrieval layers.

By the end, you’ll have everything you need to develop a functional RAG pipeline in Ruby from scratch.

## What is a RAG application in Ruby?

A RAG application in Ruby combines retrieval, augmentation, and generation to create smart, context-aware LLM responses. For details on how it works, you can take a peek at[ our guide on what RAG is](https://www.meilisearch.com/blog/what-is-rag). 

In essence, RAG retrieves relevant information from a vector knowledge base, augments a prompt with that data, and then sends it to a large language model (LLM) to generate accurate answers.

![How RAG Works.png](https://unable-actionable-car.media.strapiapp.com/How_RAG_Works_7352c5302f.png)

A frequent question is whether you can implement RAG without relying on heavy frameworks. Yes, you can. 

Ruby developers can rely on embeddings, search engines, and API calls to connect with LLM services for seamless integration. As a result, the AI-powered tools built through Ruby are incredibly lightweight. 

The following section explains why Ruby is a suitable option for using RAG.

## Why use RAG with Ruby?

In a RAG system, Ruby’s clean syntax and expressive language makes retrieval and embedding management relatively easy. Plus, you can connect to LLM APIs without any heavy frameworks. 

While the focus of this guide isn’t on Rails, generally, many teams already run production systems on Ruby or Rails. Adding RAG capabilities on top of that lets them integrate vector search and generation directly into existing apps without migrating their stack. 

Ruby’s ecosystem, although not as mature as Python's, simplifies API calls and background job handling, allowing developers to build lightweight scripts that optimize retrieval, augmentation, and generation.

Next, let’s look at the essential tools you’ll need to build a RAG application in Ruby.

## What tools are needed for RAG in Ruby?

To build a RAG application in Ruby, you need a combination of libraries and APIs that cover retrieval, embedding generation, and LLM integration. 

While the Ruby ecosystem is still growing, several reliable options are available to set up a complete RAG pipeline, many of which overlap with those discussed in [our RAG tools comparison](https://www.meilisearch.com/blog/rag-tools). 

Let’s look at the best ones below:

* **Langchainrb:** A Ruby library that aids in building retrieval and generation workflows through building blocks. It excels at supporting document loading, embedding generation, and basic orchestration.
* **Qdrant-ruby:** A popular open-source vector database as well as a Ruby client for Qdrant. It allows you to store and search embeddings efficiently from Ruby code.
* **Hugging Face or Transformer APIs:** These APIs can be called from Ruby to generate embeddings or run models remotely. They’re useful when you want to avoid maintaining your own ML infrastructure.
* **Other vector store gems:** Some developers use custom wrappers for databases, like PostgreSQL with pgvector, or they integrate with Meilisearch for hybrid keyword and [vector search](https://www.meilisearch.com/blog/what-is-vector-search).
* **LLM API gems:** These are gems for OpenAI, Anthropic, or other model providers that allow you to generate responses directly from Ruby.

There you have it: a set of tools associated with Ruby that provide the foundation for handling data retrieval and connecting to powerful language models.


## Step-by-step guide to building a RAG app in Ruby

This section is your walkthrough of a five-step process that covers data preparation, embedding generation, retrieval setup with Meilisearch, connecting to an LLM, and building a minimal interface to tie it all together.


### 1. Load and prepare your data

The first step is to bring text into your Ruby environment in a structured way. This could be from local files, APIs, or databases. 

You will need to clean the text, split it into logical chunks, and attach metadata (such as titles or URLs) so that retrieval later on is meaningful.


```ruby
# Example: Loading and cleaning a text file
file_path = "docs/guide.txt"
content = File.read(file_path).strip
chunks = content.split(/\n\n+/) # split into paragraphs
docs = chunks.map.with_index do |chunk, i|
  { id: i, text: chunk, metadata: { source: file_path } }
end
```


This preparation is the foundation that ensures your retriever can quickly find relevant information without having to parse messy inputs later.


### 2. Generate embeddings and store them

Once the documents are ready, you’re going to convert each chunk into a numerical embedding. You can use an API like OpenAI or Hugging Face for this purpose. 

Next, you will store the resulting vectors alongside the original text in a lightweight database or pair them with Meilisearch for hybrid retrieval.


```ruby
require 'net/http'
require 'json'

def embed(text)
  uri = URI("https://api.openai.com/v1/embeddings")
  headers = { "Content-Type" => "application/json", "Authorization" => "Bearer #{ENV['OPENAI_API_KEY']}" }
  body = { input: text, model: "text-embedding-3-small" }.to_json
  JSON.parse(Net::HTTP.post(uri, body, headers).body)["data"].first["embedding"]
end

embeddings = docs.map { |d| d.merge(embedding: embed(d[:text])) }
```


This gives your pipeline a semantic backbone for accurate retrieval.


### 3. Implement a retriever with Meilisearch

Now, it’s time to index your documents in Meilisearch. Why? To enable fast keyword and hybrid search. Feel free to store metadata alongside text for filtering as well. This gives you typo tolerance, relevance ranking, and blazing-fast queries.


```ruby
require 'meilisearch'

client = MeiliSearch::Client.new('http://127.0.0.1:7700', 'masterKey')
index = client.create_index('documents', { primaryKey: 'id' })
index.add_documents(docs)

# Query example
results = index.search('embedding generation')["hits"]
```


Developers often underestimate how significantly Meilisearch outperforms manual filtering or SQL full-text search in text retrieval. A clean index makes every later step more reliable.


### 4. Pass retrieved context to an LLM

Now that you have the retrieved context, let’s move it on to the LLM so the answers are actually grounded in data. You’ll build a prompt that includes the essentials, such as user query, brief instructions, and a trimmed context window. 

You can avoid the bloat by focusing on only the most relevant snippets. If you really want to show where the facts came from, just use citation markers. 


```ruby
require "openai"

class AnswerGenerator
  def initialize(model: ENV.fetch("OPENAI_MODEL", "gpt-4o-mini"))
    @client = OpenAI::Client.new(access_token: ENV.fetch("OPENAI_API_KEY"))
    @model  = model
  end

  def answer(query:, hits:)
    context = build_context(hits, max_chars: 6_000) # keep under model limits
    messages = [
      { role: "system", content: "You are a helpful assistant. Answer using only the provided context. Cite sources like [doc:index]. If the answer is not in the context, say you do not know." },
      { role: "user",   content: "Query: #{query}\n\nContext:\n#{context}\n\nAnswer:" }
    ]

    @client.chat(parameters: { model: @model, messages: messages, temperature: 0.2 })
           .dig("choices", 0, "message", "content")
  end

  private

  def build_context(hits, max_chars:)
    buf = +""
    hits.each_with_index do |hit, i|
      snippet = hit.dig("_formatted", "content") || hit["content"] || hit["snippet"] || ""
      block = "[doc:#{i+1}] #{snippet}\n"
      break if buf.size + block.size > max_chars
      buf << block
    end
    buf
  end
end
```


Several factors enhance the retrieval-to-generation step. 

Keep the temperature low so the model sticks to facts. If the context doesn’t support an answer, it should say so clearly. Trim or rerank longer contexts to stay within token limits. 

And don’t forget to log document IDs for easy traceability and later evaluation.


### 5. Build a CLI or service interface

Here’s a tidbit you’ll enjoy: wrap retrieval and generation in a small interface so you and your teammates can try the flow quickly. 

A CLI keeps things dependency-light and works well for scripts, cron jobs, and demos. Add retries and basic timeouts to handle flaky networks or rate limits.


```ruby
require "json"
require "timeout"
require_relative "indexer"          # your Meili client and search method
require_relative "answer_generator" # the class above

class RagCli
  def initialize
    @indexer  = Indexer.new(index: ENV.fetch("MEILI_INDEX", "docs"))
    @llm      = AnswerGenerator.new
  end

  def run
    puts "RAG on Ruby. Type a question, or 'exit'."
    loop do
      print "> "
      q = STDIN.gets&.strip
      break if q.nil? || q.downcase == "exit"
      next if q.empty?

      hits = safe { @indexer.search(q, limit: 5) } || []
      if hits.empty?
        puts "No context found."
        next
      end

      response = safe { @llm.answer(query: q, hits: hits) }
      puts "\n#{response}\n\n"
      puts "Sources:"
      hits.each_with_index do |h, i|
        id = h["id"] || h.dig("_formatted", "id") || "unknown"
        puts "  [doc:#{i+1}] #{id}"
      end
      puts
    end
  end

  private

  def safe(max_retries: 3)
    tries = 0
    begin
      Timeout.timeout(30) { return yield }
    rescue => e
      tries += 1
      sleep(0.5 * tries)
      retry if tries < max_retries
      warn "Error: #{e.class} #{e.message}"
      nil
    end
  end
end

RagCli.new.run if $PROGRAM_NAME == __FILE__
```


A few production checks go a long way. Ensure API keys are validated upfront and with clear errors when any information is missing. If your provider allows it, stream tokens so users see answers as they’re generated. 

Log the essentials, such as queries, selected document IDs, answers, and citations, for easy debugging and evaluation later. Keep prompts within safe length limits and sanitize inputs to block prompt injection attempts. 

And if you’re exposing the app over HTTP, run it through a small Rack or Sinatra service with sensible timeouts and rate limits to maintain stability.

This completes the core loop: retrieve with Meilisearch, ground the answer with your context, and ship an interface that is easy to run and extend.

## How is RAG in Ruby different from Python?

While Python dominates the RAG ecosystem, Ruby has its own strengths for developers who prefer a leaner, language-level approach. 

The key differences lie in the ecosystem, available tools, and performance characteristics.

![Ruby vs. Python for RAG.png](https://unable-actionable-car.media.strapiapp.com/Ruby_vs_Python_for_RAG_4790deaa9a.png)

Ruby focuses on simplicity and direct integrations rather than building entire RAG stacks internally. 

Python offers more built-in support for machine learning and orchestration, but can feel heavier for smaller apps. 

The right choice depends on your goals, scale, and existing stack.

## Unlocking the power of RAG with Ruby

RAG unlocks new possibilities for Ruby developers who want intelligent, context-aware applications without shifting their entire stack. 

For those working with Rails, our [guide to building RAG apps on Rails](https://www.meilisearch.com/blog/build-rag-apps-on-rails) shows how to extend these ideas further. 

By combining Ruby’s flexibility with the right tools, you can build retrieval and generation pipelines that are both lightweight and powerful.

Meilisearch latest news and company updates