What is semantic search? How it works, use cases & more

10 of the best Algolia alternatives [With features, pricing & more]

When you're handling millions of documents in a search engine, every millisecond counts. That's why we've completely reimagined how Meilisearch processes data with our new indexer in v1.12. We've rewritten our indexer—the core component that processes your documents and builds search structures—from the ground up. The result? A faster, more efficient, and more scalable system.

This complete redesign fundamentally changes how our search engine handles data. In this article, we'll show you how we've achieved up to 4x faster document updates and 30% smaller database sizes, and what this means for your applications.

## A foundation for the future

We've named this release `Indexer edition 2024`, reflecting our vision to build a foundation that will power Meilisearch's performance for years to come. Inspired by [Rust's edition model](https://doc.rust-lang.org/edition-guide/editions/index.html), this isn't just an update—it's a new chapter in Meilisearch's evolution, setting the stage for future innovations in search technology.

(Also like Rust editions, we took the year it was developed rather than the year it was released 😁)

## Why would you write a new indexer?

![the-doctor-meme.jpg](https://unable-actionable-car.media.strapiapp.com/the_doctor_meme_1f1323b0c6.jpg "The Doctor would not rewrite an indexer")

_The Doctor would not rewrite an indexer_

You need a _very_ strong reason to re-write an indexer. And we had one.

As our CTO Kero put it bluntly in his blog post: [Meilisearch is too slow](https://blog.kerollmops.com/meilisearch-is-too-slow). The new indexer is our answer to that challenge—a ground-up rewrite to dramatically improve speed.

The main goal of the new indexer is to improve the indexing speed of larger indexes with dozens of millions of documents, to relieve some of the stress on our infrastructure, to better exploit the resources of the machine, and to address some long-running pain points in a structured way.

## The crux of the matter: performance improvements

![Meme from the Jurassic Park movie with a t-rex (10m document updates) chasing a Jeep (Meilisearch)](https://unable-actionable-car.media.strapiapp.com/sometimes_more_speed_e32b6e5390.jpg "Sometimes you really gotta go faster")

_Sometimes you really gotta go faster_

In various scenarios, the new indexer is faster, sometimes multiple times faster, than the already optimized v1.11 indexer.

On machines with multiple cores, good IO and a lot of RAM, performance is vastly increased. Inserting new documents is twice as fast, while incrementally updating documents in a large database is x4 faster!

While the goal was to improve performance on bigger machines, we were successful in keeping similar or better performance for more modest machines with fewer cores and humble RAM (good IO remains vital to a database, though).

Additionally, for Cloud customers, we’re exploring optimizing our build by using native CPU instruction and [link-time-optimization (LTO),](https://llvm.org/docs/LinkTimeOptimization.html) yielding up to 12% improvements. We’re also considering advanced techniques like [profile-guided optimization (PGO)](https://doc.rust-lang.org/rustc/profile-guided-optimization.html), which so far has yielded an additional 12% improvement over native instructions and LTO.

## How it’s done: more parallelism, fewer I/O operations, pipelining writes

Meilisearch used to split a payload of documents into multiple chunks (roughly one per indexing thread), and would then work each chunk throughout the entire extraction process. Once a chunk was finished, it would then be persisted in the database.

This approach had a couple of drawbacks, performance-wise:

- As a lot of temporary data was extracted at the same time, it had to be written to disk and then read back to be persisted in the database, causing a lot of I/O operations.

- As the chunks were built independently of each other, they could result in us writing the same key to the database for each chunk, again causing unnecessary writes.

- As the chunks would finish computing at the same time on average, the indexing process would intensively use all the CPUs while computing the chunks, but then be solely single-threaded while writing everything to the database.

![old-process.png](https://unable-actionable-car.media.strapiapp.com/old_process_75e046f6ab.png "Simplified representation of the previous indexing process")

_Simplified representation of the previous indexing process_

In the new indexer, Meilisearch iterates in parallel over documents, performing a single extraction operation at a time. The result of that extraction is then merged in parallel, to compute the list of keys and values to send to the database. These keys and values are sent to the writer thread in a channel that acts as a pipeline, reducing the time where the merger threads wait for the writer thread.

![waits-faster.jpg](https://unable-actionable-car.media.strapiapp.com/waits_faster_36160c6cfb.jpg "The merger threads wait less thanks to pipelining")

_The merger threads wait less thanks to pipelining_

We successfully made the merger step parallel thanks to a key insight: while the extraction steps split documents between multiple threads, the merger step has to split database keys between multiple threads. For instance, to build the reverse index that matches a word with the list of documents that contain that word, each thread reads some of the documents to extract the words it contain, then all the documents containing a word are merged into a single list.

To make this possible, all threads during the extraction step deterministically hashes each word it encounters. Then during the merger step, words to handle are partitioned between threads depending on their hash value.

![new-process.png](https://unable-actionable-car.media.strapiapp.com/new_process_a34768ffec.png "Simplified representation of indexing process edition 2024")

_Simplified representation of indexing process edition 2024_

The drawback to this new approach is that some CPU-intensive operations, such as document tokenization, have to be duplicated across all steps. Still, merging in parallel utilizes a machine's multiple cores much more, reducing the time Meilisearch spends single-threaded.

Furthermore, by not duplicating key writes, the database size has been reduced by more than 30%.

## Better RAM control and usage

Meilisearch uses [bumpalo](https://docs.rs/bumpalo/latest/bumpalo/) to improve RAM control. Bumpalo is an arena allocator that allows to bulk deallocate memory and to query how much memory was allocated in an arena.

Arena allocators are often used in video games for objects that have to be recreated on each game frame, because allocating in an arena is very cheap (just a pointer bump), and deallocating can happen in bulk (yet another pointer bump) as long as no destructor has to be run.

In some regards, indexing documents with Meilisearch also has “frames”. In the new indexing approach, we can even detect two kinds of such frames: A very short-lived frame that corresponds to the work done on each document. And a longer-lived frame that starts at the beginning of each extraction step, and ends right after the merger is done sending everything to the writer. The “document” arena contains all allocations related to reading and tokenizing that document, while the “extraction” arena contains all allocations related to the data being extracted that will then be merged and written to the database.

![new-process(1).png](https://unable-actionable-car.media.strapiapp.com/new_process_1_ffaf674cd9.png)

During this process, Meilisearch queries the size of the “extraction” arena to detect when its RAM usage is above the threshold. When this happens, Meilisearch spills excess data to disk, so that the RAM usage stays in control.

If the extracted data fits in RAM, it never hits the disk, yielding fewer I/O operations.

## Faster and more reliable task cancellation

Historically, in the previous indexer, chunks of documents used to be very small, so it made sense to allow cancellation only after a chunk was done processing. Unfortunately, small chunks were inefficient when writing temporary data to disk, so we decided to switch to a “one chunk per core” model a few versions ago. This had an adverse effect on cancellation, as it would be processed almost at the end of the indexing process after all threads finished processing their chunk.

As the new Meilisearch iterates on documents, and further split the process by each extraction step, task cancellation is faster than ever, routinely being processed under 1s!

![cancel-all-the-tasks.jpg](https://unable-actionable-car.media.strapiapp.com/cancel_all_the_tasks_16b21cbbec.jpg "For all of you who regret sending all these documents at once")

_For all of you who regret sending all these documents at once_

## Better error messages

The new architecture exposes the document id to all extractors. This means that error messages can be enriched with this information as was done in [this PR](https://github.com/meilisearch/meilisearch/pull/5153): knowing which document caused the error allows for more effective troubleshooting.

## Better observability with progress

Lastly, the new indexer brings better observability to the table: because we know both which step we’re at and how many documents have been handled for this step, we can expose this information in the newly added `batches` [](https://www.notion.so/Enhance-visibility-on-batched-tasks-1194b06b651f810b8fe0fab5d72846a8?pvs=21)[route](https://www.notion.so/Enhance-visibility-on-batched-tasks-1194b06b651f810b8fe0fab5d72846a8?pvs=21) as a `progress` object.

```
{
  "uid": 160,
  "progress": {
    "steps": [
      {
        "currentStep": "processing tasks",
        "finished": 0,
        "total": 2
      },
      {
        "currentStep": "indexing",
        "finished": 2,
        "total": 3
      },
      {
        "currentStep": "extracting words",
        "finished": 3,
        "total": 13
      },
      {
        "currentStep": "document",
        "finished": 12300,
        "total": 19546
      }
    ],
    "percentage": 37.986263
  },
  "details": {
    "receivedDocuments": 19547,
    "indexedDocuments": null
  },
  "stats": {
    "totalNbTasks": 1,
    "status": {
      "processing": 1
    },
    "types": {
      "documentAdditionOrUpdate": 1
    },
    "indexUids": {
      "mieli": 1
    }
  },
  "duration": null,
  "startedAt": "2024-12-12T09:44:34.124726733Z",
  "finishedAt": null
}
```

In the above, we know that we finished extracting the words of 12300 out of 19546 documents. There are a total of 13 indexing steps, and we finished 3 of them already, for a global completion of 37.986263%.

Note that the `progress` object should only be used for display purposes, we do not guarantee that the step names or number remain the same from one version to the next.

Adding the `progress` object is a boon for the engine's observability, which already scored us concrete wins. We identified the bottleneck in a customer’s workload just by repeatedly calling the `batches` route and looking at which step was frequently seen.

## The best is yet to come!

The new indexer reaches new heights in maintainability with its structure that decouples the “business” extractors from its support functions (cancel, progress, reading a document), allowing us to quickly iterate on features, such as new settings to fine-tune the indexing behavior and the [AI stabilization](https://www.meilisearch.com/blog/meilisearch-1-13). In the future, we plan on adding progress to the settings tasks, enriching finished batches with the timing of each step. Feel free to share your ideas about what you’d like to see in Meilisearch on the [product discussion](https://github.com/orgs/meilisearch/discussions). If you’re new to Meilisearch, you can also try it for free on our Cloud.

Introducing Meilisearch's next-generation indexer: 4x faster updates, 30% less storage

What is latent semantic indexing (LSI) and how does it work?

You know what's amazing about search? It's at the heart of how people interact with your application. But let's be honest: building and managing search infrastructure can feel like you're spending more time wrestling with configuration than creating value for your users. That's why we're excited to share something that changes this completely: the Meilisearch MCP server.

## Why MCP? Building bridges between search and AI

Before we dive in, let's talk about what MCP (Model Context Protocol) is and why we're excited about it. [MCP](https://www.anthropic.com/news/model-context-protocol) is an emerging open protocol that standardizes how applications provide context to Large Language Models (LLMs). Think of it as a universal translator between your tools and AI models.

Even though MCP isn't yet fully standardized, we believe it represents the future of how developers will interact with their tools. Here's why:
- It creates a consistent way for AI models to interact with different tools and services
- It puts security and user control at the forefront
- It enables natural language interactions while maintaining precise control
- It's being built in the open, with input from the developer community

## From complex commands to natural conversations

Remember the last time you needed to update your search settings? It probably involved looking up API documentation, crafting the perfect JSON payload, and triple-checking your syntax. We've all been there. But what if you could just tell your search engine what you want, in plain English?

That's exactly what we've built. By implementing MCP, we've enabled natural conversations with Meilisearch through AI assistants like Claude. Here's what this means in practice using [our movies database](https://where2watch.meilisearch.com/) as an example:

### Before:
```bash
curl \
  -X POST 'http://localhost:7700/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer MASTER_KEY' \
  --data-binary '{
    "rankingRules": [
      "words",
      "typo",
      "proximity",
      "attribute",
      "sort",
      "exactness"
    ]
  }'
```

### After:
> "Update the ranking rules for the movies index to prioritize word matches and handle typos."

That's it. When paired with advanced LLMs like Claude Sonnet, the AI understands your intent and handles all the technical details behind the scenes. No more context-switching between documentation tabs.

## What can you do with it today?

Thanks to the capabilities of modern LLMs, our MCP integration already enables natural language control over your Meilisearch instance. Here's what you can do right now:

1. **Index management made simple**
```"Create a new index for our product catalog and optimize it for e-commerce search"```
The AI will guide you through setting up your index with appropriate configurations for e-commerce.

2. **Search optimization**
```"Our users are having trouble finding products when they make typos. How can we fix this?"```
Get instant guidance on configuring typo tolerance and seeing real-time results.

3. **Performance insights**
```"How is our search performance looking? Any areas we should optimize?"```
Understand your search analytics and get actionable recommendations.

## How does it work?

Our current MCP implementation exposes Meilisearch's capabilities through a set of carefully designed tools that AI assistants can use to manage your search infrastructure. When paired with advanced LLMs like [Claude Sonnet](https://www.anthropic.com/news/claude-3-5-sonnet), these tools enable natural conversations about complex search operations.

The best part? It works seamlessly with Claude Desktop and other MCP-compatible clients, giving you:
- Natural language control over your search infrastructure
- Instant access to best practices and optimization tips
- Real-time insights and analytics

## Getting started in 2 minutes

1. Install [Claude Desktop](https://claude.ai/download) (it's free!)
2. Add our MCP server to your configuration:

```json
{
  "mcpServers": {
    "meilisearch": {
      "command": "uvx",
      "args": ["-n", "meilisearch-mcp"]
    }
  }
}
```
3. Start managing your search like you're chatting with a colleague!

## Building the future of search

This is just the beginning of our mission to make search infrastructure more accessible. Here's what we're working on next:

### Smart prompts
We're developing pre-built templates for common scenarios like:
- Optimizing index settings for specific use cases (e-commerce, documentation, etc.)
- Debugging search behavior and performance issues
- Understanding and improving search analytics
- Implementing industry-specific best practices

### Enhanced capabilities
- More sophisticated analytics and monitoring tools
- Expanded AI assistant integrations
- Advanced collaborative features for teams
- Deeper integration with Meilisearch Cloud

## Open source and open for contributions

True to our values, the Meilisearch MCP server is completely open source. We believe in building in public and learning from our community. Check out our implementation, contribute, or adapt it for your needs:

[GitHub repository](https://github.com/meilisearch/meilisearch-mcp)

## Join us in simplifying search

- ⭐ Star [our repository](https://github.com/meilisearch/meilisearch)
- 💬 Join our [Discord community](https://discord.meilisearch.com/) & share your use cases (or do it on social media and tag us)
- 🛠️ Contribute to development on [GitHub](https://github.com/meilisearch/meilisearch-mcp)

We're excited to see how you'll use this to build better search experiences. Try it out and let us know what you think!

How we made Meilisearch talk to AI: introducing our MCP server

We're excited to introduce Meilisearch v1.13. In this article, we'll cover the most significant changes. For a full list of updates, visit the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.13.0) on GitHub.

<Callout intent="tip">
Meilisearch 1.13 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=release-v1-13&utm_source=blog), too—upgrade now!
</Callout>

## AI-powered search

We are thrilled to announce that AI-powered search is now fully stable! 

First released in v1.3, our vector store has evolved to auto-embedding from common providers, semantic and hybrid search, always based on our community's invaluable feedback. This feature is now ready to leave the experimental phase. 

Therefore, you no longer need to activate it using the `/experimental-features` route.

v1.13 introduces the last breaking changes to the AI-powered search feature.

### Breaking: accepted Ollama URLs

Ollama URLs will only be accepted if they end with `/api/embed` and `/api/embeddings`, passing an unsupported endpoint will now trigger an error on Meilisearch

### Breaking: error code modifications
Prior to 1.13, if the embedder configuration was missing or wasn't a string, Meilisearch would return an `invalid_embedder`error for both search and similarity requests.

It has been split into:
    - `invalid_search_embedder`: for search requests
    - `invalid_similar_embedder`: for similar requests
    
When the `hybrid` search parameter is invalid, the error `invalid_search_hybrid_query` is now returned instead of `invalid_hybrid_search`.


## Experimental: remote federated search 

The new [network route](https://www.meilisearch.com/docs/reference/api/network) enables distributed search across multiple Meilisearch instances. Combined with federated search, this simplifies implementing horizontal database partitioning strategies like sharding.

[Activate](https://www.meilisearch.com/docs/reference/api/experimental_features) this feature via the `/experimental-features` route. If you are using Meilisearch Cloud, contact support to enable this feature in your projects.

Use the `remotes` object to define the Meilisearch instances that should participate in the search. 



```json
PATCH /network
{
  "remotes": {
    "ms-01": {
      "url": "http://ms-1.example.meilisearch.io",
      "searchApiKey": "SEARCH_KEY_1"
    },
    "ms-02": {
      "url": "http://ms-2.example.meilisearch.io",
      "searchApiKey": "SEARCH_KEY_2"
    }
  }
}
```

Set the same remotes on all set of Meilisearch instances, then configure each instance with its own identifier using `self`. The value of `self` must match the corresponding key in remotes:

```json
PATCH /network
{
  "self": "ms-01"
}
```

You can now perform a remote federated search:

```json
POST /multi-search
{
  "federation": { // enable federated mode
    "limit": 5
  },
  "queries": [
    { // #0
      "q": "Batman",
      "indexUid": "movies",
      "federationOptions": {
        "remote": "ms-01"
      }
    },
    { // #1
      "q": "Batman",
	    "indexUid": "movies",
	    "federationOptions": {
        "remote": "ms-02"
      }
    }
  ]
}
```

## Experimental: new dumpless upgrade

You can now upgrade from Meilisearch >=v1.12 to Meilisearch >=v1.13 without using a [dump](https://www.meilisearch.com/docs/reference/api/dump).

Simply install the latest version of Meilisearch and relaunch it with either [instance option](https://www.meilisearch.com/docs/learn/self_hosted/configure_meilisearch_at_launch#dumpless-upgrade):
- the `--experimental-dumpless-upgrade` flag
- or the `MEILI_EXPERIMENTAL_DUMPLESS_UPGRADE` environment variable

⚠️ As this is an experimental feature, please use [snapshots](https://www.meilisearch.com/docs/learn/advanced/snapshots) to back up all important data first.

## New stats 

Meilisearch 1.3 introduces 3 new fields to the [stats object](https://www.meilisearch.com/docs/reference/api/stats#stats-object):

- `numberOfEmbeddedDocuments`: total number of documents with at least one embedding
- `numberOfEmbeddings`: total number of embeddings in an index 
- `usedDatabaseSize`: storage space used by the database in bytes, excluding unused space claimed by LMDB.  Unlike `databaseSize`, this value decreases when documents are removed.

Here is an example stats object returned (as of Meilisearch 1.13):

```json
GET /stats

{
  "databaseSize": 447819776,
  "usedDatabaseSize": 196608,
  "lastUpdate": "2019-11-15T11:15:22.092896Z",
  "indexes": {
    "movies": {
      "numberOfDocuments": 19654,
      "numberOfEmbeddedDocuments": 1,
      "numberOfEmbeddings": 1,
      "isIndexing": false,
      "fieldDistribution": {
        "poster": 19654,
        "overview": 19654,
        "title": 19654,
        "id": 19654,
        "release_date": 19654
      }
    },
    "books": {
      "numberOfDocuments": 5,
      "numberOfEmbeddedDocuments": 5,
      "numberOfEmbeddings": 10,
      "isIndexing": false,
      "fieldDistribution": {
        "id": 5,
        "title": 5,
        "author": 5,
        "price": 5, 
        "genres": 5
      }
    }
  }
}
```

## Contributors shout-out

We're incredibly grateful to the external contributors who helped make this release possible. Thanks to [@takaebato](https://github.com/takaebato), [@Sherlouk](https://github.com/Sherlouk), [@jameshiew](https://github.com/jameshiew), [@K-Kumar-01](https://github.com/K-Kumar-01), [@mhmoudr](https://github.com/mhmoudr), [@eltociear](https://github.com/eltociear), [@Gnosnay](https://github.com/Gnosnay), [@DerTimonius](https://github.com/DerTimonius), [@manojks1999](https://github.com/manojks1999), [@ellnix](https://github.com/ellnix), [@Guikingone](https://github.com/Guikingone), [@migueltarga](https://github.com/migueltarga) for their contributions to [Meilisearch](https://github.com/meilisearch/meilisearch).

---

And that’s a wrap for v1.13! These release notes only highlight the most significant updates. For an exhaustive listing, read the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.13.0) on GitHub.

---

For more information, subscribe to our [monthly newsletter](https://dub.sh/meili-newsletter?utm_campaign=release-v1-13&utm_source=blog), check out the [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-13&utm_source=blog), or join our [Product Discussions](https://github.com/orgs/meilisearch/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-13&utm_source=blog).


Meilisearch 1.13

## 🇯🇵 New Meilisearch Cloud region

[
![New Meilisearch Cloud region: Japan](https://unable-actionable-car.media.strapiapp.com/Ggs8g_MXXMA_Ar_GY_8_4134d2969a.jpeg)](https://cloud.meilisearch.com/?utm_campaign=newsletter&utm_source=blog&utm_medium=january-25-updates)

Meilisearch Cloud is expanding! You can now **deploy Meilisearch in Japan**. Get faster, localized search today.

[Get started](https://cloud.meilisearch.com/?utm_campaign=newsletter&utm_source=blog&utm_medium=january-25-updates)

## 🔔 Upgrade to v1.12

[![Meilisearch 1.12 release notes](https://unable-actionable-car.media.strapiapp.com/meilisearch_1_12_7139b280d2.jpg)](https://www.meilisearch.com/blog/meilisearch-1-12)

Last month, we shipped v1.12 which brings 2x faster document insertion, 4x faster incremental updates, and 1.5x faster embedding generation. Update now!

[Learn more](https://www.meilisearch.com/blog/meilisearch-1-12)

## 📰 In the news

### 💬 Building a RAG system with Meilisearch

Learn the best practices for building a retrieval-augmented generation system.

[Read the guide](https://www.meilisearch.com/blog/how-to-build-rag)

### 🔍 Semantic search with Mistral embeddings

Build semantic search with the latest Mistral Small 3 model from Anthropic.

[Read the guide](https://www.meilisearch.com/docs/guides/ai/mistral)

### ✨ How personalization and recommendations enhance search and discovery

A case study of how Amazon, Reddit, Notion, and more build engaging search UX.

[Read the case study](https://www.meilisearch.com/blog/personalization-recommendations)

### ⚛️ How to add AI-powered search to a React app

Learn to build a React movie search and recommendation app with Meilisearch.

[Read the guide](https://www.meilisearch.com/blog/add-ai-powered-search-to-react)

### 🚀 Building docs search with Docs Crawler

Integrate search in your documentation website using the documentation crawler.

[Read the guide](https://www.meilisearch.com/blog/documentation-crawler-release)

## 🤗 Made with Meilisearch

[LibreChat](https://www.librechat.ai/) is an open-source AI Conversation app compatible with all AI providers. Use a single, customizable UI to interact with any LLM models. LibreChat uses Meilisearch to search through conversation history.

[![LibreChat](https://unable-actionable-car.media.strapiapp.com/demo_light_70288907aa.webp)](https://www.librechat.ai/)

_We’re always looking to promote new projects from the community. Post yours in the [#built-with-meilisearch](https://dub.sh/meili-discord?utm_campaign=newsletter&utm_source=blog&utm_medium=january-25-updates) channel!_


## 😏 Did you know?

The fastest way to get started is with Meilisearch Cloud.

[Start free trial](https://www.meilisearch.com/cloud)

---

Stay in the loop of everything Meilisearch by subscribing to our [monthly newsletter](https://dub.sh/meili-newsletter?utm_campaign=newsletter&utm_source=blog&utm_medium=january-25-updates). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=newsletter&utm_source=blog&utm_medium=january-25-updates) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions).

For anything else, join our developers community on [Discord](https://dub.sh/meili-discord?utm_campaign=newsletter&utm_source=blog&utm_medium=january-25-updates).

Meilisearch January Updates

Enterprise search: a comprehensive guide

When your AI application starts hemorrhaging $500 daily in OpenAI API calls and users are drumming their fingers waiting 2 seconds for responses, semantic caching isn't just a nice-to-have—it's a lifeline.

While traditional caches match queries word-for-word, semantic caching understands the deeper meaning.

In this guide, you'll discover how to implement semantic caching that slashes response times and API costs. Whether you're building a chatbot, search engine, or AI-powered application, these strategies will help you create lightning-fast, cost-effective semantic search that keeps users happy and your infrastructure lean. You'll learn how to:

- Reduce response times dramatically
- Achieve massive API cost savings
- Handle similar questions without repeatedly querying language models

## Introduction to semantic caching

Semantic caching revolutionizes how we store and retrieve information in modern applications. Traditional caching relies on exact matches, while semantic caching understands the meaning behind queries. This allows systems to recognize and respond to similar questions, even if they're worded differently.

This technique is valuable for AI-powered applications. Processing similar queries repeatedly can be both time-consuming and expensive.

### What is semantic caching

Semantic caching is a sophisticated data retrieval technique. It stores both the data and the contextual meaning of queries and their results. When a user asks a question, the system analyzes its semantic meaning rather than looking for an exact match.

For example, "What's the weather in New York?" and "Tell me New York's forecast" can pull from the same cached response, saving processing time and resources.

### Comparing semantic caching vs traditional caching

![Semantic Cache](https://unable-actionable-car.media.strapiapp.com/how_to_cache_semantic_search_1996414fc7.avif)

Traditional caching and semantic caching serve different purposes in modern applications. Here's a comparison:

- **Traditional caching**: Relies on exact matches and specific keys to retrieve data. Think of it as using a filing cabinet where you need the exact file name.
- **Semantic caching**: Understands context and meaning. It's like having a smart assistant who can find related information even when you ask in different ways.

### Advantages of semantic caching

Semantic caching offers several advantages for real-world applications:

- **Reduced API costs** by avoiding redundant queries.
- **Faster response times** for similar questions.
- **More consistent answers** across related queries.
- **Better scalability** for AI-powered applications.

### When to choose semantic caching

Consider semantic caching for your application if:

- **Users might ask the same question in different ways.**
- Response consistency is crucial.
- API costs need to be optimized.
- Real-time response is essential.

## Core concepts of semantic caching

Understanding the fundamental components and applications of semantic caching is crucial for successful implementation. Let's explore how these elements work together to create efficient caching systems that enhance your application's performance.

### Key components of semantic caching systems

Several critical components form the foundation of semantic caching.

- [**Embedding model**](https://www.meilisearch.com/blog/what-are-vector-embeddings): Converts queries into vector representations capturing their meaning.
- [**Vector database**](https://www.meilisearch.com/blog/what-is-a-vector-database): Stores these vectors, allowing for quick similarity searches for new queries.
- **Cache layer**: Sits between your application and underlying systems, managing query-response pairs and their semantic meanings. It decides whether to serve cached content or process new requests.
- **Vector search mechanism**: Evaluates incoming queries against stored embeddings using similarity thresholds to determine cache hits.

### Semantic cache in RAG

**Retrieval-Augmented Generation (RAG)** systems use semantic caching to improve performance and reduce costs. When a user submits a query, the system first checks the semantic cache for similar questions before going through the full retrieval and generation process. This is especially effective for handling common queries in customer support or knowledge base applications.

The integration process involves three main steps:

1. Embed the incoming query into a vector representation.
2. Search the cache for semantically similar queries.
3. If a match is found, return the cached response immediately. If not, proceed with the traditional RAG pipeline and store the new result for future use.

By understanding these core concepts and components, you can effectively implement and optimize semantic caching in your AI applications.

## Implementing semantic caching for an e-commerce site

Semantic caching can significantly enhance how e-commerce platforms handle product searches and customer queries. By adopting a smart caching mechanism, businesses can boost search performance and improve user experience.

![how to cache semantic search](https://unable-actionable-car.media.strapiapp.com/how_to_cache_semantic_search_ba174395a9.avif)

We will go through these key steps:

1. Configure embedding model by initializing OpenAI embeddings and Meilisearch client with appropriate connection parameters.
2. Create a vector store with custom configuration, specifying index name, embedding dimensions, and text key for efficient semantic search storage.
3. [Implement semantic search](https://www.meilisearch.com/docs/guides/ai/openai) function to convert queries into vector embeddings and perform similarity-based retrieval from the vector store.
4. Develop cache management logic to handle cache hits and misses, including fallback mechanisms for database searches and optional cache updates.

### Setting up the semantic caching infrastructure

Building a semantic caching system requires careful planning and the right tools. Here's a step-by-step guide to implementation.

#### Choosing core technologies

Start by [selecting the technologies for your system](https://www.meilisearch.com/blog/choosing-the-best-model-for-semantic-search). **Meilisearch is an excellent vector search engine** for semantic caching. For this example, we'll use Python with OpenAI embeddings to create a context-aware caching solution.

#### Installing required dependencies

Set up your project environment by installing the necessary libraries:

```bash
pip install meilisearch langchain openai python-dotenv
```

This command installs essential tools like Meilisearch for vector storage and LangChain for embedding management.

#### Configuring the embedding model

The core of semantic caching is converting queries into meaningful vector representations. Below is a sample configuration using OpenAI embeddings:

```python
from langchain.embeddings.openai import OpenAIEmbeddings
import meilisearch

# Set up OpenAI embeddings
embeddings = OpenAIEmbeddings()

# Initialize Meilisearch client
client = meilisearch.Client(
    url="your_meilisearch_url",
    api_key="your_api_key"
)
```

#### Creating a vector store for semantic caching

Setting up a vector store is crucial for implementing an effective semantic cache. Here's a comprehensive approach to creating a vector store using Meilisearch and OpenAI embeddings:

```python
# Create vector store with custom configuration
vector_store = Meilisearch(
    client=client,
    embedding=embeddings,
    index_name="semantic_cache",
    text_key="content"
)

# Optional: Define embedder configuration
embedder_config = {
    "custom": {
        "source": "userProvided",
        "dimensions": 1536  # OpenAI embedding dimensions
    }
}
```

This setup allows you to create a robust semantic cache that can efficiently store and retrieve contextually similar queries and their responses.

### Implementing the semantic cache mechanism

The key to semantic caching is creating a **retrieval system that understands query intent**.

#### Creating a semantic search function

Here’s an example of a semantic search function:

```python
def semantic_cache(query, k=3):
    # Convert query to vector embedding
    query_embedding = embeddings.embed_query(query)
    
    # Perform similarity search in Meilisearch
    results = vector_store.similarity_search(
        query=query,
        k=k
    )
    
    return results
```

This function:

- Converts the query into a vector representation.
- Searches for semantically similar results in the vector store.
- Returns the most relevant matches.

#### Handling cache hits and misses

A robust caching strategy must handle both cache hits and misses effectively:

```python
def process_product_search(query):
    # Check the semantic cache
    cached_results = semantic_cache(query)
    
    if cached_results:
        # Cache hit: return cached results
        return {
            "source": "cache",
            "results": cached_results
        }
    else:
        # Cache miss: perform full database search
        full_results = perform_database_search(query)
        
        # Optionally, store new results in cache
        update_semantic_cache(query, full_results)
        
        return {
            "source": "database",
            "results": full_results
        }
```

### Performance optimization techniques

To ensure efficient semantic caching, consider these strategies:

- **Set a similarity threshold** (e.g., 0.85–0.95) to filter results.
- Limit cache size to prevent memory issues.
- Use time-to-live (TTL) to refresh outdated cache entries.
- Regularly update your embedding model for better accuracy.

### Testing your semantic cache

Testing is critical to ensure reliability. Test scenarios should include:

- Exact match queries.
- Semantically similar queries.
- Completely new queries.
- Edge cases with minimal context.

**Thorough testing ensures your system performs well across diverse search patterns.**

Meilisearch’s fast vector search capabilities make semantic caching not only feasible but also highly efficient for e-commerce platforms.

## Adding semantic caching to RAG system

Retrieval-Augmented Generation (RAG) systems can greatly benefit from semantic caching. This integration **improves efficiency, reduces costs, and enhances responsiveness** in AI applications.

### Understanding RAG and semantic caching integration

Semantic caching in RAG systems focuses on storing and retrieving query responses based on their meaning, not just exact text matches. This allows AI applications to quickly access previously generated responses while maintaining contextual relevance.

Key steps in integrating semantic caching into a RAG workflow include:

1. **Embedding Transformation**: Convert user queries into vector representations that capture semantic meaning. Use embedding models like BERT or OpenAI's embedding APIs for this transformation.

2. **Cache Lookup Mechanism**: Use a vector database for fast, similarity-based searches. **Meilisearch is a strong choice** for its efficient semantic search capabilities, enabling quick matches between incoming queries and cached responses.

### Implementing the semantic caching layer

Implementing a semantic caching layer requires creating a specialized class that manages query embeddings, similarity checks, and cached response retrieval. This code provides a practical approach to efficiently storing and accessing semantically similar query responses.

```python
class RAGSemanticCache:
    def __init__(self, embedding_model, vector_store):
        self.embedding_model = embedding_model
        self.vector_store = vector_store
        self.similarity_threshold = 0.85

    def retrieve_cached_response(self, query):
        query_embedding = self.embedding_model.embed_query(query)
        similar_results = self.vector_store.similarity_search(
            query_embedding, 
            threshold=self.similarity_threshold
        )
        return similar_results[0] if similar_results else None
```

This class:

- Embeds queries into vectors.
- Searches for semantically similar results in the vector store.
- Returns a cached response if it meets the similarity threshold.

<Callout intent="note">
	
	To ensure your semantic caching layer performs efficiently, apply these strategies:
	
	- **Dynamic Similarity Thresholds**: Adjust thresholds based on your domain and use case.
	- **Cache Maintenance**: Use expiration and refresh mechanisms to keep responses relevant.
	- **Multi-Turn Context Tracking**: For conversational systems, include conversation history in cache retrieval to maintain context.
	
	
</Callout>

### Practical implementation example

Here’s how semantic caching can be integrated into a RAG workflow:

```python
def rag_with_semantic_cache(query, rag_system, semantic_cache):
    # Check semantic cache first
    cached_response = semantic_cache.retrieve_cached_response(query)
    
    if cached_response:
        return cached_response
    
    # If no cached response, use traditional RAG
    retrieved_docs = rag_system.retrieve_documents(query)
    generated_response = rag_system.generate_response(query, retrieved_docs)
    
    # Store the new response in the semantic cache
    semantic_cache.store_response(query, generated_response)
    
    return generated_response
```

This function:

- Checks the semantic cache for a response.
- Falls back to the RAG system if no cached response exists.
- Stores new responses in the cache for future use.

### Benefits for different applications

Semantic caching is especially useful in:

- Customer support chatbots.
- Enterprise knowledge management systems.
- Educational platforms.
- Research and analytics tools.

## Optimization strategies

Once your semantic caching system is running, focus on optimization to ensure it continues to deliver maximum value. Let's explore key strategies to maintain and improve your system's performance over time.

### Ensuring accuracy & consistency

Maintaining high accuracy in semantic caching requires regular monitoring and updates.

- **Implement version control** for cached responses to track changes and ensure users receive the most current information.
- **Invalidate or update related cache entries** when source data updates to prevent serving outdated information.
- **Cache consistency** becomes particularly important in distributed systems. Use a publish-subscribe system to propagate updates across your infrastructure, ensuring all cache nodes remain synchronized.
- **Regular validation of cached responses** helps maintain quality. Set up automated checks to verify cached responses still match their original context and meaning. Periodically compare cached responses against fresh generations to identify any drift in accuracy.

### Improving performance and efficiency

**Performance optimization** starts with careful monitoring of key metrics. Track cache hit rates, response times, and system resource usage to identify bottlenecks and optimization opportunities. Use dashboards to **visualize these metrics** and establish baseline performance expectations.

Consider these advanced optimization techniques:

- **Predictive caching** for frequently accessed queries.
- **Tiered caching** with different storage solutions for hot and cold data.
- **Optimize vector similarity calculations** for faster retrieval.
- **Compress embeddings** while maintaining accuracy.

Regular performance audits help identify improvement areas. Compare your system's performance against benchmarks and adjust your caching strategy accordingly. Document performance improvements and their impact on user experience and operational costs.

### Common pitfalls and how to avoid them

**Over-aggressive caching** can lead to memory issues and degraded performance. Start with conservative cache sizes and grow based on actual usage patterns rather than theoretical maximums.

**Poor similarity threshold selection** represents another challenge. Setting thresholds too low leads to false positives and incorrect responses. Setting them too high results in unnecessary cache misses. Monitor your threshold settings and adjust based on user feedback and performance metrics.

Watch out for these additional pitfalls:

- **Neglecting cache maintenance and cleanup**.
- **Failing to handle edge cases** in query processing.
- **Ignoring the impact of cache invalidation** on system performance.
- **Not implementing proper error handling** and fallback mechanisms.

**Focusing on these optimization strategies** ensures your semantic caching system remains efficient and effective, providing consistent performance improvements over time.

## Semantic caching: the path to high performance search

**Implementing semantic caching for your search system is a powerful way to boost performance, reduce costs, and enhance user experience.**

By following this guide's systematic approach to setting up semantic caching—from infrastructure assessment to optimization strategies—you'll be well-equipped to handle growing query volumes while maintaining fast response times.

<Callout intent="tip">
	
	No search engine expertise required! [Get started with Meilisearch Cloud](https://meilisearch.com/get-started) and experience an intuitive platform that makes implementing powerful search effortless, even for developers new to search technologies.
	
	
</Callout>

Meilisearch latest news and company updates