In today's interconnected world, many applications need to handle content in multiple languages. Meilisearch offers powerful capabilities for multilingual search, combining intelligent language handling with specialized models for optimal relevance. This guide will walk you through setting up multilingual search with Meilisearch, covering both full-text and semantic search approaches.

Why multilingual search matters
===============================

Before we dive into the technical details, let's consider why multilingual search is crucial:

1.  Global reach: Expand your audience by catering to users in their native languages.
2.  Improved user experience: Provide relevant results regardless of the language used in queries or content.
3.  Increased engagement: Users are more likely to interact with content they can easily find and understand.

Now, let's explore how Meilisearch makes multilingual search a breeze.

Full-text search: intelligent language handling
-----------------------------------------------

At its core, Meilisearch excels at handling diverse languages out of the box. Our [advanced tokenizer](https://www.meilisearch.com/docs/learn/indexing/tokenization?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) is designed to work with:

*   Languages that use whitespace to separate words (like English, Spanish, and French)
*   Languages with different writing systems (such as Chinese, Japanese, and Thai)

### Automatic language detection

When you add documents to your Meilisearch index, the engine automatically detects the language of each field. This means you don't need to specify languages manually in most cases - Meilisearch will handle it for you.

For example, if your documents contain a mix of English and Spanish content:

```json
[
  {
    "id": 1,
    "title": "The quick brown fox",
    "description": "El zorro marrón rápido salta sobre el perro perezoso"
  },
  {
    "id": 2,
    "title": "La luna y las estrellas",
    "description": "A beautiful night sky filled with twinkling stars"
  }
]
```

Meilisearch will automatically detect and properly handle the different languages in each field, ensuring accurate search results across your multilingual content.

### Specifying languages for specific fields

While automatic detection works well in most cases, you might want more control over language handling for certain fields. Meilisearch allows you to [specify languages](https://www.meilisearch.com/docs/reference/api/settings#localized-attributes?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) for particular attributes using the `localizedAttributes` setting.

Here's how you can set it up:

```json
{
  "localizedAttributes": [
    {
      "locales": ["eng"],
      "attributePatterns": ["title_en", "*_english"]
    },
    {
      "locales": ["spa"],
      "attributePatterns": ["description_es", "*_spanish"]
    }
  ]
}
```

This configuration tells Meilisearch to treat fields ending with `_en` or `_english` as English, and fields ending with `_es` or `_spanish` as Spanish. This approach is particularly useful when you have a consistent naming convention for your multilingual fields or when you want to ensure specific treatment for certain attributes.

Semantic search: Enhancing multilingual relevance
=================================================

While full-text search works well for many multilingual scenarios, semantic search can provide even better results, especially for complex queries or when dealing with nuanced meanings across languages.

Meilisearch supports various embedding models for semantic search, offering powerful auto-embedding capabilities. This means you can easily set up semantic search without having to manage the embedding process yourself. Meilisearch handles the generation and storage of embeddings automatically, seamlessly integrating them into your search experience.

For multilingual applications, we recommend using specialized multilingual models rather than relying on general-purpose models. These models are designed to understand and represent meaning across multiple languages, leading to more relevant search results in multilingual contexts.

One excellent option for multilingual semantic search is the [Cohere Multilingual embedding model](https://docs.cohere.com/docs/multilingual-language-models). It offers a great balance of speed and relevance across a wide range of languages. Cohere's model is particularly adept at understanding context and nuance in different languages, making it an ideal choice for diverse, multilingual content. Follow [the guide](https://www.meilisearch.com/docs/guides/ai/cohere?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) if you want to do it now.

To explore different models and their performance, you can use the [Meilisearch playground](https://playground.meilisearch.com). This interactive tool allows you to test various embedding models and see how they perform with your specific multilingual queries and content.

Conclusion
----------

We encourage you to experiment with different settings and models to find the optimal configuration for your specific use case. And remember, the Meilisearch community is always here to help - join us on [Discord](https://discord.meilisearch.com/) or check out our documentation for more detailed information and support.

Ready to get started with multilingual search? Try Meilisearch today [with a 14 day trial](https://cloud.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search)!

Searching across multiple languages

How to add AI-powered search to a React app

  
[Semantic search](/blog/what-is-a-vector-database/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison#what-is-similarity-search) is transforming search technology by providing more accurate and relevant results. However, with many embedding models available, choosing the right one can be challenging. This guide will help you understand the key factors to consider when selecting a model to build semantic search.

Overview
--------

In this guide, we will use the open-source search engine [Meilisearch](https://www.meilisearch.com?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) to perform the semantic searches. For the purpose of the tests, we're using the entry tier of [Meilisearch Cloud](https://www.meilisearch.com/cloud?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) (i.e., the [Build plan](https://www.meilisearch.com/pricing?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) .)

This guide will cover the following models:

| Model/Service | Dimensions | Context Length |
| --- | --- | --- |
| [Cohere embed-english-v3.0](https://docs.cohere.com/docs/embeddings) | 1024 | 512 |
| [Cohere embed-english-light-v3.0](https://docs.cohere.com/docs/embeddings) | 384 | 512 |
| [Cohere embed-multilingual-v3.0](https://docs.cohere.com/docs/embeddings) | 1024 | 512 |
| [Cohere embed-multilingual-light-v3.0](https://docs.cohere.com/docs/embeddings) | 384 | 512 |
| [OpenAI text-embedding-3-small](https://platform.openai.com/docs/guides/embeddings) | 1536 | 8192 |
| [OpenAI text-embedding-3-large](https://platform.openai.com/docs/guides/embeddings) | 3072 | 8192 |
| [Mistral](https://mistral.ai/product/embeddings/) | 1024 | 8192 |
| [VoyageAI voyage-2](https://voyageai.com/) | 1024 | 4000 |
| [VoyageAI voyage-large-2](https://voyageai.com/) | 1536 | 16000 |
| [VoyageAI voyage-multilingual-2](https://voyageai.com/) | 1024 | 32000 |
| [Jina Colbert v2](https://jina.ai/news/jina-colbert-v2-multilingual-late-interaction-retriever-for-embedding-and-reranking/) | 128, 96, or 64 | 8192 |
| [OSS all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 384 | 512 |
| [OSS bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | 1024 | 512 |
| [OSS bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | 1536 | 512 |

Factors to consider
-------------------

### 1\. Results relevancy

[Relevancy](https://www.meilisearch.com/docs/learn/relevancy/relevancy?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) is crucial for effective search, as it ensures that users find the most pertinent results quickly. In the realm of semantic search, achieving a balance between relevancy and speed is essential to provide a seamless user experience. It's important to consider the tradeoffs of [vector search vs full-text search](/blog/full-text-search-vs-vector-search/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison)

When selecting a model, consider your specific use case, such as the need for multilingual support, handling multi-modal data, or addressing domain-specific requirements. If you have a highly specialized use case or need to support a particular language, it may be beneficial to explore models that can be trained on your data or opt for multilingual models.

The performance difference between a very small model and a large model is not always substantial. **Smaller models are generally less expensive and faster**, making them a practical choice in many scenarios. Therefore, it is often worth considering smaller models for their cost-effectiveness and speed.

Additionally, you should always consider the context you're providing to the model. In Meilisearch, this comes in the form of a [document template](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison#embedders-object). The more accurately the template describes the data, the better the search results will be, leading to a more satisfying user experience.

### 2\. Search performance

You read everywhere now. Time is money. And the web is no different. Nowadays, [search-as-you-type](https://www.meilisearch.com/docs/learn/getting_started/what_is_meilisearch?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison#features) is the baseline for customer-facing applications. Saving users time greatly enhances their satisfaction and keeps them engaged with your platform.

To achieve lightning-fast search performance, **consider using a local model** to minimize latency by eliminating the need for round trips to the embedding service. If you need to use a remote model, then hosting your search service (e.g., your Meilisearch database) in close proximity to the embedding service can significantly reduce latency.

The table below showcases latency benchmarks for various local embedding models and embedding APIs. All requests are originated from a Meilisearch instance hosted on AWS (London datacenter.)

| Model/Service | Latency |
| --- | --- |
| [Cloudflare bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | ±800ms |
| [Cloudflare bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | ±500ms |
| [Cohere embed-english-v3.0](https://docs.cohere.com/docs/embeddings) | ±170ms |
| [Cohere embed-english-light-v3.0](https://docs.cohere.com/docs/embeddings) | ±160ms |
| [Local gte-small](https://huggingface.co/sentence-transformers/gte-small) | ±20ms |
| [Local all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | ±10ms |
| [Local bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | ±20ms |
| [Local bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | ±60ms |
| [Mistral](https://mistral.ai/product/embeddings/) | ±200ms |
| [Jina colbert](https://huggingface.co/jinac/cobert-v1) | ±400ms |
| [OpenAI text-embedding-3-small](https://platform.openai.com/docs/guides/embeddings) | ±460ms |
| [OpenAI text-embedding-3-large](https://platform.openai.com/docs/guides/embeddings) | ±750ms |
| [VoyageAI voyage-2](https://voyageai.com/) | ±350ms |
| [VoyageAI voyage-large-2](https://voyageai.com/) | ±400ms |

Here you can see that there are some clear winners in terms of latency. Unfortunately, latency is not the same as throughput, so we also need to take a close look at the indexing time.

### 3\. Indexing performance

Indexing performance is another critical aspect when [comparing search solutions](https://www.meilisearch.com/docs/learn/resources/comparison_to_alternatives?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison). The embedding model performance will directly impact the indexing speed of your search solution. And the speed at which your data can be indexed directly impacts the overall efficiency and scalability of your search solution.

Local **models without GPUs may have slower indexing** due to limited processing power. In contrast, third-party services offer varying speeds and limitations based on their infrastructure and service agreements. It is essential to evaluate these factors to ensure that your chosen model and service can meet your requirements effectively.

Several factors come into play when optimizing indexing. Again, the latency plays a big role: reducing the time taken for data to travel between your application and the model is always going to improve your experience. Additionally, the **maximum size of API calls** the API accepts, the provider's **rate limiting**, and the model's supported **number of dimensions** can all influence the efficiency and scalability of the indexing process.

The benchmark below compares the indexing of a 10k e-commerce documents (with [automatic embedding generation](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison#generate-auto-embeddings)):

| Model/Service | Indexation Time |
| --- | --- |
| [Cohere embed-english-v3.0](https://docs.cohere.com/docs/embeddings) | 43s |
| [Cohere embed-english-light-v3.0](https://docs.cohere.com/docs/embeddings) | 16s |
| [OpenAI text-embedding-3-small](https://platform.openai.com/docs/guides/embeddings) | 95s |
| [OpenAI text-embedding-3-large](https://platform.openai.com/docs/guides/embeddings) | 151s |
| [Cloudflare bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | 152s |
| [Cloudflare bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | 159s |
| [Jina Colbert V2](https://huggingface.co/jinac/cobert-v1) | 375s |
| [VoyageAI voyage-large-2](https://voyageai.com/) | 409s |
| [Mistral](https://mistral.ai/product/embeddings/) | 409s |
| [Local all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 880s |
| [Local bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | 3379s |
| [Local bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | 9132s |

### 4\. Pricing

While local embedders are free, most services charge per million of tokens. Here's a breakdown of the pricing for each platform:

| Provider      | Price                                      |
|---------------|--------------------------------------------|
| Cohere        | $0.10 per million tokens                   |
| OpenAI        | $0.13 per million tokens for text-embedding-3-large  |
|               | $0.02 per million tokens for text-embedding-3-small  |
| Cloudflare    | $0.011 per 1,000 Neurons                   |
| Jina          | $0.18 per million tokens                   |
| Mistral       | $0.10 per million tokens                   |
| VoyageAI      | $0.10 per million tokens for voyage-2      |
|               | $0.12 per million tokens for voyage-large-2 |
|               | $0.12 per million tokens for voyage-multilingual-2 |
| Local model   | Free                                       |


As your search needs grow and scale, it may become more cost-effective to invest in your own GPU machine. By having your own hardware, you can have greater control over the performance and scalability of your search solution and potentially reduce costs in the long run.

It is often best to start with a well-known model from the list provided. They are generally easy to setup and you will easily find community resources to help you.. As the need arises, you can consider migrating the model to a cloud provider like AWS. Many services offer this option, allowing you to leverage their infrastructure for improved performance and scalability.

Alternatively, you can choose an equivalent open-source model to self-host, giving you even more flexibility and control over your search solution in the long term. Please note that optimizing local models for performance or high volume may require to scale your infrastructure accordingly.

* * *

Ready to elevate your search experience?

[Talk to a search expert](https://dub.sh/meili-sales?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison)

* * *

Going further
-------------

While this article provides a comprehensive overview, we did not delve deeply into optimization techniques. There are several additional optimizations that can be explored to further enhance the performance of semantic search.

Here is a list of additional areas to investigate when choosing a model for your search experience:

*   Experiment with different presets (query vs. document) for models that offer this option to potentially improve relevancy
*   Evaluate specialized models for specific applications to assess their performance and suitability for your use case
*   Explore models that provide a reranking function to further refine search results
*   Test higher-tier accounts on each platform to check for improved performance and reduced rate limiting
*   Investigate parameters for receiving quantized data directly from the API to optimize data transfer and processing

Conclusion
----------

| Model/Service | Dimensions | Context Length | Latency | Indexation Time | Pricing (per million tokens) |
| --- | --- | --- | --- | --- | --- |
| [Cohere embed-english-v3.0](https://docs.cohere.com/docs/embeddings) | 1024 | 512 | ±170ms | 43s | $0.10 |
| [Cohere embed-english-light-v3.0](https://docs.cohere.com/docs/embeddings) | 384 | 512 | ±160ms | 16s | $0.10 |
| [OpenAI text-embedding-3-small](https://platform.openai.com/docs/guides/embeddings) | 1536 | 8192 | ±460ms | 95s | $0.02 |
| [OpenAI text-embedding-3-large](https://platform.openai.com/docs/guides/embeddings) | 3072 | 8192 | ±750ms | 151s | $0.13 |
| [Mistral](https://mistral.ai/product/embeddings/) | 1024 | 8192 | ±200ms | 409s | $0.10 |
| [VoyageAI voyage-2](https://voyageai.com/) | 1024 | 4000 | ±350ms | 330s | $0.10 |
| [VoyageAI voyage-large-2](https://voyageai.com/) | 1536 | 16000 | ±400ms | 409s | $0.12 |
| [Jina Colbert v2](https://jina.ai/news/jina-colbert-v2-multilingual-late-interaction-retriever-for-embedding-and-reranking/) | 128, 96, or 64 | 8192 | ±400ms | 375s | $0.18 |
| [OSS all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 384 | 512 | ±10ms | 880s | Free |
| [OSS bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | 1024 | 512 | ±20ms | 3379s | Free |
| [OSS bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | 1536 | 512 | ±60ms | 9132s | Free |

Choosing the right model and service for semantic search involves carefully balancing several key factors: relevancy, search performance, indexation performance, and cost.

Each option presents its own set of trade-offs:

*   Cloud-based services like Cohere and OpenAI offer excellent relevancy and reasonable latency, with Cohere's embed-english-light-v3.0 standing out for its balance of speed and performance.
*   Local models provide the fastest search latency but may struggle with indexation speed on limited hardware.
*   Emerging services like Mistral and VoyageAI show promise with competitive pricing and performance.
*   Open-source models offer cost-effective solutions for those willing to manage their own infrastructure.

Ultimately, the best choice depends on your specific use case, budget, and performance requirements. For many applications, starting with a cloud-based service like Cohere or OpenAI provides a good balance of ease of use, performance, and cost. As your needs grow, consider exploring local or specialized models, or contact Meilisearch's sales team for tailored solutions.

* * *

[Meilisearch](https://www.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) is an open-source search engine enabling developers to build state-of-the-art experiences while enjoying simple, intuitive DX.

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison). You can learn more about the product by checking out its [roadmap](https://roadmap.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions).

Choosing the best model for semantic search

_This is part 4 of a series of blog posts_ [_originally published on Clément Renault's blog_](https://blog.kerollmops.com/how-meilisearch-updates-a-millions-vector-embeddings-database-in-under-a-minute)_. Begin the journey with_ [_part 1_](/blog/spotify-inspired-hybrid-search-and-rust/)_,_ [_part 2_](/blog/refining-ann-performance-with-rust/)_ and_ [_part 3_](/blog/arroy-filtered-disk-ann/)_ This blog post considers that you’ve already read parts 1 and 2._

In this blog post, we’ll talk about how we implemented incremental indexing in [Arroy](https://github.com/meilisearch/arroy). By incremental, we mean that when inserting or deleting items in a tree, we can update only the required parts instead of rebuilding everything.

At Meilisearch, that’s essential because our users usually send updates regularly, and their content constantly changes. Rebuilding everything from scratch is very costly and won’t give a good experience; that’s why we already spent a lot of time on our latest release to make all the [data structures support incremental indexing](https://github.com/meilisearch/meiliSearch/releases/tag/v1.6.0) and arroy being one of them, we _HAD TO_ implement it as well!

The theory
----------

The following schema shows a high-level view of what should happen when adding and deleting documents into an existing tree.

As a reminder, arroy is a vector store based on [LMDB](http://www.lmdb.tech/doc/), an embedded transactional memory-mapped key-value store. It stores your vectors in trees composed of three kinds of nodes:

*   The split nodes - Store information on which item IDs are spread out between its left and right.
*   The descendants' nodes - Store multiple item IDs.
*   The items - A single vector.

For the following examples, we’ll consider the descendants node cannot contain **more** than two items and that the closer the IDs are, the closer the items are in space. For example, the item `1` is close to the item `2` but is farther from the item `10`.

![First step before inserting elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_1_9ea29125b9.png)

Here, we’re trying to insert items `2` and `7` and delete items `3` and `11`.

Following what we do on the search, we split the items to insert between the left and right split nodes. But the deleted items are no longer present in the database! We don't have their associated vector, and therefore we'll need to iterate over all the leaves to find and delete them.

Fortunately, since we use roaring bitmaps, this list doesn't consume much memory, and since we never update it, we can share it with every thread without ever copying it 🎉

![Second step inserting element in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_2_e4cd099c77.png)

Notice how the items to insert were balanced between the left and right split nodes. In the next step, we’ll follow the same process and split both lists of items to insert on the split node.

![Third step inserting into elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_3_ab539d2465.png)

It’s on this step that all the good stuff happens. From the left to the right:

*   The first descendant node gets its item `3` removed, and item `2` added.
*   The second descendant node gets too big and must be replaced by a new split node.
*   The item `8` becomes a descendant containing both the items `7` and `8`.
*   After deleting item `10` in the last descendant node, we replace it with a direct link to an item to reduce the number of nodes in the tree (and improve search time by reducing the number of lookups).

![Last step inserting elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_4_b07ebb4b8f.png)

Now that you’ve seen all the steps of our incremental indexing process, we’re still going to take a few notes on what happened:

*   In the first three steps, we had to read all of the tree nodes, which, by default, is equal to the total number of items in the database.
*   In the last step, we had to write four new tree nodes. Previously we would have rewritten the whole tree. The number of writes, which is the slowest operation in databases, is reduced to the bare minimum.
*   The process works on a single tree which means we can still multi-thread the computation of every tree.
*   The IDs aren’t sequential anymore, which doesn't play well with the ID generation strategy described in [Writing the tree in parallel](https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy).

How did we fix the ID generation?
---------------------------------

At the start of the indexing process, we need to gather all the existing tree IDs.  
The information we need to generate new nodes are:

*   The total number of tree nodes used to know when to stop building trees.
*   The « holes » in the used IDs that we want to re-use. This fragmentation is produced when we edit the trees.
*   The highest ID existing in the trees to generate the next new IDs.

![Selecting the next ID in the Roaring Bitmap](https://unable-actionable-car.media.strapiapp.com/tamo_5_1f7bb6a273.png)

The « simple » idea is to have a counter for the total number of tree nodes. A counter that we can increment to generate new fresh IDs. The most challenging part is to pick an available ID shared between threads and without big mutexes.

It took me quite some time to find a solution for this last point; I implemented a whole complex solution (150loc) that I ended up [rewriting entirely from scratch](https://github.com/meilisearch/arroy/pull/56/commits/0bf68e21cfd97ff00c7ef5a0ec116ba34c43c6a5) in one hour for an easy and more efficient solution. We're going to see all the ideas we went through before finding our final solution.

### Sharing an iterator over the available IDs

The first and most straightforward idea that comes to mind is to create and share an iterator over all the available IDs.

```rust
let mut available_ids = available_ids.into_iter();

roots.into_par_iter().for_each(|root| {
    // stuff
    let next_id = available_ids.next();
    // stuff
});
```

If you’re familiar with Rust, you’ll notice immediately that the `for_each` _cannot_ capture a mutable reference over the `available_ids` because we use the `into_par_iter` method from Rayon that execute the next methods on the iterator in a multi-threaded way. Which means we can't call `.next()`.

By using a mutex (short for [mutual exclusion](https://en.wikipedia.org/wiki/Mutual_exclusion)), we can make it compile:

```rust
let mut available_ids = Mutex::new(available_ids.into_iter());

roots.into_par_iter().for_each(|root| {
    // stuff
    let next_id = available_ids.lock().unwrap().next();
    // stuff
});
```

That is safe and will yield the right results, but won’t scale well as all the threads have to wait for each other on the lock.

### Let each thread own its list of IDs

When you need to remove a lock, a common solution is to split the work beforehand so each thread can work to its full capacity without ever having to know what’s happening on the other threads. That’s the solution [I implemented initially](https://github.com/meilisearch/arroy/pull/56/files/dc782ea61ef84f0ad8ad20e092476494d6eacc0c). The idea is to explode the roaring bitmap into as many smaller roaring bitmaps as there are trees to update.

The following function can split a roaring bitmap into `n` sub-bitmaps of equal size:

```rust
pub fn split_ids_in(available: RoaringBitmap, n: usize) -> Vec<RoaringBitmap> {
    // Define the number of elements per sub-bitmap
    let chunk_size = available.len() / n as u64;
    let mut ret = Vec::new();

    let mut iter = available.into_iter();
    for _ in 0..(n - 1) {
        // Create a roaring bitmap of `chunk_size` elements from the iterator without consuming it
        let available: RoaringBitmap = iter.by_ref().take(chunk_size as usize).collect();
        ret.push(available);
    }
    // the last element is going to contain everything remaining
    ret.extend(iter);

    ret
}
```

With this function available, it's then trivial to use one bitmap per tree to update with the [`zip`](https://docs.rs/rayon/latest/rayon/iter/trait.IndexedParallelIterator.html#method.zip) method on the parallel iterators of Rayon:

```rust
let available_ids = split_ids_in(available_ids, roots.len());

roots.into_par_iter().zip(available_ids).for_each(|(root, available_ids)| {
    let mut available_ids = available_ids.into_iter();
    // stuff
    let next_id = available_ids.next();
    // Here, we can use the available_ids without any locks or inter-thread synchronization
    // stuff
});
```

This works well while we're updating the already existing root trees. But later on, we may want to create new trees from scratch, and we can't know in advance how many new trees we’ll need to create. That's an issue because, without this information, we don't know how many sub-bitmaps we need to create.

At this point, I couldn’t see an easy fix to this problem, but I made the assumption that we rarely create a lot of new trees and that all new trees would use a lot of IDs. This means, giving all the IDs to the **first** new tree would _probably work out_ (it’s hard to be 100% sure without monitoring it in prod).

### The final solution

The previous solution was complex and did not even work perfectly. While I was scrolling through the documentation of the Roaring Bitmap, I saw [the `select` method](https://docs.rs/roaring/latest/roaring/bitmap/struct.RoaringBitmap.html#method.select) and immediately understood how I could make the initial approach works.

This method lets you get the values in your bitmap by index in an efficient way.  
For example, with the following values in a bitmap: `13, 14, 15, 98, 99, 100`:

*   `select(0)` returns `13`.
*   `select(3)` returns `98`.
*   `select(5)` returns `100`.
*   `select(6)` returns `None`.

With that in mind, if we share the bitmap with all threads in **read-only** and share the index in the bitmap as an atomic number, we can have multiple threads get available IDs at the same time with lock-free synchronization.

Once the `select` method returns `None`, it means we can stop picking IDs from the bitmap and instead simply generate new IDs from scratch with the old method.  
Even if it's super fast, calling the `select` method still takes some time, so once a thread gets a `None` returned from the method, we're going to update an atomic boolean telling us if there are still values to look at in the bitmap. If not, we can skip calling `select` and directly generate a new ID.

With that in mind, the cool `ConcurrentNodeIds` structure we showed you before in part 2 got a little bit more complex but it still let us generate IDs fairly without any locks!

```rust
/// A concurrent ID generator that will never return the same ID twice.
pub struct ConcurrentNodeIds {
    /// The current tree node ID we should use if no other IDs are available.
    current: AtomicU32,
    /// The total number of tree node IDs used.
    used: AtomicU64,

    /// A list of IDs to exhaust before picking IDs from `current`.
    available: RoaringBitmap,
    /// The current Nth ID to select in the bitmap.
    select_in_bitmap: AtomicU32,
    /// Tells if you should look in the roaring bitmap or if all the IDs are already exhausted.
    look_into_bitmap: AtomicBool,
}

impl ConcurrentNodeIds {
    /// Creates an ID generator returning unique IDs, avoiding the specified used IDs.
    pub fn new(used: RoaringBitmap) -> ConcurrentNodeIds {
        let last_id = used.max().map_or(0, |id| id + 1);
        let used_ids = used.len();
        let available = RoaringBitmap::from_sorted_iter(0..last_id).unwrap() - used;

        ConcurrentNodeIds {
            current: AtomicU32::new(last_id),
            used: AtomicU64::new(used_ids),
            select_in_bitmap: AtomicU32::new(0),
            look_into_bitmap: AtomicBool::new(!available.is_empty()),
            available,
        }
    }

    /// Returns a new unique ID and increases the count of IDs used.
    pub fn next(&self) -> Result<u32> {
        if self.look_into_bitmap.load(Ordering::Relaxed) {
            let current = self.select_in_bitmap.fetch_add(1, Ordering::Relaxed);
            match self.available.select(current) {
                Some(id) => Ok(id),
                None => {
                    self.look_into_bitmap.store(false, Ordering::Relaxed);
                    Ok(self.current.fetch_add(1, Ordering::Relaxed))
                }
            }
        } else {
            Ok(self.current.fetch_add(1, Ordering::Relaxed))
        }
    }
}
```

Performances in practice
------------------------

I haven’t run a lot of benchmarks yet (I would like to, though, and may update this post later), but on my benchmarks with a few hundred thousand items, here are the results I get:

![Performance improvement showed on a graph](https://unable-actionable-car.media.strapiapp.com/tamo_6_8c737d7e95.png)

*   On average, we get more than a 10x speedup after three batches of items indexed.
*   The little bumps we observe afterward that make us go from ~700ms to ~3s are due to the creation of new trees as the number of items increases.
*   I expect this benchmark to represent a worst-case scenario:
    *   All my item IDs are randomly generated, which means there are very few duplicates/updates but always new insertions and more writing.
    *   My vectors are also randomly generated with a uniform distribution, which means there shouldn’t be clusters as we observe in a real-life dataset.

This feature is now being used on production data with millions of documents, and we've seen numbers in the range of 9000 items added on a 2M items database in under a minute.

* * *

[Meilisearch](https://meilisearch.com/?ref=blog.meilisearch.com) is an open-source search engine enabling developers to build state-of-the-art experiences while enjoying simple, intuitive DX. 

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://dub.sh/meili-newsletter). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?ref=blog.meilisearch.com).

How Meilisearch updates a database with millions of vector embeddings in under a minute

On March 7th, we hosted a live showcase session introducing hybrid search and other Meilisearch product updates. Watch a recording and see the hybrid search demo [here](https://www.youtube.com/watch?v=Lgv8iTdWDQc&t=881s).

* * *

AI's influence on search
------------------------

The rapid advancement of AI technology is undeniably reshaping human-machine interactions. Historically, communication with machines necessitated intermediaries like programming languages and structured scripts.

Now, with the arrival of GPTs and AI-assisted bots, machine-learning models are approaching a deeper comprehension of natural human language. This evolution brings humans and machines closer, changing the way we seek and find information.

From keyword-based searches to discovery-driven inquiries, people’s approach to search evolves: we are moving from searching for known items to exploring unknown territories, searching and finding the concepts instead of focusing solely on words.

Take a streaming media company: the searches across their platform might include the inquiry for a specific movie title, such as "Back to the Future," as well as conceptually-driven searches like "Give me recommendations for a feel-good sci-fi movie from the 80s".

Consider everyday scenarios: spotting an item on the street and using an image to search for it, or conducting voice searches instead of typing in words on keyboards. These are manifestations of our changing search behaviors.

The pinnacle: hybrid search
---------------------------

While traditional search methods excel in keyword-based queries, they falter with complex, nuanced requests. This is where Meilisearch’s new hybrid search approach comes into play, enriching the efficiency and precision of our robust full-text search engine with a depth of semantic understanding, catering to a wide array of use cases driven by customer-generated data.

Integrating models from AI giants like [OpenAI](https://www.meilisearch.com/docs/learn/experimental/vector_search#generate-auto-embeddings-with-openai) (open-source embedders generation feature is set for the next release cycle), Meilisearch enables users to create and fine-tune vector embeddings, tailoring their search engine to specific business needs. Moreover, evolving AI models learn from user interactions, continuously improving search precision.

What‘s included in the new release package?
-------------------------------------------

The AI-enhanced hybrid search from Meilisearch leverages our existing robust search infrastructure, including advanced sorting, filtering, faceting, and geo-sorting capabilities, among others.

New release features
--------------------

*   **Enhanced search relevancy**: The AI-enhanced layers bring a deeper semantic understanding, significantly improving result quality.
*   **AI embedders autogeneration:** Choose between using a third-party solution like OpenAI or your own locally generated embeddings. An open-source embedders generation feature is set for the next release cycle.

![](https://unable-actionable-car.media.strapiapp.com/Group_143725866_7f8309846d.png)

Add a new AI embedder in Meilisearch Cloud.

*   **Improved prompting:** Enhanced to incorporate key information templates, aiding the system in prompt ingestion and relevant metadata interpretation.
*   **Semantic ratio control:** Customize your search to span anywhere from purely keyword-based to entirely semantic.  
    

![](https://unable-actionable-car.media.strapiapp.com/Group_143725865_1_f3910ea1ce.png)

Choose where you want your results to fall on the spectrum from pure keyword-based to fully semantic search.

*   **Multimodal search capabilities:** Expanding search functionality to include image, sound, and video content.

**Democratizing search**
------------------------

At Meilisearch, our journey has always been anchored in simplicity, performance, and out-of-the-box relevancy. Our core mission remains steadfast: make it easy to find anything anywhere by providing a go-to-search solution for every team.

> “Meilisearch's hybrid search will democratize advanced search technology, focusing on simplicity and open-source customizability”  
> \- **Quentin de Quelen, Meilisearch’s CEO and Co-founder.**

As we evolve, our methods become more sophisticated, adapting to the ever-changing needs of businesses and developers alike. In a world brimming with data, effective content discovery is essential, not just a luxury. Our vision combines innovation with accessibility, aiming to make cutting-edge search technology inclusive for all.

The launch of our hybrid search product marks a significant milestone for Meilisearch and the search technology landscape. We invite you to try the new solution and join us in shaping the future of search.

* * *

Our AI-powered hybrid search is currently in beta. Eager to try it out? Sign up for our waitlist by using the link below, and our team will be in contact soon.

[Sign up now](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_source=Blog&utm_campaign=HybridSearchRelease)

* * *

Stay in the loop by subscribing to our [newsletter](https://dub.sh/meili-newsletter?&utm_source=blog&utm_campaign=HybridSearchRelease). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_source=Blog&utm_campaign=HybridSearchRelease) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions?utm_source=Blog&utm_campaign=HybridSearchRelease).

For anything else, join our developer community on [Discord](https://discord.com/invite/hTD5hU2yWV?utm_source=Blog&utm_campaign=HybridSearchRelease).

Introducing hybrid search: combining full-text and semantic search for optimal balance

What is a vector?
-----------------

Vectors are mathematical entities that possess both **magnitude** and **direction**. This ability to not only convey how much (magnitude) but also in which direction, makes them powerful tools for representing and manipulating different types of complex data. 

Vectors are often pictured as **arrows** pointing in a certain direction **within a space**, but they can also be translated into numbers. Representing vectors as **numerical values** makes it easier to work with them mathematically. 

Vectors are a broad mathematical concept used for various purposes across multiple fields.

Vector embeddings are a particular application of vectors in the domain of [machine learning](https://en.wikipedia.org/wiki/Machine_learning) and AI.

What is the purpose of vector embeddings?
-----------------------------------------

Machine learning models are designed to uncover patterns and relationships. They **map complex entities**, like text or images, **to points within a** [**vector space**](https://en.wikipedia.org/wiki/Vector_space). We call these vector embeddings. 

Vector embeddings, also called just embeddings, represent non-numeric data in a numerical format while **preserving the semantic meaning and relationships** of these non-numeric entities. Their goal is to allow computational models in machine learning and natural language processing (NLP) to **"understand" similarities and differences** between entities.

What is vector space embedding?
-------------------------------

In a vector space, similar entities are **positioned closely together**, indicating their semantic or contextual similarity. For instance, in the context of word embeddings, words with similar meanings are embedded near each other in the vector space. 

This spatial configuration is what enables embeddings to effectively capture and organize the **semantic relationships** between entities, a concept known as the semantic space.

What is the semantic space?
---------------------------

The semantic space is like a **virtual landscape** that each [large language model](https://en.wikipedia.org/wiki/Large_language_model) (LLM) builds as it trains. During this training phase, the model analyzes huge amounts of data to represent and understand language. The diversity and nature of the information it absorbs play a significant role in **shaping this semantic space**, which in turn influences how the LLM interprets and generates language.

Given the intricate nature of the data to be transformed, vector embeddings require **multidimensional spaces** to encompass the depth of these relationships and nuances. Depending on the **complexity of the features** they are meant to capture and the size of the dataset involved, these spaces can span a wide range, from dozens to thousands of dimensions.

This multi-dimensional space allows algorithms to interpret and process complex data in a way that mirrors human intuition and understanding.

### Illustrating the semantic space

Let’s illustrate the semantic space with a very simple example. Consider a diagram with three axes corresponding to the following semantic properties: `feline`, `juvenile`, and `canine`.

![](https://unable-actionable-car.media.strapiapp.com/semantic_space_56a01e7496.png)

*   On the `feline` axis, we have the `cat`
*   On the `juvenile` axis, the `baby`
*   On the `canine` axis, we have the `dog`

By combining these axes, we can find intersections that give us more specific entities:

*   `Feline` and `juvenile` combined give us the `kitten`
*   `Juvenile` and `canine` combined give us the `puppy`

![](https://unable-actionable-car.media.strapiapp.com/semantic_pets_9d93f8fecc.png)

By assigning numerical values to these properties, we can construct a simple semantic space:

| word   | canine | feline | juvenile |
|--------|--------|--------|----------|
| dog    | 1      | 0      | 0        |
| cat    | 0      | 1      | 0        |
| baby   | 0      | 0      | 1        |
| kitten | 0      | 1      | 1        |
| puppy  | 1      | 0      | 1        |

### Embedding vectors in the semantic space

In reality, **the semantic space is more complex**, and properties are not always clearly defined. We don’t know if this is actually the `canine` property, but it’s correlated to something `canine`, and the `dog` ranks very high on this property. The numbers are not `1` or `0` but some real numbers. This complexity allows for a nuanced understanding of how words and concepts relate to each other. The actual semantic space could look like the following:

| word   | ??canine?? | ??feline?? | ??juvenile?? |
|--------|------------|------------|--------------|
| dog    | 0.95973    | 0.12       | 0.22         |
| cat    | 0.05       | 0.99954    | 0.08         |
| baby   | 0.001      | 0.002      | 0.8973       |
| kitten | 0.02       | 0.9212     | 0.8647       |
| puppy  | 0.897      | 0.04       | 0.8834       |


From these detailed values, vector embeddings are created, capturing the essence of each word in a multidimensional vector, such as `[0.95973, 0.12, 0.22]` for `dog`. These vectors do more than just position words in a space; they build a **detailed network of meanings**, with each aspect designed to reveal a bit of what the word really means. The specific dimensions and what they represent can **vary from one model to another**, reflecting the complexity of the semantic meanings they encapsulate. 

Conclusion
----------

Vector embeddings are numerical representations of complex, non-numerical data. They are generated by machine learning models using vectors of real numbers. Embedded in a high-dimensional space, these vectors encapsulate the original data’s relationships and characteristics in a format suitable for computation. 

[**Vector databases**](/blog/what-is-a-vector-database/?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search like Meilisearch, are the go-to for working with embeddings, as they facilitate **similarity searches**, also known as semantic searches, which allow identification of items that are semantically similar based on their vector representations.

AI Search is coming to Meilisearch Cloud, join the waitlist:

[Join now](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search)

* * *

[Meilisearch](https://meilisearch.com/?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience. 

A long-time actor in keyword search, Meilisearch enables users to address search use cases by building upon AI-powered solutions, not only supporting vector search as a vector store but also by providing hybrid search. This hybrid approach blends full-text search with semantic search, enhancing both the accuracy and comprehensiveness of search results.

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search).

What are vector embeddings?

Understanding vector databases
------------------------------

Vector databases are the go-to for performing searches based on similarity, which plays a key role in **AI-driven applications** like recommending your next favorite movie, identifying someone in a photo, or digging up texts that resonate with your search. At the core of these applications are **vector embeddings**, complex data forms that exceed the storage and retrieval capabilities of traditional databases. 

### The role of vector embeddings

[Vector embeddings](/blog/what-are-vector-embeddings/?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search are a way to **transform complex non-numeric data**, such as words, sentences, or even images, **into a numerical format** while preserving their semantic **meaning** and relationships. 

Embeddings are **multidimensional objects** generated by machine learning models where each dimension represents a different feature or aspect of the data. To properly capture the data complexity, vectors can range from dozens to thousands of dimensions, depending on the size and nature of the data. 

### Vector databases vs traditional databases

This complexity makes **traditional databases**–designed to store structured data in tables– unfit to handle embeddings. The volume and complexity of these vectors, each potentially containing thousands of dimensions, challenge the row-and-column format. This mismatch necessitates alternative storage and retrieval solutions tailored to the requirements of vector data.

This is where **vector databases** like [Meilisearch](https://www.meilisearch.com/?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search) come into play. They are designed to address the unique demands of vector embeddings, facilitating efficient storage and retrieval of the information they contain. In particular, they enable performing **similarity searches**, also called semantic searches, which are central to leveraging embeddings effectively. 

<Callout>
  Learn more about [how Meilisearch built Arroy in Rust](/blog/spotify-inspired-hybrid-search-and-rust/?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search) an open-source vector store.
</Callout>

In other words, vector databases allow us to interact easily and efficiently with vector embeddings, making them essential for applications that require semantic understanding and similarity matching.

What is similarity search?
--------------------------

If we think of **vector embeddings** as stars in vast cosmic constellations, similarity search, or **vector search**, would be like trying to **find the nearest** stars to your current position in space. In practical terms, this means finding the most relevant documents, images, or products based on your search query.

To do so, you need to **measure the distance** between the **query** **vector** and other **vectors in the database**, typically using methods like [cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) or [Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_distance). These are just different techniques for determining how close or distant other data points are from your query, much like gauging the **proximity** of stars in the night sky. 

### The role of machine learning models

However, the **success** of this search isn't just about mathematical calculations; it's highly dependent on the **machine learning model** used to generate and query the vectors. Each vector's meaning is intrinsically tied to the semantic space of the model that created it. Consistency here is crucial, ensuring that all vectors 'speak the same language' and adhere to the same contextual rules, making searches **meaningful and accurate**. That is to say, to achieve relevant search results, it's essential to **use the same model** for generating and querying the embeddings.

**Similarity search** is where vector databases like **Meilisearch** truly shine, as they allow for a wide array of applications such as face recognition, movie recommendations, and personalized content discovery. By allowing users to store vector embeddings alongside their documents, Meilisearch not only facilitates similarity searches but also introduces **hybrid search** capabilities, expanding its potential applications. Through the integration of models from various AI solution providers, Meilisearch enables users to **refine vector embeddings** to better suit their specific needs.

In summary, the ability of these databases to analyze and compare complex data patterns allows for highly relevant and accurate results across diverse fields, enhancing user experiences and operational efficiency.

AI Search is coming to Meilisearch Cloud, join the waitlist:

[Join now](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search)

* * *

[Meilisearch](https://meilisearch.com/?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience. 

A long-time actor in keyword search, Meilisearch enables users to address search use cases building upon AI-powered solutions, not only supporting vector search as a vector store but also by providing hybrid search. This hybrid approach blends full-text search with semantic search, enhancing both the accuracy and comprehensiveness of search results.

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search).

What is a vector database?

_This is part 3 of a series of blog posts_ [_originally published on my personal blog_](https://blog.kerollmops.com/meilisearch-expands-search-power-with-arroy-s-filtered-disk-ann)_._ _Begin the journey with_ [_part 1_](/blog/spotify-inspired-hybrid-search-and-rust/) _and_ [_part 2_](/blog/refining-ann-performance-with-rust/)_._

We can store vectors in [Arroy](https://github.com/meilisearch/arroy) and efficiently compute the search tree nodes, but we still need some features to make it usable in Meilisearch. Our full-text search engine has great support for filtering; selecting subsets of documents is an essential feature to support. For instance, one of our biggest clients requires the ability to filter through over 100 million YouTube video metadata and their associated image embeddings to effectively select videos released within specific time frames, such as a single day or week. This represents the scalability and responsiveness challenges we aim to address with our filtering system, making it a perfect use case to tailor our developments.

Meilisearch supports the following operators on the filterable attributes of your documents: `<`, `<=`, `=`, `!=` `>=`, and `>`. Internally, we extensively [use `RoaringBitmap`s](https://github.com/RoaringBitmap/roaring-rs), which are well-optimized sets of integers that support fast binary operations like unions and intersections. When the engine receives a user request with filters, it first computes the subset of documents from the filter that will be inputted into the search algorithms, the one that ranks documents by quality. This subset is represented by a `RoaringBitmap`.

How was it working before Arroy?
--------------------------------

Ranking only the subset of filtered documents has been working well since 2018, but now that we have a new data structure to search on, we need to look at how to implement it. The engine had already supported the vector store feature for months but was inefficient. We were using an in-memory HNSW and were deserializing the whole data structure in memory, searching for the nearest neighbors of the target vector, which was returning an iterator.

```Rust
fn vector_store(db: Database, subset: &RoaringBitmap, target_vec: &[f32]) -> Vec<DocumentID> {
    // This takes a lot of time and memory.
    let hnsw = db.deserialize_hnsw();

    let mut output = Vec::new();
    for (vec_id, vec, dist) in hnsw.nearest_neighbors(target_vec) {
        let doc_id = db.document_associated_to_vec(vec_id).unwrap();
        if !output.contains(&doc_id) {
          output.push(doc_id);
          if output.len() == 20 { break }
        }
    }

    output
}
```

You may wonder why we are retrieving the document ID associated with the vector ID. Meilisearch supports multiple vectors by document since the beginning of the vector store feature. This is unfortunate because we must look up every vector we are iterating. We need to maintain this lookup table. The iterator can iterate on the whole vector dataset if the subset is small enough, e.g., `document.user_id = 32`. We want the document operations to be atomic and consistent, so we have to store the HNSW on disk and avoid having to maintain synchronization between the LMDB transactions and this vector store data structure. Ho! And the library doesn't support incremental insertion. We must reconstruct the HNSW in memory from scratch every time we insert a single vector.

Integrating Arroy in Meilisearch
--------------------------------

As we worked on updating Meilisearch to include the new vector store, arroy, we tried out mob programming for the first time. This is where we all code together at the same time. It might sound like it would slow us down, but it actually made us super productive! By tackling problems together right as they came up, we got arroy fitted into Meilisearch much faster than if we'd worked alone.

Arroy is different because it doesn’t return an iterator to give back search results. Now, our search engine is smarter and can figure out the exact number of results it needs to give back, even when there are filters to consider. This teamwork improved our search tool and showed us that working together is key when facing big tech challenges.

#### Filtering while searching in Arroy

You can find a description of the arroy internal data structure in [part 1 of this series](https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust). Here is a list of the different kinds of nodes you can find:

*   The item nodes. The original vectors that the user provides. The small points on the left.
*   The normal nodes, also called split planes. They represent hyperplanes that split subsets of item nodes in half.
*   The descendant nodes. They are the tree leaves composed of the item IDs you'll find if you follow a certain normal node path.

![split-plane-combined-schema](https://unable-actionable-car.media.strapiapp.com/spli_plane_b53ab895e8.png)

A typical search will load all the normal nodes in a binary heap associated with an infinite distance. Remember that there are a lot of randomly generated trees, and there are, therefore, a lot of entry points. Ascending distances order the binary heap; the shortest nodes found will be popped first.

In the search algorithm, we pop the nearest item from this heap. If it's a normal node, we fetch the left and right sides of the hyperplane:

*   If we find normal nodes again, we associate the distance from the hyperplane with positive and negative distances from the targeted/queried vector.
*   If we find descendant nodes, we do not push them into the queue but rather directly add them to the potential output list as they represent the nearest vectors we have found.

You probably noticed where I want to go, but this is where magic happens. We modified arroy to store the list of descendants in `RoaringBitmap`s instead of a raw list of integers. This is another improvement compared to the original Spotify library, as those lists weigh less. Doing an intersection with the filtered subset is way easier now.

However, there is always an issue: the vector IDs are not the document IDs, and Meilisearch, after executing the filters, only knows about the documents. Iterating on the lookup table, I talked about before, constructing the final bitmap with all the vector IDs corresponding to the filtered documents would not be efficient when many documents are part of this subset, e.g., `document.user_id != 32`. I do not recommend using an `O(n)` algorithm in the search functions.

#### Using multiple indexes for efficient filtering

Fortunately, we developed a fun feature that wasn't meant to be used that way in arroy: the support for multiple indexes in a single LMDB database. We originally developed the multiple indexes feature to be able only to open a single LMDB database to store different vector types. Yes! In Meilisearch v1.6, you can describe the different embedders that live in a single index. You can identify the different vectors with different numbers of dimensions and distance functions you can store in a single document. Defining multiple vectors for a single document associated with the same embedder is also possible.

The indexes are identified by an `u16`. This feature can trick the algorithm into being even more efficient than the previous HNSW solution. Using one store by embedder and vector in a document is interesting because we can now identify the vectors using the document ID. No more lookup database and vector IDs. The vector IDs are reduced to the document IDs. We can use the output of the filters to filter the arroy indexes.

The search algorithm is different on the Meilisearch side. We request the nearest neighbors in every arroy index, sort the results by document ID to be able to deduplicate them and not return the same document multiple times, sort them again by distance, and then return only the top twenty documents. It may seem complex, but we are talking about twenty documents by the number of vectors for a single document. Usually, users will have a single vector.

```Rust
fn vector_search(
    rtxn: &RoTxn,
    database: Database,
    embedder_index: u8,
    limit: usize,
    candidates: &RoaringBitmap,
    target_vector: &[f32],
) -> Vec<(DocumentId, f32)> {
    // The index represents the embedder index shifted and
    // is later combined with the arroy index. There is an arroy
    // index by vector for a single embedded in a document.
    let index = (embedder_index as u16) << 8;
    let readers: Vec<_> = (0..=u8::MAX)
        .map(|k| index | (k as u16))
        .map_while(|index| arroy::Reader::open(rtxn, index, database).unwrap())
        .collect();

    let mut results = Vec::new();
    for reader in &readers {
        let nns = reader.nns_by_vector(rtxn, target_vector, limit, None, Some(candidates)).unwrap();
        results.extend(nns_by_vector);
    }

    // Documents can have multiple vectors. We store the different vectors
    // into different arroy indexes, we must make sure we don't find the nearest neighbors
    // vectors that correspond to the same document.
    results.sort_unstable_by_key(|(doc_id, _)| doc_id);
    results.dedup_by_key(|(doc_id, _)| doc_id);

    // Sort back the documents by distance
    results.sort_unstable_by_key(|(_, distance)| distance);
    results
}
```

The design philosophy behind these improvements
-----------------------------------------------

We're trying to make Meilisearch flexible enough for everyone's needs. Whether your documents are powered by a single embedding, like the kind OpenAI's nifty tools generate, or mixing it up with text and image embeddings, as many e-commerce sites do, we want your search experience to be seamless. We haven't forgotten you for the power users working with multiple embeddings from multi-modal embedders. Our latest improvements ensure that everyone gets to update and fine-tune their search indexes incrementally, making the whole process as smooth as butter.

Big thanks to everyone: Wrapping this up, a huge shout-out to the squad – [@dureuill](https://github.com/dureuill), [@irevoire](https://github.com/irevoire), and [@ManyTheFish](https://github.com/ManyTheFish) – for their genius and grunt work that brought our ideas to life. And watch out for [@irevoire](https://github.com/irevoire)'s upcoming article, where he'll explain how we've implemented incremental indexing in Arroy—meaning you can add new vectors without rebuilding everything from scratch. More on that soon!

You can comment about this article on [Lobste.rs](https://lobste.rs/s/hdokyh/meilisearch_expands_search_power_with), [Hacker News](https://news.ycombinator.com/), [the Rust Subreddit](https://www.reddit.com/r/rust/comments/18pqxdz/meilisearch_expands_search_power_with_arroys), or [X (formerly Twitter)](https://x.com/kerollmops/status/1738837058628551148?s=46&t=0s6OFYJM-Oyl_WD-1jijQA).

* * *

[Meilisearch](https://meilisearch.com/?ref=blog.meilisearch.com) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience.

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?ref=blog.meilisearch.com).

Meilisearch expands search power with Arroy's Filtered Disk ANN

_This is part 2 of a series of blog posts_ [_originally published on my personal blog_](https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy)_. Begin the journey with_ [_part 1_](/blog/spotify-inspired-hybrid-search-and-rust/)_._

Wouldn't it be great to show you how a single-threaded, memory-mapped key-value store can be more efficient than a hand-written memory-mapped solution? I faced issues while porting the Spotify disk-based Approximate Nearest Neighbors library to Rust and, more specifically, to LMDB. Those issues were primarily due to LMDB and memory safety. Here is the story.

To remind you, [Arroy](https://github.com/meilisearch/arroy) is a library that stores embeddings (vectors of floats) on disk. Some data structures are generated on top of those vectors, which look like trees governed by normals used to recursively split the vectors dataset into subdomains. But you can read more about this in [part 1](/blog/spotify-inspired-hybrid-search-and-rust/ of this series.

### How does Annoy generate the tree nodes?

Annoy, the Spotify library stores the nodes on disk, the item nodes (the ones containing the embeddings), and the other nodes that we will call tree nodes in this article. The advantage of doing this is double:

*   The program's memory usage is low and optimized by the OS as all the nodes live in a memory-mapped file.
*   The concept is simple: Access a node by using its ID. The library will find its offset on disk by using a simple multiplication.

However, there are downsides to doing that. All nodes: items with embeddings along with the tree nodes must have the same size, and If a user wants to identify its embedding using the ID 231, the library will increase the file size to the ID multiplied by the node size. For example, with a vector of 768 dimensions, storing a single item with the ID 231 will generate a file of more than 6.5 TiB.

In generating the final data structure, the tree nodes are all written to disk in the same file as the user item containing the embeddings. Those tree-building processes run in parallel, one running process by tree, and, therefore, requires a lot of synchronization when defining the newly created node ID and the memory area reserved for it, most importantly, when the memory-mapped file is too small to accept more nodes, only a single thread must grow the file, so a mutex is used on top of the mutable memory-map and around a `node_id` variable. One interesting property of tree generation is that the generation process only requires the original user item with embeddings.

### The challenges we encountered when porting it to LMDB

A fun fact is that it is the first time in a long time that I have written C++, and the first time I asked ChatGPT to code for me because I was not confident in doing C++ and feared falling into some segfault. I needed a small program to deserialize embeddings from stdin and give them to Annoy. The chatbot's code was mostly effective, but it omitted a critical [empty vector check](https://gist.github.com/Kerollmops/d0e434016b1a698ec9bdbd86d268885e#file-par_build_tree-cpp-L74), which led to a segmentation fault...

The main obstacle to porting it to LMDB is that writing into this key-value store is single-threaded. It doesn't support concurrent write operations to maintain a correctly balanced BTree. Fun incoming!

#### Reading the user item nodes from different threads

We have used LMDB at Meilisearch since the beginning. It is a well-maintained key-value store used in Firefox and maintained by the OpenLDAP developers. It is memory-mapped and does not maintain a user-end cache of the entries in memory but instead gives you a pointer to the memory-mapped area on disk. The main advantage is that you can keep a pointer to this area for as long as your read transaction lives. Ho yeah! Because it is a transactional key-value store that supports atomic committing, too!

But tree generation doesn't require referring to the generated nodes but only the user items. We previously saw that LMDB gives direct pointers into the memory-mapped file without maintaining any intermediate cache (that could be invalidated). There is another quirk with LMDB: you cannot share a read-only transaction between multiple threads, i.e., `RoTxn: !Sync` and you cannot move the write transaction between threads, i.e., `RwTxn: !Send + !Sync`. Ho! And there is no way to create a read-transaction on uncommitted changes. This is an issue because we want to generate the data-structures trees in the same transaction where we store the items.

But magic is everywhere, starting with the following small and fun data structure. The principle is to keep the pointers to the internal user items with embeddings in a `Vec<*const u8>`. Thanks to Rust, we can ensure, at compile time, that the pointers will live long enough by keeping the lifetime in the struct. Using the `&'t RwTxn` to get the `&'t RoTxn` by using `Deref` also ensures that we cannot modify the database while reading in it by using a `&'t mut RwTxn`. [According to the leading developer of LMDB](https://bugs.openldap.org/show_bug.cgi?id=10138#c9), it is safe to share those pointers between threads and why I implemented `Sync` for this structure.

```Rust
pub struct ImmutableItems<'t, D> {
    item_ids: RoaringBitmap,
    constant_length: Option<usize>,
    offsets: Vec<*const u8>,
    _marker: marker::PhantomData<(&'t (), D)>,
}

impl<'t, D: Distance> ImmutableItems<'t, D> {
    pub fn new(rtxn: &'t RoTxn, database: Database<D>, index: u16) -> heed::Result<Self> {
        let mut item_ids = RoaringBitmap::new();
        let mut constant_length = None;
        let mut offsets = Vec::new();

        for result in database.items()? {
            let (item_id, bytes) = result?;
            assert_eq!(*constant_length.get_or_insert(bytes.len()), bytes.len());
            item_ids.push(item_id);
            offsets.push(bytes.as_ptr());
        }

        Ok(ImmutableLeafs { item_ids, constant_length, offsets, _marker: marker::PhantomData })
    }

    pub fn get(&self, item_id: ItemId) -> heed::Result<Option<Leaf<'t, D>>> {
        let len = match self.constant_length {
            Some(len) => len,
            None => return Ok(None),
        };
        let ptr = match self.item_ids.rank(item_id).checked_sub(1).and_then(|offset| self.offsets.get(offset)) {
            Some(ptr) => *ptr,
            None => return Ok(None),
        };
        let bytes = unsafe { slice::from_raw_parts(ptr, len) };
        ItemCodec::bytes_decode(bytes).map(Some)
    }
}

unsafe impl<D> Sync for ImmutableItems<'_, D> {}
```

You have also probably noticed some other fun optimizations in this simplified version of the data structure. We also know that the user-item nodes have a constant length, so I decided to store it only once, reducing the offsets vector's size by two. Considering that our objective is to store 100M vectors and that this vector is in memory, we shrunk its size from 1526 MiB to 763MiB, which is not much, but better than nothing.

#### Writing the tree nodes in parallel

Ok! Now that we know how to store pointers to items and share them between threads without any user-end synchronization, we need to generate the tree nodes using it. We already know how to deal with LMDB at Meilisearch and decided to implement the same workaround. To write in parallel into LMDB, write into different, independent files and merge everything afterward. This leverages the [Share-Nothing principle](https://en.wikipedia.org/wiki/Shared-nothing_architecture) and isolates the algorithms. This drastically reduces the number of synchronization points [compared to the original C++ code](https://github.com/spotify/annoy/blob/2be37c9e015544be2cf60c431f0cccc076151a2d/src/annoylib.h#L1344-L1445) (look at the `.lock/unlock` calls) to a single line in our code: the atomically increasing global tree node ID.

```Rust
pub fn next_node_id(&self) -> u64 {
    self.0.fetch_add(1, Ordering::Relaxed)
}
```

Our functions that generate normal split nodes based on the user-item nodes are now simply writing the nodes into independent files. The [nodes are appended into a file](https://github.com/meilisearch/arroy/blob/4f193fd534acd357b65bfe9eec4b3fed8ece2007/src/parallel.rs#L20-L71), and the on-disk offsets and bounds are stored into a vector, a single `usize` by node. Using Rayon, we run all tree-generation functions in parallel and, once completed, retrieve the files and boundaries to write the uniquely identified nodes into LMDB sequentially.

### Performances comparison

Our objective at Meilisearch is to support 100M embedding of around 768 dimensions. If we store those as `f32` vectors without any [dimensionality reduction](https://en.wikipedia.org/wiki/Dimensionality_reduction), it would be equivalent to `100M * 768 * 4 = 307B`, in other words, 286 GiB to store the vectors raw, without any internal tree nodes, i.e., no way to search in them efficiently.

If you don't specify the number of trees to generate, the algorithm will continue to create trees while the number of tree nodes is smaller than the number of item vectors. At the end, there must be roughly the same number of tree nodes and items.

#### Discovering the limit of vectors we can index

Arroy and Annoy use memory mapping extensively but in slightly different ways. In a previous article from [@dureuill](https://github.com/dureuill), we saw that [operating systems do not have infinite memory-map space](/blog/dynamic-virtual-address-management/ So, let's dive into performance results.

I noticed something strange when running Arroy with 25M vectors of 768 dimensions. The CPUs were not used, and the program seemed to read the SSD/NVMe a lot, too much 🤔 The tree generation algorithm is splitting the vector space into subdomains with fewer vectors. Still, it must first find the appropriate normal to split the entire domain, and for that, it randomly selects many items. Unfortunately, my machine only has 63 GiB of memory, and indexing those 25M items requires more than 72 Gib. Annoy was also struggling in the same way.

![We cannot index 25M vectors of 768 dimensions with 63 GiB](https://unable-actionable-car.media.strapiapp.com/annoy_arroy_e83f58691c.png)

After investigating, I understood why the whole swap space and memory mapping limits were reached. Not only the item nodes were not only _768 \* 4 bytes_ because we store the norm and other stuff alongside the vectors, but in the case of Arroy, LMDB needs to maintain BTree data structures around the entries, and those are also tacking memory-mapped space. Both programs request random item nodes unavailable in memory, so the OS fetches them from the disk, which takes time. Ho and every single thread is doing that in parallel. CPUs are simply awaiting the disk.

So, after some dichotomy, I found the point where arroy successfully used all of the memory without being bound to the disk speed. It can index 15.625M on a 63 GiB machine. You can see on this htop screenshot that the disk read speed is at zero as all of the vectors fit in RAM, that arroy is writing the tree nodes to disk, and that the CPUs are doing their best. It took less than seven hours to process.

![Successfully indexing 15.6M vector of 768 dimensions with 63 GiB](https://unable-actionable-car.media.strapiapp.com/annoy_arroy2_a873d31d2d.png)

But... Annoy cannot index this same number of documents. It suffers from the same issue we saw before: high disk read and low CPU usage. But why? I needed clarification because the nodes have the same format, the number of item vectors to index is the same, and the algorithm has been ported. So, what is the difference between both solutions?

For those who looked into the C++ codebase and were probably hinted by the memory mapping issue described earlier, you probably noticed this slight difference: Arroy is writing the generated tree nodes into different raw files when Annoy, on the contrary, is [reserving space into the memory-mapped file and directly writing into it](https://github.com/spotify/annoy/blob/2be37c9e015544be2cf60c431f0cccc076151a2d/src/annoylib.h#L1354-L1370). By doing this trick, the OS needs to keep much more space in the memory-mapped area and is forced to invalidate the item nodes from the cache to keep the just-written tree nodes hot in the cache, slowing down the system for no reason, as the tree nodes are not necessary for the algorithm.

So, after even more dichotomy to find the Annoy limits on a 63 GiB machine, I discovered that it could roughly index 10M vectors in five hours.

#### The share nothing principle to the win

So Arroy can index 36% more vectors on the same machine, but how long does it take? Wouldn't it be faster to write into the same file in parallel instead of copying all those tree nodes in a single-threaded way? It will be a much shorter paragraph as I only did some small tests, but Arroy is always faster!

|  | number of vectors | number of threads | building time |
| --- | --- | --- | --- |
| Annoy | 96k | 1 | 5min 38s |
| arroy | 96k | 1 | **3min 45s** |
| Annoy | 96k | 12 | 1min 9s |
| arroy | 96k | 12 | **54s** |
| Annoy | 10M | 12 | 5h 40min |
| arroy | 10M | 12 | **4h 17min** |
| Annoy | 15.625M | 12 | \-- |
| arroy | 15.625M | 12 | **7h 22min** |

Now, you probably tell me that I will need around 400GiB of memory to index 100M vectors, and you are probably right, but [@irevoire](https://github.com/irevoire) will talk about incremental indexing in a future article. I did a linear regression for fun. With those 12 threads and the required memory, I predicted it would take one day, 22 hours, and 43 min 😬. Ho! As we also use this vector store in Meilisearch, we need to provide ways to filter this data structure at search time. This is the next article in this series.

You can comment about this article on [Lobste.rs](https://lobste.rs/s/87aheu/multithreading_memory_mapping_refining), [Hacker News](https://news.ycombinator.com/), [the Rust Subreddit](https://old.reddit.com/r/rust/comments/18jmuoe/multithreading_and_memorymapping_refining_ann), or [X (formerly Twitter)](https://x.com/Kerollmops/status/1735941213516140814).

[Part 3](/blog/arroy-filtered-disk-ann/) of this series is now available, exploring further advancements in our search with Arroy's Filtered Disk ANN.

* * *

[Meilisearch](https://meilisearch.com/?ref=blog.meilisearch.com) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience.

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions).

Multithreading and Memory-Mapping: Refining ANN performance with Arroy

Headquartered in New York and Paris, Hugging Face is an open-source provider of machine learning technologies, allowing users to train, deploy, and share AI models using Hugging Face open-source libraries and the Hub.

**By partnering with Meilisearch Cloud, Hugging Face actively fulfills its visionary commitment to advancing the democratization of AI.**

![](https://unable-actionable-car.media.strapiapp.com/ezgif_com_video_to_gif_1_e697b647e1.gif)

> _“Today, Meilisearch is used by Hugging Face to power the discoverability of 300,000+ AI models, datasets, and demos. It’s really important for AI democratization because if the knowledge is out there, but you can’t access it - what’s the point?” -_ Mishig Davaadorj, software engineer at Hugging Face

Challenge
---------

On its platform, Hugging Face enables users to upload their AI models, datasets, and demos. With a vibrant community of users constantly engaging with the [Hugging Face Hub](http://hf.co) - a platform for sharing and searching for Machine Learning artifacts - the question of discoverability takes center stage.

The Hugging Face Hub hosts over 220,000 AI [models](https://huggingface.co/models) catering to a variety of [machine-learning tasks](https://huggingface.co/tasks), all neatly stored in repositories. These model repositories are designed to make the exploration and utilization of models as seamless as possible. Each AI model in the Hugging Face repository is accompanied by a model card, a project file containing valuable metadata, which plays a crucial role in enhancing discoverability, reproducibility, and sharing. Model cards also provide important information about AI models’ biases and limitations, model descriptions, and training guidelines, and serve as comprehensive guides for users seeking models on the Hub or uploading their own.

Before the introduction of Meilsearch, Hugging Face relied on a simple filtering and keyword search solution. However, it became evident that there was a growing demand for a more flexible and typo-tolerant full-text search solution. This solution needed to ensure higher default relevancy for every search query and leverage the additional attributes and metadata stored within model cards.

Why Hugging Face chose Meilisearch Cloud
----------------------------------------

Before integrating Meilisearch into its ML model repository, Hugging Face had already been utilizing Meilisearch's free-of-charge open-source solution as a search engine for its [open-source libraries](https://huggingface.co/docs)’ documentation, including transformers & diffusers. This documentation comprised approximately 500 pages and had been in use for over a year. Because the team had already gained positive experiences from implementing and operating Meilisearch, when the need for enhanced discoverability in model card searches arose, there was no requirement for additional testing or proof of concept.  

**1\. Customizability of the ranking rules**

During the evaluation process, the Hugging Face team also considered Mongo Atlas Search. However, they weren't satisfied with the customizations available for ranking. Several factors were taken into account in order to enhance model card discoverability, such as project names, descriptions, and the number of likes or downloads on each card. Meilisearch has demonstrated greater flexibility and adaptability in accommodating these search criteria.

**2\. Transitioning to Meilisearh Cloud for its ease of use**

Having achieved successful implementation of Meilisearch in its documentation, the Hugging Face team smoothly transitioned to incorporating Meilisearch Cloud into their model cards repository, benefiting from out-of-the-box relevancy.

**3\. Quality of support and infrastructure outsourcing**

The Hugging Face team chose Meilisearch Cloud, in part due to its dedicated support capabilities. Given that achieving highly relevant search results within the repositories is of utmost importance to Hugging Face, the decision to opt for the Cloud version was further influenced by entrusting the infrastructure to the expertise of Meilisearch Cloud specialists. Outsourcing the infrastructure allows the Hugging Face team to enhance their development speed.

Implementation
--------------

With Meilisearch already implemented for Hugging Face documentation, the expansion of the search solution to other use cases was smooth, once the ranking and internal rules were in place.

Today, the Meilisearch engine powers the discovery of 220,000 model cards, 38,000 datasets, and 60,000 demos in the Hugging Face repository. The keyword filtering mechanism has already been implemented on the Hugging Face homepage for users who are already familiar with specific model names.

However, with the implementation of Meilisearch, users are given the option to perform full-text searches as well. This search functionality operates not only on the model names and IDs but also includes the entire content of model cards.

The additional button to “Try full-text search” was implemented in order to minimize changes in the search behavior, where users are free to choose the search experience that most fits their preference.

![](https://unable-actionable-car.media.strapiapp.com/ezgif_com_video_to_gif_bce9ae142e.gif)

Vision
------

With the ongoing evolution of AI, Hugging Face anticipates a shift in search behavior and the methods people use to find the projects they need. Hugging Face is looking forward to incorporating Semantic Search into its documentation and model card searches. Currently, the platform is actively testing Meilisearch [VectorDB](/blog/langchain-semantic-search-tutorial/ as one of the potential solutions.

Hugging Face also anticipates gaining more detailed usage insights, encompassing information about users' daily search patterns and the level of activity certain model card projects receive over time.

Meilisearch latest news and company updates