🚀 Announcing Meilisearch 1.7
-----------------------------

[![](https://unable-actionable-car.media.strapiapp.com/138_84d37ea815.jpg)](/blog/meilisearch-1-7/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

Meilisearch 1.7 **stabilizes ranking score details**, adds **GPU support for Hugging Face embeddings**, and integrates the **latest OpenAI embedding models**. It’s time to upgrade. 👇

[Read release notes](/blog/meilisearch-1-7/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

⚖️ Introducing hybrid search
----------------------------

[![](https://unable-actionable-car.media.strapiapp.com/Group_143725858_1cf2d7de65.png)](/blog/introducing-hybrid-search/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

Meilisearch 1.7 introduced new OpenAI embedders, adding to Meilisearch's **AI-enhanced hybrid search capabilities**. Read the blog post to learn more about hybrid search and [**sign up to our waiting list**](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24).

[Read announcement](/blog/introducing-hybrid-search/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

🍿 Watch our launch event
-------------------------

On March 7th, we discussed our roadmap, our recent releases and **showcased** Meilisearch Cloud’s new features, including **monitoring metrics** and **hybrid search**.

[Watch it!](https://youtu.be/Lgv8iTdWDQc?si=GOZ7HiV0jM0KJQ7y)

🤔 Full-text search vs vector search
------------------------------------

[![](https://unable-actionable-car.media.strapiapp.com/fts_vs_vs_71fd339c9c.jpg)](/blog/full-text-search-vs-vector-search/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

Join [@StriftCodes](https://twitter.com/StriftCodes) in this **comparative analysis** as he delves into the pros and cons of each solution and how **hybrid search** might offer the best of both worlds.

[Read blog post](/blog/full-text-search-vs-vector-search/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

* * *

🖇️ Links, links, links
-----------------------

### 🟧 [Laravel multitenancy guide](https://www.meilisearch.com/docs/learn/cookbooks/laravel_multitenancy?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

Implement search in a multitenant Laravel application.

### 💻 [Computing Hugging Face embeddings with the GPU](https://www.meilisearch.com/docs/learn/cookbooks/computing_hugging_face_embeddings_gpu?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

Learn to compile a Meilisearch binary that leverages an Nvidia GPU to generate Hugging Face embeddings.

### 🔢 [From ranking to scoring](/blog/from-ranking-to-scoring/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

Our journey to add relevancy scores to search results in Meilisearch.

### 🪼 [Meilisearch v1.8 release candidate](https://github.com/meilisearch/meilisearch/releases/tag/v1.8.0-rc.0)

Get a preview of what’s coming next (and give your feedback!)

### 🚀 [Strapi v4 guide](https://www.meilisearch.com/docs/learn/cookbooks/strapi_v4?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

Learn how to integrate Meilisearch with Strapi.

* * *

🥰 Made with Meilisearch
------------------------

The folks at **Blazity** have created this impressive **Next.js storefront** designed for high-performance eCommerce, featuring AI capabilities and one-click deployment.

[![](https://unable-actionable-car.media.strapiapp.com/blazity_e97c54672a.png)](https://github.com/Blazity/enterprise-commerce)

_We’re always looking to promote new projects from the community. Post yours in the_ [_#built-with-meilisearch_](https://discord.meilisearch.com/?ref=blog.meilisearch.com) _Discord channel!_

🏡 Life at Meili
----------------

Join our Head of Engineering, [@clurquizar](https://twitter.com/clurquizar), as she walks us through key points of Meilisearch’s history and our evolution as an open-source company.

[![](https://unable-actionable-car.media.strapiapp.com/clem_itw_af4c14f089.png)](/blog/how-meilisearch-became-open-source-a-retrospective/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

[Read blog post](/blog/how-meilisearch-became-open-source-a-retrospective/?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

* * *

🦸 We're hiring!
----------------

See our open positions:

*   [Growth Marketing Manager](https://www.meilisearch.com/careers?utm_campaign=newsletter&utm_source=blog&utm_medium=march-updates-24)

* * *

Get these monthly updates straight to your inbox by subscribing to our [newsletter](https://dub.sh/meili-newsletter).

For more things Meilisearch, join our developers community on [Discord](https://discord.meilisearch.com/?ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

Until next time!

Meilisearch March Updates

_This is part 4 of a series of blog posts_ [_originally published on Clément Renault's blog_](https://blog.kerollmops.com/how-meilisearch-updates-a-millions-vector-embeddings-database-in-under-a-minute)_. Begin the journey with_ [_part 1_](/blog/spotify-inspired-hybrid-search-and-rust/)_,_ [_part 2_](/blog/refining-ann-performance-with-rust/)_ and_ [_part 3_](/blog/arroy-filtered-disk-ann/)_ This blog post considers that you’ve already read parts 1 and 2._

In this blog post, we’ll talk about how we implemented incremental indexing in [Arroy](https://github.com/meilisearch/arroy). By incremental, we mean that when inserting or deleting items in a tree, we can update only the required parts instead of rebuilding everything.

At Meilisearch, that’s essential because our users usually send updates regularly, and their content constantly changes. Rebuilding everything from scratch is very costly and won’t give a good experience; that’s why we already spent a lot of time on our latest release to make all the [data structures support incremental indexing](https://github.com/meilisearch/meiliSearch/releases/tag/v1.6.0) and arroy being one of them, we _HAD TO_ implement it as well!

The theory
----------

The following schema shows a high-level view of what should happen when adding and deleting documents into an existing tree.

As a reminder, arroy is a vector store based on [LMDB](http://www.lmdb.tech/doc/), an embedded transactional memory-mapped key-value store. It stores your vectors in trees composed of three kinds of nodes:

*   The split nodes - Store information on which item IDs are spread out between its left and right.
*   The descendants' nodes - Store multiple item IDs.
*   The items - A single vector.

For the following examples, we’ll consider the descendants node cannot contain **more** than two items and that the closer the IDs are, the closer the items are in space. For example, the item `1` is close to the item `2` but is farther from the item `10`.

![First step before inserting elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_1_9ea29125b9.png)

Here, we’re trying to insert items `2` and `7` and delete items `3` and `11`.

Following what we do on the search, we split the items to insert between the left and right split nodes. But the deleted items are no longer present in the database! We don't have their associated vector, and therefore we'll need to iterate over all the leaves to find and delete them.

Fortunately, since we use roaring bitmaps, this list doesn't consume much memory, and since we never update it, we can share it with every thread without ever copying it 🎉

![Second step inserting element in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_2_e4cd099c77.png)

Notice how the items to insert were balanced between the left and right split nodes. In the next step, we’ll follow the same process and split both lists of items to insert on the split node.

![Third step inserting into elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_3_ab539d2465.png)

It’s on this step that all the good stuff happens. From the left to the right:

*   The first descendant node gets its item `3` removed, and item `2` added.
*   The second descendant node gets too big and must be replaced by a new split node.
*   The item `8` becomes a descendant containing both the items `7` and `8`.
*   After deleting item `10` in the last descendant node, we replace it with a direct link to an item to reduce the number of nodes in the tree (and improve search time by reducing the number of lookups).

![Last step inserting elements in the tree](https://unable-actionable-car.media.strapiapp.com/tamo_4_b07ebb4b8f.png)

Now that you’ve seen all the steps of our incremental indexing process, we’re still going to take a few notes on what happened:

*   In the first three steps, we had to read all of the tree nodes, which, by default, is equal to the total number of items in the database.
*   In the last step, we had to write four new tree nodes. Previously we would have rewritten the whole tree. The number of writes, which is the slowest operation in databases, is reduced to the bare minimum.
*   The process works on a single tree which means we can still multi-thread the computation of every tree.
*   The IDs aren’t sequential anymore, which doesn't play well with the ID generation strategy described in [Writing the tree in parallel](https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy).

How did we fix the ID generation?
---------------------------------

At the start of the indexing process, we need to gather all the existing tree IDs.  
The information we need to generate new nodes are:

*   The total number of tree nodes used to know when to stop building trees.
*   The « holes » in the used IDs that we want to re-use. This fragmentation is produced when we edit the trees.
*   The highest ID existing in the trees to generate the next new IDs.

![Selecting the next ID in the Roaring Bitmap](https://unable-actionable-car.media.strapiapp.com/tamo_5_1f7bb6a273.png)

The « simple » idea is to have a counter for the total number of tree nodes. A counter that we can increment to generate new fresh IDs. The most challenging part is to pick an available ID shared between threads and without big mutexes.

It took me quite some time to find a solution for this last point; I implemented a whole complex solution (150loc) that I ended up [rewriting entirely from scratch](https://github.com/meilisearch/arroy/pull/56/commits/0bf68e21cfd97ff00c7ef5a0ec116ba34c43c6a5) in one hour for an easy and more efficient solution. We're going to see all the ideas we went through before finding our final solution.

### Sharing an iterator over the available IDs

The first and most straightforward idea that comes to mind is to create and share an iterator over all the available IDs.

```rust
let mut available_ids = available_ids.into_iter();

roots.into_par_iter().for_each(|root| {
    // stuff
    let next_id = available_ids.next();
    // stuff
});
```

If you’re familiar with Rust, you’ll notice immediately that the `for_each` _cannot_ capture a mutable reference over the `available_ids` because we use the `into_par_iter` method from Rayon that execute the next methods on the iterator in a multi-threaded way. Which means we can't call `.next()`.

By using a mutex (short for [mutual exclusion](https://en.wikipedia.org/wiki/Mutual_exclusion)), we can make it compile:

```rust
let mut available_ids = Mutex::new(available_ids.into_iter());

roots.into_par_iter().for_each(|root| {
    // stuff
    let next_id = available_ids.lock().unwrap().next();
    // stuff
});
```

That is safe and will yield the right results, but won’t scale well as all the threads have to wait for each other on the lock.

### Let each thread own its list of IDs

When you need to remove a lock, a common solution is to split the work beforehand so each thread can work to its full capacity without ever having to know what’s happening on the other threads. That’s the solution [I implemented initially](https://github.com/meilisearch/arroy/pull/56/files/dc782ea61ef84f0ad8ad20e092476494d6eacc0c). The idea is to explode the roaring bitmap into as many smaller roaring bitmaps as there are trees to update.

The following function can split a roaring bitmap into `n` sub-bitmaps of equal size:

```rust
pub fn split_ids_in(available: RoaringBitmap, n: usize) -> Vec<RoaringBitmap> {
    // Define the number of elements per sub-bitmap
    let chunk_size = available.len() / n as u64;
    let mut ret = Vec::new();

    let mut iter = available.into_iter();
    for _ in 0..(n - 1) {
        // Create a roaring bitmap of `chunk_size` elements from the iterator without consuming it
        let available: RoaringBitmap = iter.by_ref().take(chunk_size as usize).collect();
        ret.push(available);
    }
    // the last element is going to contain everything remaining
    ret.extend(iter);

    ret
}
```

With this function available, it's then trivial to use one bitmap per tree to update with the [`zip`](https://docs.rs/rayon/latest/rayon/iter/trait.IndexedParallelIterator.html#method.zip) method on the parallel iterators of Rayon:

```rust
let available_ids = split_ids_in(available_ids, roots.len());

roots.into_par_iter().zip(available_ids).for_each(|(root, available_ids)| {
    let mut available_ids = available_ids.into_iter();
    // stuff
    let next_id = available_ids.next();
    // Here, we can use the available_ids without any locks or inter-thread synchronization
    // stuff
});
```

This works well while we're updating the already existing root trees. But later on, we may want to create new trees from scratch, and we can't know in advance how many new trees we’ll need to create. That's an issue because, without this information, we don't know how many sub-bitmaps we need to create.

At this point, I couldn’t see an easy fix to this problem, but I made the assumption that we rarely create a lot of new trees and that all new trees would use a lot of IDs. This means, giving all the IDs to the **first** new tree would _probably work out_ (it’s hard to be 100% sure without monitoring it in prod).

### The final solution

The previous solution was complex and did not even work perfectly. While I was scrolling through the documentation of the Roaring Bitmap, I saw [the `select` method](https://docs.rs/roaring/latest/roaring/bitmap/struct.RoaringBitmap.html#method.select) and immediately understood how I could make the initial approach works.

This method lets you get the values in your bitmap by index in an efficient way.  
For example, with the following values in a bitmap: `13, 14, 15, 98, 99, 100`:

*   `select(0)` returns `13`.
*   `select(3)` returns `98`.
*   `select(5)` returns `100`.
*   `select(6)` returns `None`.

With that in mind, if we share the bitmap with all threads in **read-only** and share the index in the bitmap as an atomic number, we can have multiple threads get available IDs at the same time with lock-free synchronization.

Once the `select` method returns `None`, it means we can stop picking IDs from the bitmap and instead simply generate new IDs from scratch with the old method.  
Even if it's super fast, calling the `select` method still takes some time, so once a thread gets a `None` returned from the method, we're going to update an atomic boolean telling us if there are still values to look at in the bitmap. If not, we can skip calling `select` and directly generate a new ID.

With that in mind, the cool `ConcurrentNodeIds` structure we showed you before in part 2 got a little bit more complex but it still let us generate IDs fairly without any locks!

```rust
/// A concurrent ID generator that will never return the same ID twice.
pub struct ConcurrentNodeIds {
    /// The current tree node ID we should use if no other IDs are available.
    current: AtomicU32,
    /// The total number of tree node IDs used.
    used: AtomicU64,

    /// A list of IDs to exhaust before picking IDs from `current`.
    available: RoaringBitmap,
    /// The current Nth ID to select in the bitmap.
    select_in_bitmap: AtomicU32,
    /// Tells if you should look in the roaring bitmap or if all the IDs are already exhausted.
    look_into_bitmap: AtomicBool,
}

impl ConcurrentNodeIds {
    /// Creates an ID generator returning unique IDs, avoiding the specified used IDs.
    pub fn new(used: RoaringBitmap) -> ConcurrentNodeIds {
        let last_id = used.max().map_or(0, |id| id + 1);
        let used_ids = used.len();
        let available = RoaringBitmap::from_sorted_iter(0..last_id).unwrap() - used;

        ConcurrentNodeIds {
            current: AtomicU32::new(last_id),
            used: AtomicU64::new(used_ids),
            select_in_bitmap: AtomicU32::new(0),
            look_into_bitmap: AtomicBool::new(!available.is_empty()),
            available,
        }
    }

    /// Returns a new unique ID and increases the count of IDs used.
    pub fn next(&self) -> Result<u32> {
        if self.look_into_bitmap.load(Ordering::Relaxed) {
            let current = self.select_in_bitmap.fetch_add(1, Ordering::Relaxed);
            match self.available.select(current) {
                Some(id) => Ok(id),
                None => {
                    self.look_into_bitmap.store(false, Ordering::Relaxed);
                    Ok(self.current.fetch_add(1, Ordering::Relaxed))
                }
            }
        } else {
            Ok(self.current.fetch_add(1, Ordering::Relaxed))
        }
    }
}
```

Performances in practice
------------------------

I haven’t run a lot of benchmarks yet (I would like to, though, and may update this post later), but on my benchmarks with a few hundred thousand items, here are the results I get:

![Performance improvement showed on a graph](https://unable-actionable-car.media.strapiapp.com/tamo_6_8c737d7e95.png)

*   On average, we get more than a 10x speedup after three batches of items indexed.
*   The little bumps we observe afterward that make us go from ~700ms to ~3s are due to the creation of new trees as the number of items increases.
*   I expect this benchmark to represent a worst-case scenario:
    *   All my item IDs are randomly generated, which means there are very few duplicates/updates but always new insertions and more writing.
    *   My vectors are also randomly generated with a uniform distribution, which means there shouldn’t be clusters as we observe in a real-life dataset.

This feature is now being used on production data with millions of documents, and we've seen numbers in the range of 9000 items added on a 2M items database in under a minute.

* * *

[Meilisearch](https://meilisearch.com/?ref=blog.meilisearch.com) is an open-source search engine enabling developers to build state-of-the-art experiences while enjoying simple, intuitive DX. 

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://dub.sh/meili-newsletter). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?ref=blog.meilisearch.com).

How Meilisearch updates a database with millions of vector embeddings in under a minute

Navigating today’s ocean of information can be both a marvel and a maze. Full-text and vector search enable us to build search experiences empowering users to find relevant products, content, and more. As our quest for precision and context in search evolves, a question arises: can we balance the lexical flexibility of full-text search with the semantic depth of vector search?

Let’s explore together the pros and cons of each solution and discover the synergies that are redefining modern search & discovery experiences.

Full-text search
----------------

[Full-text search](https://www.meilisearch.com/docs/learn/what_is_meilisearch/overview?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector#features) refers to matching some or all of a text query with documents stored in a database. Compared to traditional database queries, full-text search provides results even in case of partial matches. It allows building more flexible search interfaces for users, thus enabling them to find accurate results more quickly.

In practice, an efficient full-text search solution comes with tolerance to typos, synonyms, prefix search, and fuzzy matching. These features enable end users to find what they’re looking for even when the information at their disposal is limited. Allowing searches with incomplete information makes discovery faster.

Although modern SQL and NoSQL databases come with some full-text search capabilities, they often fall short compared to search-focused databases. For a deeper dive on this topic, consider reading on areas where [Postgres falls short for full-text search](/blog/postgres-full-text-search-limitations/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector

<Callout>
  Learn about the [inner workings of a search engine](/blog/how-full-text-search-engines-work/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) in our dedicated article.
</Callout>

Vector search
-------------

[Vector search](https://www.meilisearch.com/solutions/vector-search?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) is the AI-powered approach to search. Instead of looking for documents matching a text query, it allows finding documents that have a similar _semantic_ meaning. This is achieved by building upon the semantic understanding of text that [Large Language Models](https://en.wikipedia.org/wiki/Large_language_model) (LLMs) provide.

LLMs can process database records and generate [vector embeddings](/blog/what-are-vector-embeddings/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector)—the numeric representation of the semantic meaning of documents. These embeddings are then stored in [vector databases](/blog/what-is-a-vector-database/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector As the name suggests, vector search works by comparing similar vectors.

By performing vector comparisons, vector search effectively enables [semantic search](/blog/what-is-a-vector-database/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector#what-is-similarity-search Despite relying on different underlying technologies, it also solves a similar purpose as full-text search: empowering users by improving the discovery aspect of search. It notably enables multi-modal search—searching with non-textual input, like in Google Images.

Comparative analysis: full-text search vs vector search
-------------------------------------------------------

Full-text search works by finding words in documents. That makes it good at understanding the **lexical intent** behind a user query. But it falls short when it has to work with vague queries that cannot be fuzzy-matched. The lack of semantic understanding of documents makes it unable to understand context and make generalizations.

To summarize, **full-text search**:

*   ✅ Corrects typos — e.g. `reutrn of the jedi`
*   ✅ Handles precise queries — e.g. `an exact product name`
*   ✅ Handles incomplete queries — e.g. `return of the j`
*   ❌ Cannot handle vague queries — e.g. `folks fighting with lightsabers`
*   ❌ Cannot understand context — e.g. `winter clothes`

On the other hand, vector search works by finding documents with similar meanings. That makes it good at understanding the **semantic intent** behind a user query. It’s great at understanding synonyms and descriptions of documents. It’s also adapted to the discovery of similar documents.

To summarize, **vector search**:

*   ✅ Handles vague queries — e.g. first released star wars movie
*   ✅ Understands context — e.g. winter clothes
*   ✅ Can suggest similar documents
*   ❌ Cannot handle precise queries
*   👎 Requires more resources

As we can see, neither solution is perfect. They each come with clear advantages for given use cases. They complement each other. Building state-of-the-art search experiences would require combining the benefits of both full-text and vector search.

![“Why don't we have both?“ meme](https://unable-actionable-car.media.strapiapp.com/why_dont_we_have_both_1e31f67a5c.gif)

Indeed, can we have both?

The case for hybrid search
--------------------------

Hybrid search combines the strengths of full-text search and vector search. It builds upon the accessible, search-as-you-type experience of full-text search and integrates the enhanced discovery capabilities that AI search enables.

Solutions like Meilisearch’s [hybrid search](https://www.meilisearch.com/solutions/hybrid-search?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) bring the best of both worlds without compromise. This allows developers to use a unified API for all types of searches. The code sample below demonstrates a usage of such API.

```js
// Hybrid search with Meilisearch

const results = await client.multiSearch({
	queries: [{ 
		indexUid: 'movies', 
		q: 'batman',
		hybrid: { embedder: 'default', semanticRatio: 0.5 }
	}]
})

```
<Callout>
  Not using JavaScript? Meilisearch supports [multiple SDKs](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector).
</Callout>

This code uses a Meilisearch client to search for the `batman` query in the movies index. It uses hybrid search, which is configured via the `hybrid` parameter. Let’s review the fields in the `hybrid` object and see what they enable.

The `embedder` field allows choosing which embedder will process the query (Meilisearch allows [configuring multiple embedders](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector#generate-vector-embeddings) for a given index.)

The `semanticRatio` field allows us to balance the importance of results coming from semantic search. A higher value will allow documents that are further distanced in the [semantic space](/blog/what-are-vector-embeddings/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector#what-is-the-semantic-space to make it to the final search results.

The `semanticRatio` value works as follows:

*   Semantic ratio = 0 — it’s a full-text search
*   Semantic ratio = 1 — it’s a vector search
*   0 < semantic ratio < 1 — it’s a hybrid search ✨

As semantic search will always return _some_ documents, it’s important to strike a good balance to keep the overall search results relevant. The demo below allows you to search for movies and play with the semantic ratio to see how it affects results.

[![Searching for movies using hybrid search](https://unable-actionable-car.media.strapiapp.com/hybrid_search_gif_cfa6283c31.gif)](https://where2watch.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector)

Searching for ”green ogre in fairy tale parody” ([Live demo](https://where2watch.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector))

* * *

Upd: Hybrid search is now generally available on Meilisearch Cloud.

[Try it now](https://www.meilisearch.com/cloud)

Conclusion
----------

As engineers will tell you, the question of “What is the best tool for X?” is always: _it depends_. On their own, neither full-text search nor vector search can meet all the criteria to build a fast, relevant search experience. Hybrid search merges the accessibility of full-text search with improved discovery that AI enables.

**Hybrid search** is the modern approach to search, unifying behind a single API the most advanced search capabilities. This hybrid approach blends full-text search with semantic search, enhancing both the accuracy and comprehensiveness of search results. It is a modern, flexible solution adapted to today’s search use cases.

* * *

[Meilisearch](https://meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) is an open-source search engine enabling developers to build state-of-the-art experiences while enjoying simple, intuitive DX. 

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector). You can learn more about the product by checking out its [roadmap](https://roadmap.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions).

Full-text search vs vector search

We're excited to unveil Meilisearch v1.7. In this article, we’ll review the most impactful changes. For an exhaustive listing, check out the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0) on Github.

<Callout>
  Meilisearch 1.7 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog) too—upgrade now!
</Callout>

Improved hybrid & vector search
-------------------------------

<Callout>
  AI search is an experimental feature, read the docs to [enable AI search](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-7&utm_source=blog#activate-vector-search).
</Callout>

### New OpenAI embedding models

Starting with Meilisearch 1.7, you can use OpenAI’s third-generation embedding models:

*   `text-embedding-3-small`
*   `text-embedding-3-large`

Compared to the previous `text-embedding-ada-002` model, these new models offer improvements allowing to balance search results relevancy and cost efficiency.

The `small` model is more cost-effective, generating smaller vector embeddings that require less storage. The `large` focuses on delivering higher precision in search results, making it ideal to optimize relevancy.

### Configuring OpenAI model dimensions

When using the new `text-embedding-3-small` and `text-embedding-3-large` OpenAI models, you can now configure **dimensions** for your model. When not provided, the model's defaults dimensions are used.

You can do so when updating your settings:

```json
{
  // other settings...

  "embedders": {
    "small": {
      "source": "openAi",
      "model": "text-embedding-3-large",
      "dimensions": 512 // must be >0, must be <= 3072 for "text-embedding-3-large"
    },
  }
}

```

Higher dimensions typically offer more complex embeddings which can lead to more accurate search results. However, increased dimensions are more costly to compute and require more storage.

<Callout>
  You cannot customize dimensions when using OpenAI `text-embedding-ada-002`.
</Callout>

### Use AI Search on Meilisearch Cloud

Meilisearch hybrid search and vector search are coming to Meilisearch Cloud. We're rolling out invites every day. Join the beta. 👇

[Get early access](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_campaign=release-v1-7&utm_source=blog)

GPU support for computing Hugging Face embeddings
-------------------------------------------------

Meilisearch 1.7 comes with the ability to enable GPU support for [self-hosted Meilisearch](https://www.meilisearch.com/docs/learn/getting_started/installation?utm_campaign=release-v1-7&utm_source=blog) for computing Hugging Face embeddings. This can significantly speed up the embedding generation process.

Find detailed instructions in our dedicated guide on [Using GPU to compute Hugging Face embeddings](https://www.meilisearch.com/docs/learn/cookbooks/computing_hugging_face_embeddings_gpu?utm_campaign=release-v1-7&utm_source=blog).

Display ranking scores details at search
----------------------------------------

Meilisearch 1.7 stabilizes the `showRankingScoreDetails` search parameter that was introduced in [Meilisearch 1.3](/blog/v1-3-release/?utm_campaign=release-v1-7&utm_source=blog This parameter adds a `_rankingScoreDetails` field to each document in search results, enabling further inspection of the search results.

You can include the parameter when making searches:

```sh
curl \
  -X POST 'https://edge.meilisearch.com/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "batman",
    "showRankingScoreDetails": true
  }'

```

Learn more about the [`_rankingScoreDetails` object](https://www.meilisearch.com/docs/reference/api/search#ranking-score-details?utm_campaign=release-v1-7&utm_source=blog) and how to use it with your preferred SDK in the API reference.

Contributors shout-out
----------------------

We want to give a massive thank you to all contributors that made this release possible: [@capJavert](https://github.com/capJavert), [@codesmith-emmy](https://github.com/codesmith-emmy), [@Elliot67](https://github.com/Elliot67) and [@Gosti](https://github.com/Gosti) for their PRs in Meilisearch, and [@agourlay](https://github.com/agourlay), [@choznerol](https://github.com/choznerol), [@ngdbao](https://github.com/ngdbao), [@timvisee](https://github.com/timvisee), and [@xshadowlegendx](https://github.com/xshadowlegendx) for their work on [charabia](https://github.com/meilisearch/charabia).

Special thanks to our SDKs maintainers, thanks to whom Meilisearch is available across so many languages. 🫶

* * *

And that’s a wrap for v1.7! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0).

Stay in the loop of everything Meilisearch by subscribing to our [newsletter](https://dub.sh/meili-newsletter). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog).

Meilisearch 1.7

On March 7th, we hosted a live showcase session introducing hybrid search and other Meilisearch product updates. Watch a recording and see the hybrid search demo [here](https://www.youtube.com/watch?v=Lgv8iTdWDQc&t=881s).

* * *

AI's influence on search
------------------------

The rapid advancement of AI technology is undeniably reshaping human-machine interactions. Historically, communication with machines necessitated intermediaries like programming languages and structured scripts.

Now, with the arrival of GPTs and AI-assisted bots, machine-learning models are approaching a deeper comprehension of natural human language. This evolution brings humans and machines closer, changing the way we seek and find information.

From keyword-based searches to discovery-driven inquiries, people’s approach to search evolves: we are moving from searching for known items to exploring unknown territories, searching and finding the concepts instead of focusing solely on words.

Take a streaming media company: the searches across their platform might include the inquiry for a specific movie title, such as "Back to the Future," as well as conceptually-driven searches like "Give me recommendations for a feel-good sci-fi movie from the 80s".

Consider everyday scenarios: spotting an item on the street and using an image to search for it, or conducting voice searches instead of typing in words on keyboards. These are manifestations of our changing search behaviors.

The pinnacle: hybrid search
---------------------------

While traditional search methods excel in keyword-based queries, they falter with complex, nuanced requests. This is where Meilisearch’s new hybrid search approach comes into play, enriching the efficiency and precision of our robust full-text search engine with a depth of semantic understanding, catering to a wide array of use cases driven by customer-generated data.

Integrating models from AI giants like [OpenAI](https://www.meilisearch.com/docs/learn/experimental/vector_search#generate-auto-embeddings-with-openai) (open-source embedders generation feature is set for the next release cycle), Meilisearch enables users to create and fine-tune vector embeddings, tailoring their search engine to specific business needs. Moreover, evolving AI models learn from user interactions, continuously improving search precision.

What‘s included in the new release package?
-------------------------------------------

The AI-enhanced hybrid search from Meilisearch leverages our existing robust search infrastructure, including advanced sorting, filtering, faceting, and geo-sorting capabilities, among others.

New release features
--------------------

*   **Enhanced search relevancy**: The AI-enhanced layers bring a deeper semantic understanding, significantly improving result quality.
*   **AI embedders autogeneration:** Choose between using a third-party solution like OpenAI or your own locally generated embeddings. An open-source embedders generation feature is set for the next release cycle.

![](https://unable-actionable-car.media.strapiapp.com/Group_143725866_7f8309846d.png)

Add a new AI embedder in Meilisearch Cloud.

*   **Improved prompting:** Enhanced to incorporate key information templates, aiding the system in prompt ingestion and relevant metadata interpretation.
*   **Semantic ratio control:** Customize your search to span anywhere from purely keyword-based to entirely semantic.  
    

![](https://unable-actionable-car.media.strapiapp.com/Group_143725865_1_f3910ea1ce.png)

Choose where you want your results to fall on the spectrum from pure keyword-based to fully semantic search.

*   **Multimodal search capabilities:** Expanding search functionality to include image, sound, and video content.

**Democratizing search**
------------------------

At Meilisearch, our journey has always been anchored in simplicity, performance, and out-of-the-box relevancy. Our core mission remains steadfast: make it easy to find anything anywhere by providing a go-to-search solution for every team.

> “Meilisearch's hybrid search will democratize advanced search technology, focusing on simplicity and open-source customizability”  
> \- **Quentin de Quelen, Meilisearch’s CEO and Co-founder.**

As we evolve, our methods become more sophisticated, adapting to the ever-changing needs of businesses and developers alike. In a world brimming with data, effective content discovery is essential, not just a luxury. Our vision combines innovation with accessibility, aiming to make cutting-edge search technology inclusive for all.

The launch of our hybrid search product marks a significant milestone for Meilisearch and the search technology landscape. We invite you to try the new solution (now generally available) and join us in shaping the future of search.

* * *

Stay in the loop by subscribing to our [newsletter](https://dub.sh/meili-newsletter?&utm_source=blog&utm_campaign=HybridSearchRelease). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_source=Blog&utm_campaign=HybridSearchRelease) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions?utm_source=Blog&utm_campaign=HybridSearchRelease).

For anything else, join our developer community on [Discord](https://discord.com/invite/hTD5hU2yWV?utm_source=Blog&utm_campaign=HybridSearchRelease).

Introducing hybrid search: combining full-text and semantic search for optimal balance

🥳 Join our launch event
------------------------

[![Meilisearch product update: AI-powered hybrid search, monitoring, and more. Thursday, March 7th 4PM CET | 3PM GMT | 10AM EST](https://unable-actionable-car.media.strapiapp.com/product_launch_march_df203f4787.png)](https://us06web.zoom.us/meeting/register/tZMsdu2gqTooE9AcjcQEzj9uST040tSjWDKn#/registration)

Join us on **March 7th** to discuss our recent releases and get a sneak peak of what’s coming on Meilisearch Cloud. Spoilers—hybrid search, new embedding models, GPU support, and more. 🤫

[Register](https://us06web.zoom.us/meeting/register/tZMsdu2gqTooE9AcjcQEzj9uST040tSjWDKn#/registration)

🤔 Vector embeddings, explained
-------------------------------

[![Vector embeddings](https://unable-actionable-car.media.strapiapp.com/Vector_embeddings_1_3c8b6c87d6.png)](/blog/what-are-vector-embeddings/?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

Wondering what are vector embeddings? What is the semantic space? How is it relevant for search? Our very own [@CarolainFG](https://twitter.com/CarolainFG) dug deep into the matter to create this high-level educational piece.

[Read blog post](/blog/what-are-vector-embeddings/?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

🚀 Upgrade to 1.6
-----------------

![Meilisearch 1.6 release notes](https://unable-actionable-car.media.strapiapp.com/1_6_release_0a6958f6c2.jpeg)

We released Meilisearch 1.6 in January. Upgrade to benefit from **hybrid search**, **indexing speed improvements**, and further **customization of proximity** ranking rule. It’s time to upgrade. 👇

[Read release notes](/blog/meilisearch-1-6/utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24

* * *

🖇️ Links, links, links
-----------------------

### 💡 [Indexing best practices](https://www.meilisearch.com/docs/learn/cookbooks/indexing_best_practices?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

All our tips to index data efficiently and speed up the indexing process.

### 🔄 [Syncing with your Postgres database](https://www.meilisearch.com/docs/learn/cookbooks/meilisync_postgresql?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

Learn how to keep your Meilisearch in sync with a Postgres database.

### 🐬 [Add search to your Appwrite application](https://appwrite.io/blog/post/add-a-search-function-to-your-app)

Learn how to add a search function to your Appwrite application.

### 🐇 [Meilisearch v1.7 release candidate](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0-rc.0)

Get a preview of what’s coming next (and give your feedback!)

### 🟧 [Laravel Scout guide](https://www.meilisearch.com/docs/learn/cookbooks/laravel_scout?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

Learn how to use Meilisearch in your Laravel 10 application.

* * *

🥰 Made with Meilisearch
------------------------

**Rerun.io** added a search bar to allow users to search through all their documentation and SDKs API references. The site is built with **React** and uses [Analytics](https://www.meilisearch.com/docs/learn/cloud/analytics?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24) to allow them to improve the docs content.

[![rerun.io search interface](https://unable-actionable-car.media.strapiapp.com/Capture_2024_02_27_155651_1_62b82f8316.png)](https://rerun.io/)

[rerun.io](https://rerun.io)

_We’re always looking to promote new projects from the community. Post yours in the_ [_#built-with-meilisearch_](https://discord.meilisearch.com/) _Discord channel!_

🤓 Life at Meili
----------------

Our engineer [@dureuill](https://github.com/dureuill) wrote about how dynamic management of virtual memory enabled us to remove limitations in Meilisearch indexing policy. Attention, this one is technical.

[![Squeezing millions of documents in only 128 TB of virtual memory](https://unable-actionable-car.media.strapiapp.com/squeezing_d635cb5972.png)](/blog/dynamic-virtual-address-management/?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

[Read blog post](/blog/dynamic-virtual-address-management/?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

🦸 We're hiring!
----------------

See our open positions:

*   [Head of Developer Marketing](https://www.meilisearch.com/careers?utm_campaign=newsletter&utm_source=blog&utm_medium=february-updates-24)

Meilisearch February updates

The new dashboard incorporates a set of metrics providing insights into your project’s performance. We want to help engineers move faster by making it easier to spot issues during implementation and having better visibility on your Meilisearch database health.

What metrics can you expect?
----------------------------

This first iteration comes with:

*   30 days of data retention
*   Real-time mode

**Search performance metrics**

*   Search latency
*   Maximum search queries per second

**Operations metrics**

*   Bandwidth (in/out)
*   API Calls — Succeeded/Failed

The default view combines the data from the indexes per project, but it’s also possible to filter out the insights for a specific index of your choice. The dashboard also allows you to select a custom range using a date picker or get real-time data insights:

![](https://unable-actionable-car.media.strapiapp.com/monitoring_metrics_custom_gif_gif_382681a817.jpg)

Why use monitoring**?**
-----------------------

With our brand-new dashboard, you can:

*   Ensure your users are getting lightning-fast search responses for every query
*   Monitor how your search functionality holds up under varying loads, not only during peak traffic conditions but also while indexing
*   Identify and rectify any performance bottlenecks or errors
*   Proactively maintain and optimize your project

**Availability and plans**
--------------------------

The new Cloud monitoring metrics are available for all Cloud users across all pricing tiers. Each tier has access to the same metrics and chart functionalities of the monitoring metrics dashboard.

[Try it in Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=monitoring-v1&utm_source=blog)

Learn how to [get started with Meilisearch Cloud](https://www.meilisearch.com/docs/learn/getting_started/cloud_quick_start?utm_campaign=monitoring-v1&utm_source=blog) and read the [Monitoring documentation](https://www.meilisearch.com/docs/learn/cloud/monitoring#requirements?utm_campaign=monitoring-v1&utm_source=blog).

* * *

Stay in the loop by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&utm_campaign=monitoring-v1&utm_source=blog). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=monitoring-v1&utm_source=blog) and participate in our [Product Discussions](https://github.com/meilisearch/product/discussions/?utm_campaign=monitoring-v1&utm_source=blog).

For anything else, join our developer community on [Discord](https://discord.meilisearch.com/?utm_campaign=teams-release&utm_source=blog).

Cloud monitoring metrics have arrived!

**🚀 Announcing Meilisearch 1.6**
---------------------------------

[![Meilisearch 1.6 release notes](https://unable-actionable-car.media.strapiapp.com/meilisearch_1_6_1_25b4cb0c21.jpg)](/blog/meilisearch-1-6/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

Meilisearch 1.6 brings **hybrid search**, significant **improvements in indexing speed**, and a new API to **customize the proximity** ranking rule. Read the release notes to get the full picture.

[Read release notes](/blog/meilisearch-1-6/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

🔍 Typesense comparison
-----------------------

[![Meilisearch vs Typesense](https://unable-actionable-car.media.strapiapp.com/meilisearch_vs_typesense_cover_1_3b17eec4c2.jpg)](/blog/meilisearch-vs-typesense/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

We sat down to compare the key features between Meilisearch and open-source search engine **Typesense**. You can read the comparison on our blog.

[Read comparison](/blog/meilisearch-vs-typesense/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

🤗 New docs home
----------------

![Meilisearch documentation](https://unable-actionable-car.media.strapiapp.com/Screenshot_2024_01_23_at_17_16_39_e046f3c318.png)

We spent some time reworking our **docs home page** to make it more welcoming for developers. We also made **example apps** easier to find. Let us know what you think!

[View documentation](https://www.meilisearch.com/docs?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

🖇️ Links of the month
----------------------

🔒 [Master key and API keys](https://www.meilisearch.com/docs/learn/security/master_api_keys?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)  
Learn to secure your Meilisearch instance and manage public API keys.

⚡ [Meilisearch indexes documents 2x faster](https://twitter.com/StriftCodes/status/1747566185791553639)  
Take a look at the performance boosts in Meilisearch 1.6.

🐬 [MySQL Full-Text Search Limits and New Alternatives Solutions](https://data-sleek.com/blog/the-limits-of-mysql-full-text-search-and-the-quest-for-alternative-solutions/)  
Comparing MySQL full-text search features with other search engines.

🎓 [Dropdown Autocomplete Search Anything in Laravel](https://codecourse.com/watch/dropdown-autocomplete-search-anything-in-laravel)  
Learn to build instant dropdown search with Laravel and Meilisearch.

⚛️ [The Two Reacts](https://overreacted.io/the-two-reacts/)  
Dan Abramov writes on components state, server data, and React.

🥰 Made with Meilisearch
------------------------

**YoungCapital NEXT** introduced their new website with a fresh branding and design. Built with **Next.js** and **Vercel**, their [job portal search](https://next.youngcapital.nl/vacatures) is powered by Meilisearch.

[![YoungCapital NEXT](https://unable-actionable-car.media.strapiapp.com/Screenshot_2024_01_23_at_17_34_37_1_552b1d480f.png)](https://next.youngcapital.nl/vacatures)

_We’re always looking to promote new projects from the community. Post yours in the_ [_#built-with-meilisearch_](https://discord.meilisearch.com/) _Discord channel!_

🏡 Life at Meili
----------------

[![Discord](https://unable-actionable-car.media.strapiapp.com/discord_2yf3soo8o238y21k_2d54bc8412.jpg)](/blog/from-slack-to-discord-our-migration/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

Our team wrote a blog post explaining our reasons for migrating our community to a Discord server, how we went about it, and why we shut down our public Slack.

[Read blog post](/blog/from-slack-to-discord-our-migration/?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

🦸 We're hiring!
----------------

See our open positions:

*   [Head of Developer Marketing](https://www.meilisearch.com/careers?utm_campaign=newsletter&utm_source=january-news-24&utm_medium=blog)

* * *

Get these monthly updates straight to your inbox by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com).

For more things Meilisearch, join our developers community on [Discord](https://discord.meilisearch.com/). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

Until next time.

Meilisearch latest news and company updates