_This is part 2 of a series of blog posts_ [_originally published on my personal blog_](https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy)_. Begin the journey with_ [_part 1_](/blog/spotify-inspired-hybrid-search-and-rust/)_._

Wouldn't it be great to show you how a single-threaded, memory-mapped key-value store can be more efficient than a hand-written memory-mapped solution? I faced issues while porting the Spotify disk-based Approximate Nearest Neighbors library to Rust and, more specifically, to LMDB. Those issues were primarily due to LMDB and memory safety. Here is the story.

To remind you, [Arroy](https://github.com/meilisearch/arroy) is a library that stores embeddings (vectors of floats) on disk. Some data structures are generated on top of those vectors, which look like trees governed by normals used to recursively split the vectors dataset into subdomains. But you can read more about this in [part 1](/blog/spotify-inspired-hybrid-search-and-rust/ of this series.

### How does Annoy generate the tree nodes?

Annoy, the Spotify library stores the nodes on disk, the item nodes (the ones containing the embeddings), and the other nodes that we will call tree nodes in this article. The advantage of doing this is double:

*   The program's memory usage is low and optimized by the OS as all the nodes live in a memory-mapped file.
*   The concept is simple: Access a node by using its ID. The library will find its offset on disk by using a simple multiplication.

However, there are downsides to doing that. All nodes: items with embeddings along with the tree nodes must have the same size, and If a user wants to identify its embedding using the ID 231, the library will increase the file size to the ID multiplied by the node size. For example, with a vector of 768 dimensions, storing a single item with the ID 231 will generate a file of more than 6.5 TiB.

In generating the final data structure, the tree nodes are all written to disk in the same file as the user item containing the embeddings. Those tree-building processes run in parallel, one running process by tree, and, therefore, requires a lot of synchronization when defining the newly created node ID and the memory area reserved for it, most importantly, when the memory-mapped file is too small to accept more nodes, only a single thread must grow the file, so a mutex is used on top of the mutable memory-map and around a `node_id` variable. One interesting property of tree generation is that the generation process only requires the original user item with embeddings.

### The challenges we encountered when porting it to LMDB

A fun fact is that it is the first time in a long time that I have written C++, and the first time I asked ChatGPT to code for me because I was not confident in doing C++ and feared falling into some segfault. I needed a small program to deserialize embeddings from stdin and give them to Annoy. The chatbot's code was mostly effective, but it omitted a critical [empty vector check](https://gist.github.com/Kerollmops/d0e434016b1a698ec9bdbd86d268885e#file-par_build_tree-cpp-L74), which led to a segmentation fault...

The main obstacle to porting it to LMDB is that writing into this key-value store is single-threaded. It doesn't support concurrent write operations to maintain a correctly balanced BTree. Fun incoming!

#### Reading the user item nodes from different threads

We have used LMDB at Meilisearch since the beginning. It is a well-maintained key-value store used in Firefox and maintained by the OpenLDAP developers. It is memory-mapped and does not maintain a user-end cache of the entries in memory but instead gives you a pointer to the memory-mapped area on disk. The main advantage is that you can keep a pointer to this area for as long as your read transaction lives. Ho yeah! Because it is a transactional key-value store that supports atomic committing, too!

But tree generation doesn't require referring to the generated nodes but only the user items. We previously saw that LMDB gives direct pointers into the memory-mapped file without maintaining any intermediate cache (that could be invalidated). There is another quirk with LMDB: you cannot share a read-only transaction between multiple threads, i.e., `RoTxn: !Sync` and you cannot move the write transaction between threads, i.e., `RwTxn: !Send + !Sync`. Ho! And there is no way to create a read-transaction on uncommitted changes. This is an issue because we want to generate the data-structures trees in the same transaction where we store the items.

But magic is everywhere, starting with the following small and fun data structure. The principle is to keep the pointers to the internal user items with embeddings in a `Vec<*const u8>`. Thanks to Rust, we can ensure, at compile time, that the pointers will live long enough by keeping the lifetime in the struct. Using the `&'t RwTxn` to get the `&'t RoTxn` by using `Deref` also ensures that we cannot modify the database while reading in it by using a `&'t mut RwTxn`. [According to the leading developer of LMDB](https://bugs.openldap.org/show_bug.cgi?id=10138#c9), it is safe to share those pointers between threads and why I implemented `Sync` for this structure.

```Rust
pub struct ImmutableItems<'t, D> {
    item_ids: RoaringBitmap,
    constant_length: Option<usize>,
    offsets: Vec<*const u8>,
    _marker: marker::PhantomData<(&'t (), D)>,
}

impl<'t, D: Distance> ImmutableItems<'t, D> {
    pub fn new(rtxn: &'t RoTxn, database: Database<D>, index: u16) -> heed::Result<Self> {
        let mut item_ids = RoaringBitmap::new();
        let mut constant_length = None;
        let mut offsets = Vec::new();

        for result in database.items()? {
            let (item_id, bytes) = result?;
            assert_eq!(*constant_length.get_or_insert(bytes.len()), bytes.len());
            item_ids.push(item_id);
            offsets.push(bytes.as_ptr());
        }

        Ok(ImmutableLeafs { item_ids, constant_length, offsets, _marker: marker::PhantomData })
    }

    pub fn get(&self, item_id: ItemId) -> heed::Result<Option<Leaf<'t, D>>> {
        let len = match self.constant_length {
            Some(len) => len,
            None => return Ok(None),
        };
        let ptr = match self.item_ids.rank(item_id).checked_sub(1).and_then(|offset| self.offsets.get(offset)) {
            Some(ptr) => *ptr,
            None => return Ok(None),
        };
        let bytes = unsafe { slice::from_raw_parts(ptr, len) };
        ItemCodec::bytes_decode(bytes).map(Some)
    }
}

unsafe impl<D> Sync for ImmutableItems<'_, D> {}
```

You have also probably noticed some other fun optimizations in this simplified version of the data structure. We also know that the user-item nodes have a constant length, so I decided to store it only once, reducing the offsets vector's size by two. Considering that our objective is to store 100M vectors and that this vector is in memory, we shrunk its size from 1526 MiB to 763MiB, which is not much, but better than nothing.

#### Writing the tree nodes in parallel

Ok! Now that we know how to store pointers to items and share them between threads without any user-end synchronization, we need to generate the tree nodes using it. We already know how to deal with LMDB at Meilisearch and decided to implement the same workaround. To write in parallel into LMDB, write into different, independent files and merge everything afterward. This leverages the [Share-Nothing principle](https://en.wikipedia.org/wiki/Shared-nothing_architecture) and isolates the algorithms. This drastically reduces the number of synchronization points [compared to the original C++ code](https://github.com/spotify/annoy/blob/2be37c9e015544be2cf60c431f0cccc076151a2d/src/annoylib.h#L1344-L1445) (look at the `.lock/unlock` calls) to a single line in our code: the atomically increasing global tree node ID.

```Rust
pub fn next_node_id(&self) -> u64 {
    self.0.fetch_add(1, Ordering::Relaxed)
}
```

Our functions that generate normal split nodes based on the user-item nodes are now simply writing the nodes into independent files. The [nodes are appended into a file](https://github.com/meilisearch/arroy/blob/4f193fd534acd357b65bfe9eec4b3fed8ece2007/src/parallel.rs#L20-L71), and the on-disk offsets and bounds are stored into a vector, a single `usize` by node. Using Rayon, we run all tree-generation functions in parallel and, once completed, retrieve the files and boundaries to write the uniquely identified nodes into LMDB sequentially.

### Performances comparison

Our objective at Meilisearch is to support 100M embedding of around 768 dimensions. If we store those as `f32` vectors without any [dimensionality reduction](https://en.wikipedia.org/wiki/Dimensionality_reduction), it would be equivalent to `100M * 768 * 4 = 307B`, in other words, 286 GiB to store the vectors raw, without any internal tree nodes, i.e., no way to search in them efficiently.

If you don't specify the number of trees to generate, the algorithm will continue to create trees while the number of tree nodes is smaller than the number of item vectors. At the end, there must be roughly the same number of tree nodes and items.

#### Discovering the limit of vectors we can index

Arroy and Annoy use memory mapping extensively but in slightly different ways. In a previous article from [@dureuill](https://github.com/dureuill), we saw that [operating systems do not have infinite memory-map space](/blog/dynamic-virtual-address-management/ So, let's dive into performance results.

I noticed something strange when running Arroy with 25M vectors of 768 dimensions. The CPUs were not used, and the program seemed to read the SSD/NVMe a lot, too much 🤔 The tree generation algorithm is splitting the vector space into subdomains with fewer vectors. Still, it must first find the appropriate normal to split the entire domain, and for that, it randomly selects many items. Unfortunately, my machine only has 63 GiB of memory, and indexing those 25M items requires more than 72 Gib. Annoy was also struggling in the same way.

![We cannot index 25M vectors of 768 dimensions with 63 GiB](https://unable-actionable-car.media.strapiapp.com/annoy_arroy_e83f58691c.png)

After investigating, I understood why the whole swap space and memory mapping limits were reached. Not only the item nodes were not only _768 \* 4 bytes_ because we store the norm and other stuff alongside the vectors, but in the case of Arroy, LMDB needs to maintain BTree data structures around the entries, and those are also tacking memory-mapped space. Both programs request random item nodes unavailable in memory, so the OS fetches them from the disk, which takes time. Ho and every single thread is doing that in parallel. CPUs are simply awaiting the disk.

So, after some dichotomy, I found the point where arroy successfully used all of the memory without being bound to the disk speed. It can index 15.625M on a 63 GiB machine. You can see on this htop screenshot that the disk read speed is at zero as all of the vectors fit in RAM, that arroy is writing the tree nodes to disk, and that the CPUs are doing their best. It took less than seven hours to process.

![Successfully indexing 15.6M vector of 768 dimensions with 63 GiB](https://unable-actionable-car.media.strapiapp.com/annoy_arroy2_a873d31d2d.png)

But... Annoy cannot index this same number of documents. It suffers from the same issue we saw before: high disk read and low CPU usage. But why? I needed clarification because the nodes have the same format, the number of item vectors to index is the same, and the algorithm has been ported. So, what is the difference between both solutions?

For those who looked into the C++ codebase and were probably hinted by the memory mapping issue described earlier, you probably noticed this slight difference: Arroy is writing the generated tree nodes into different raw files when Annoy, on the contrary, is [reserving space into the memory-mapped file and directly writing into it](https://github.com/spotify/annoy/blob/2be37c9e015544be2cf60c431f0cccc076151a2d/src/annoylib.h#L1354-L1370). By doing this trick, the OS needs to keep much more space in the memory-mapped area and is forced to invalidate the item nodes from the cache to keep the just-written tree nodes hot in the cache, slowing down the system for no reason, as the tree nodes are not necessary for the algorithm.

So, after even more dichotomy to find the Annoy limits on a 63 GiB machine, I discovered that it could roughly index 10M vectors in five hours.

#### The share nothing principle to the win

So Arroy can index 36% more vectors on the same machine, but how long does it take? Wouldn't it be faster to write into the same file in parallel instead of copying all those tree nodes in a single-threaded way? It will be a much shorter paragraph as I only did some small tests, but Arroy is always faster!

|  | number of vectors | number of threads | building time |
| --- | --- | --- | --- |
| Annoy | 96k | 1 | 5min 38s |
| arroy | 96k | 1 | **3min 45s** |
| Annoy | 96k | 12 | 1min 9s |
| arroy | 96k | 12 | **54s** |
| Annoy | 10M | 12 | 5h 40min |
| arroy | 10M | 12 | **4h 17min** |
| Annoy | 15.625M | 12 | \-- |
| arroy | 15.625M | 12 | **7h 22min** |

Now, you probably tell me that I will need around 400GiB of memory to index 100M vectors, and you are probably right, but [@irevoire](https://github.com/irevoire) will talk about incremental indexing in a future article. I did a linear regression for fun. With those 12 threads and the required memory, I predicted it would take one day, 22 hours, and 43 min 😬. Ho! As we also use this vector store in Meilisearch, we need to provide ways to filter this data structure at search time. This is the next article in this series.

You can comment about this article on [Lobste.rs](https://lobste.rs/s/87aheu/multithreading_memory_mapping_refining), [Hacker News](https://news.ycombinator.com/), [the Rust Subreddit](https://old.reddit.com/r/rust/comments/18jmuoe/multithreading_and_memorymapping_refining_ann), or [X (formerly Twitter)](https://x.com/Kerollmops/status/1735941213516140814).

[Part 3](/blog/arroy-filtered-disk-ann/) of this series is now available, exploring further advancements in our search with Arroy's Filtered Disk ANN.

* * *

[Meilisearch](https://meilisearch.com/?ref=blog.meilisearch.com) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience.

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions).

Multithreading and Memory-Mapping: Refining ANN performance with Arroy

Headquartered in New York and Paris, Hugging Face is an open-source provider of machine learning technologies, allowing users to train, deploy, and share AI models using Hugging Face open-source libraries and the Hub.

**By partnering with Meilisearch Cloud, Hugging Face actively fulfills its visionary commitment to advancing the democratization of AI.**

![](https://unable-actionable-car.media.strapiapp.com/ezgif_com_video_to_gif_1_e697b647e1.gif)

> _“Today, Meilisearch is used by Hugging Face to power the discoverability of 300,000+ AI models, datasets, and demos. It’s really important for AI democratization because if the knowledge is out there, but you can’t access it - what’s the point?” -_ Mishig Davaadorj, software engineer at Hugging Face

Challenge
---------

On its platform, Hugging Face enables users to upload their AI models, datasets, and demos. With a vibrant community of users constantly engaging with the [Hugging Face Hub](http://hf.co) - a platform for sharing and searching for Machine Learning artifacts - the question of discoverability takes center stage.

The Hugging Face Hub hosts over 220,000 AI [models](https://huggingface.co/models) catering to a variety of [machine-learning tasks](https://huggingface.co/tasks), all neatly stored in repositories. These model repositories are designed to make the exploration and utilization of models as seamless as possible. Each AI model in the Hugging Face repository is accompanied by a model card, a project file containing valuable metadata, which plays a crucial role in enhancing discoverability, reproducibility, and sharing. Model cards also provide important information about AI models’ biases and limitations, model descriptions, and training guidelines, and serve as comprehensive guides for users seeking models on the Hub or uploading their own.

Before the introduction of Meilsearch, Hugging Face relied on a simple filtering and keyword search solution. However, it became evident that there was a growing demand for a more flexible and typo-tolerant full-text search solution. This solution needed to ensure higher default relevancy for every search query and leverage the additional attributes and metadata stored within model cards.

Why Hugging Face chose Meilisearch Cloud
----------------------------------------

Before integrating Meilisearch into its ML model repository, Hugging Face had already been utilizing Meilisearch's free-of-charge open-source solution as a search engine for its [open-source libraries](https://huggingface.co/docs)’ documentation, including transformers & diffusers. This documentation comprised approximately 500 pages and had been in use for over a year. Because the team had already gained positive experiences from implementing and operating Meilisearch, when the need for enhanced discoverability in model card searches arose, there was no requirement for additional testing or proof of concept.  

**1\. Customizability of the ranking rules**

During the evaluation process, the Hugging Face team also considered Mongo Atlas Search. However, they weren't satisfied with the customizations available for ranking. Several factors were taken into account in order to enhance model card discoverability, such as project names, descriptions, and the number of likes or downloads on each card. Meilisearch has demonstrated greater flexibility and adaptability in accommodating these search criteria.

**2\. Transitioning to Meilisearh Cloud for its ease of use**

Having achieved successful implementation of Meilisearch in its documentation, the Hugging Face team smoothly transitioned to incorporating Meilisearch Cloud into their model cards repository, benefiting from out-of-the-box relevancy.

**3\. Quality of support and infrastructure outsourcing**

The Hugging Face team chose Meilisearch Cloud, in part due to its dedicated support capabilities. Given that achieving highly relevant search results within the repositories is of utmost importance to Hugging Face, the decision to opt for the Cloud version was further influenced by entrusting the infrastructure to the expertise of Meilisearch Cloud specialists. Outsourcing the infrastructure allows the Hugging Face team to enhance their development speed.

Implementation
--------------

With Meilisearch already implemented for Hugging Face documentation, the expansion of the search solution to other use cases was smooth, once the ranking and internal rules were in place.

Today, the Meilisearch engine powers the discovery of 220,000 model cards, 38,000 datasets, and 60,000 demos in the Hugging Face repository. The keyword filtering mechanism has already been implemented on the Hugging Face homepage for users who are already familiar with specific model names.

However, with the implementation of Meilisearch, users are given the option to perform full-text searches as well. This search functionality operates not only on the model names and IDs but also includes the entire content of model cards.

The additional button to “Try full-text search” was implemented in order to minimize changes in the search behavior, where users are free to choose the search experience that most fits their preference.

![](https://unable-actionable-car.media.strapiapp.com/ezgif_com_video_to_gif_bce9ae142e.gif)

Vision
------

With the ongoing evolution of AI, Hugging Face anticipates a shift in search behavior and the methods people use to find the projects they need. Hugging Face is looking forward to incorporating Semantic Search into its documentation and model card searches. Currently, the platform is actively testing Meilisearch [VectorDB](/blog/langchain-semantic-search-tutorial/ as one of the potential solutions.

Hugging Face also anticipates gaining more detailed usage insights, encompassing information about users' daily search patterns and the level of activity certain model card projects receive over time.

Hugging Face facilitates AI accessibility with Meilisearch

At Meilisearch, we have long maintained our commitment to simplifying and streamlining the search process by prioritizing intuitive, user-friendly, and human-like search experiences.

In today's search technology landscape, _[keyword search](https://www.meilisearch.com/docs/learn/what_is_meilisearch/overview?utm_campaign=vector-search&utm_source=blog&utm_medium=launch#features)_ continues to play a pivotal role, swiftly locating content via specific keywords. Leveraging advanced algorithms and robust indexing, it ensures users find relevant information across vast data volumes.

However, traditional keyword approaches have inherent limitations in understanding user intent and at times can be overly predictable and simplistic. This is why we are introducing Meilisearch's _[vector search](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=vector-search&utm_source=blog&utm_medium=launch),_ which aims to challenge and enhance the familiar keyword search paradigm.

Vector search for AI-enhanced future
------------------------------------

The market-wide introduction of _vector search_ marks the arrival of a new era for search technology, with countless versatile applications. In _e-commerce_, it offers product recommendations based on similarities and multi-modal search capabilities such as image, audio, and video search. The fusion of vector search with conversational AI technologies opens doors for intuitive Q&A applications. Adding user data like geolocation and search history further expands its potential.

New search technology enhances relevancy alongside traditional keywords with semantic understanding going beyond simple keyword matching. It is capable of comprehending intent, returning results like **\[gloves\]** from a **\[warm clothes\]** query. These applications rely on _vector embeddings_—a type of data representation with semantic information that enables AI to gain understanding and maintain long-term memory. Stored in _vector databases_, embeddings represent different data dimensions, essential for understanding patterns and relationships. AI-powered vector database then leverages this information when matching search queries to answers.

Democratizing search
--------------------

In a data-rich world, effective content discovery is not a luxury - it’s crucial. Meilisearch's vision balances ambition with accessibility, aiming to make cutting-edge search technology inclusive. Key Meilisearch partners, such as [Hugging Face](/blog/hugging-face-democratizes-ai/ are already actively testing Meilisearch’s new vector database and storage capabilities.

> **Quentin de Quelen, Meilisearch’s CEO and Co-founder, said:** _“Meilisearch's hybrid search will democratize advanced search technology, focusing on simplicity and open-source customizability.”_

Building the future of search with vector search
------------------------------------------------

This release brings semantic and hybrid search capabilities to enhance applications for image, video, and audio search, product recommendations for e-commerce, precise personalization, and intuitive Q&A interactions.

Here are a few examples of potential applications made possible with the introduction of vector search:

*   **Semantic understanding:** Enable your search engine to act as a human-like assistant, delivering natural, context-aware results in a more conversational manner
*   **Multiformat capabilities:** Cover images, audio, and video, ensuring versatility.
*   **Personalization**: Matches preferences for tailored results, with a deeper understanding of user behavior and preferences,
*   **Precise Recommendations:** Enhance the discovery experience by suggesting similar items that align with users' genuine interests.
*   **Improved accuracy:** Improve relevancy and search accuracy through a deep understanding of each search query's meaning

Search evolution culminates in a hybrid search
----------------------------------------------

Meilisearch's journey leads to _hybrid search_, which combines traditional full-text efficiency with the innovation of semantic search and the adaptability of vector database. The hybrid approach enables search solutions to cater to a broad spectrum of use cases driven by customer-generated data sources. By integrating models from AI solution providers like OpenAI, Cohere AI, or Hugging Face, Meilisearch empowers users to calibrate _vector embeddings_ and infuse their business logic into the search engine.

Getting started
---------------

<Callout>
  For more guidance on how to get started, explore our [step-by-step tutorial on building semantic search with LangChain](/blog/langchain-semantic-search-tutorial/?utm_campaign=vector-search&utm_source=blog&utm_medium=launch)
</Callout>
```json
from langchain.vectorstores import Meilisearch
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.document_loaders import JSONLoader

# Load documents
loader = JSONLoader(
    file_path="./movies.json",
    jq_schema=".[] | {id: .id, title: .title, overview: .overview}",
    text_content=False,
)
documents = loader.load()

# Index documents
embeddings = OpenAIEmbeddings()
vector_store = Meilisearch.from_documents(documents=documents, embedding=embeddings)

# Search
query = "superhero fighting villains in a city corrupted by crime"
results = vector_store.similarity_search(
    query=query,
    k=3,
)

```

If you want to use Meilisearch vector search capabilities without LangChain, choose your favorite language and check out the [relevant SDK documentation here](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks?utm_campaign=vector-search&utm_source=blog&utm_medium=launch).

Want the quality of open-source without the headache of managing your servers and service? [Sign up for a free trial](https://meilisearch.com/cloud?utm_campaign=vector-search&utm_source=blog&utm_medium=launch) with Meilisearch Cloud today.

Introducing Meilisearch's vector search and vector database to navigate the future of search

Vector search enables efficient retrieval of objects sharing similar characteristics. This AI-powered search technique uses embedding vectors. These vectors are mathematical representations of objects generated by machine learning models (like LLMs). Starting with the 1.3 release, Meilisearch supports storing and searching vectors.

<Callout>
  Meilisearch v1.3 is out! Read the [release notes](/blog/v1-3-release/?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement)
</Callout>

Powering the Future of Search
-----------------------------

Vector search might be the dawn of a new era for search. Its use cases are numerous. In [ecommerce](https://ecommerce.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement), it enables offering recommendations for similar products. It also allows building multi-modal search, like image or audio search. Building upon conversational AI technologies enables the creation of Q&A applications. Combining vector search with user-provided information like geolocation and search history can power even more contextual search experiences.

Vector search is also the foundation for [semantic search](https://en.wikipedia.org/wiki/Semantic_search), which aims to understand the query’s meaning. Conversely, traditional lexical search only matches keywords. With semantic search, a `warm clothes` query could give results like `gloves`, `coat`, and more results related to winter clothing.

Vector search unlocks a world of new capabilities for search. Take a look at what some users have already implemented:

*   [similar videos recommendations](https://github.com/adrienpoly/rubyvideo/pull/19) on a site listing Ruby conferences
*   a [documentation chatbot](https://blazy-chat.vercel.app/) proof of concept using [GPT3.5](https://platform.openai.com/docs/models/gpt-3-5) and [LangChain](https://github.com/hwchase17/langchain)

<Callout>
  Meilisearch is available as a [LangChain vector store](https://python.langchain.com/docs/integrations/vectorstores/meilisearch).
</Callout>

Getting Started with Meilisearch Vector Store
---------------------------------------------

  
Starting with v1.3, you can use Meilisearch as a vector store. Meilisearch allows you to store vector embeddings alongside your documents conveniently. You will need to create the vector embeddings using your third-party tool of choice ([Hugging Face](https://huggingface.co/), [OpenAI](https://openai.com/)).

First, spin up a Meilisearch instance. You can [install Meilisearch locally](https://www.meilisearch.com/docs/learn/getting_started/installation?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement#local-installation) or create a [Meilisearch Cloud](https://www.meilisearch.com/cloud?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement) account.

Then, enable the _vector store_ experimental feature:

```Bash
curl \
  -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "vectorStore": true
  }'
```
<Callout>
  This guide uses \`curl\` to make HTTP requests to communicate with Meilisearch. In practice, we recommend using our [SDKs](https://github.com/meilisearch/integration-guides) instead.
</Callout>

Meilisearch now accepts a `_vector` field in your documents. Use it to store the vector embeddings corresponding to your document in this field.

```Bash
curl -X POST -H 'content-type: application/json' \
     'localhost:7700/indexes/songs/documents' \
     --data-binary '[
         { "id": 0, "_vectors": [0, 0.8, -0.2], "title": "Across The Universe" },
         { "id": 1, "_vectors": [1, -0.2, 0], "title": "All Things Must Pass" },
         { "id": 2, "_vectors": [0.5, 3, 1], "title": "And Your Bird Can Sing" }
     ]'
```

After storing your vectorized documents in Meilisearch, you can query them using the `search` or `multi-search` route. To do this, you need to compute the vector of your query (using a third-party tool) and send it in the `vector` field.

```Bash
curl -X POST -H 'content-type: application/json' \
   'localhost:7700/indexes/songs/search' \
   --data-binary '{ "vector": [0, 1, 2] }'
```

When using vector search, returned documents include a `semanticScore` field:

```Bash
{
  "hits": [
    { "id": 0, "_vectors": [0, 0.8, -0.2], "title": "Across The Universe", "_semanticScore": 0.6754 },
    { "id": 1, "_vectors": [1, -0.2, 0], "title": "All Things Must Pass", "_semanticScore": 0.7546 },
    { "id": 2, "_vectors": [0.5, 3, 1], "title": "And Your Bird Can Sing", "_semanticScore": 0.78 }
  ],
  "query": "",
  "vector": [0, 1, 2],
  "processingTimeMs": 0,
  "limit": 20,
  "offset": 0,
  "estimatedTotalHits": 2
}

```
<Callout>
  This API is experimental. You can help us improve it by sharing your feedback in [this Github discussion](https://github.com/meilisearch/product/discussions/677?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink).
</Callout>

A First Step Toward Semantic Search
-----------------------------------

Vector search is our first step towards semantic search. But our long-term goal is to provide hybrid search—combining the benefits of full-text and semantic search to provide the most relevant search experience. Clément Renault, founder and CTO of Meilisearch, shared his thoughts on Github about [exploring semantic search](https://github.com/meilisearch/meilisearch/issues/3838?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink) — read for a founder’s perspective. We can’t wait to share more with you!

Drop your email below to learn more about our progress with AI-powered search. We’ll keep you posted on all updates regarding vector search & semantic search.

Get Meilisearch AI updates straight to your inbox 💌
----------------------------------------------------

\* indicates required

Email Address \*

Marketing Permissions

Please select all the ways you would like to hear from Meilisearch:

Email

You can unsubscribe at any time.

We use [Mailchimp](https://mailchimp.com/legal/terms) to send emails.

We’re excited to walk our first steps toward semantic search. We can’t wait to hear your thoughts on integrating Meilisearch as a vector store. You can give your feedback in [this Github discussion](https://github.com/meilisearch/product/discussions/677?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink).

You can stay in the loop by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&utm_campaign=vector-search&utm_source=blog&utm_medium=textlink). To learn more about Meilisearch's future and help shape it, look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink) and participate in our [Product Discussions](https://github.com/meilisearch/product/discussions/?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement).

I’ll see you there.

Meilisearch latest news and company updates

Multithreading and Memory-Mapping: Refining ANN performance with Arroy

Hugging Face facilitates AI accessibility with Meilisearch

Introducing Meilisearch's vector search and vector database to navigate the future of search

Vector storage is coming to Meilisearch to empower search through AI