[Qogita](https://www.qogita.com/) is a leading global wholesale B2B platform that is trying to revolutionize how products are discovered and procured. On a mission to simplify B2B trade, Qogita helps its customers generate higher margins and turnover while saving time with the help of its innovative technology.

**By partnering with Meilisearch, Qogita is able to ensure high exposure for sellers and increased relevancy for buyers, resulting in an excellent buyer-seller interaction experience.**

> _"Meilisearch has empowered us to deliver lightning-fast and highly relevant search results tailored to our customers' needs. Its seamless integration with our systems has positively impacted developer experience, making local environment setup, testing, and CI/CD automation significantly more efficient. Additionally, Meilisearch has proven to be a cost-effective solution, resulting in substantial savings for Qogita."_  
> **_\- Ivo Silva, Engineering Director at Qogita_**

![](https://unable-actionable-car.media.strapiapp.com/ezgif_com_video_to_gif_87c9670025.gif)

### Challenge

The wholesale industry of today, due to its size and maturity, remains highly traditional in terms of business practices and procurement methods. This means suppliers are hard to find, hard to work with, and often resistant to even taking on new customers.

Qogita aims to address this challenge by making it incredibly easy for sellers and buyers to connect through its platform, all in one place. By offering innovative solutions such as AI-powered pricing and cost optimization, tailoring product search capabilities to users’ actual needs, and providing an excellent buyer-seller interaction via search.

### Why Qogita chose Meilisearch

Prior to Meilisearch, Qogita used Algolia as their search provider and was satisfied in terms of functionality and performance. However, Qogita's team strove for further optimization when it came to both technology and costs. An evaluation was launched to assess alternatives, and a thorough comparison between Meilisearch, Typesense, and Algolia was performed.

The evaluation primarily focused on relevancy, ensuring that the search results were not significantly disrupted with Algolia's results serving as the baseline for comparison. Key searches were conducted for popular products and categories to determine the sorting order and overall performance. Additionally, the evaluation considered the handling of typos and common misspellings, which frequently occur in user searches.

The other factors that contributed to the choice included:  

1.  ******Cost-effectiveness******  
    All the alternatives performed well in the key areas of evaluation, but Meilisearch emerged as the preferred choice due to its cost-effectiveness. Since Qogita has a relatively lean DevOps team, selecting Meilisearch Cloud for production offered a desirable combination of affordability and provided services, including confidence in terms of infrastructure reliability and the high availability assurance.  
    
2.  ******Developer experience******  
    Throughout the entire internal proof-of-concept (PoC) process, the developer experience was a key aspect for evaluation. Meilisearch proved to be advantageous for Qogita as it provided a convenient Python library which simplified the migration process. Setting up containers on Meilisearch Cloud was also a straightforward task. Additionally, Meilisearch's vibrant and engaged open source community served as a significant advantage, establishing trust within Qogita's team early on.  
    
3.  ******Product cadence and roadmap******  
    Qogita’s team was impressed by the speed with which Meilisearch was able to achieve feature parity with the alternatives, despite being one of the youngest search solutions on the market. Additionally, Meilisearch presented a solid [roadmap](https://roadmap.meilisearch.com/tabs/1-under-consideration) that included several features aligned with the value envisioned by Qogita's team for its search initiatives in the near future.

### Implementation and moving to Meilisearch Cloud

Qogita initially self-hosted to evaluate if Meilisearch fulfilled its requirements in terms of development experience. Following the evaluation, production was migrated to Meilisearch Cloud, providing Qogita's team with peace of mind regarding infrastructure outsourcing. The transition from Algolia and the rollout of Meilisearch production went seamlessly, maintaining a high level of internal developer satisfaction.

### Results and Vision

Qogita highlighted higher developer satisfaction as one of the main benefits of choosing Meilisearch Cloud as their search provider, with ease of integration and out-of-the-box relevancy being key contributing factors.

The engineering department's goal was to ensure accurate data and product delivery, as well as providing the best user experience in connecting buyers and sellers through the platform. Meilisearch empowered Qogita team to deliver lightning-fast and highly relevant search results, tailored to meet the specific needs of their customers.

In terms of their vision for search efforts, Qogita expressed that the upcoming facet value search feature (coming to Meilisearch in v1.3) would enhance the user search experience even further. Facets allow customers to refine search results by multiple categories simultaneously. For a company like Qogita, which carries tens of thousands of brands in its catalog, facet search will enable the display and exposure of a greater number of brands to users, with further improved relevancy.

Globally, AI innovation is gaining prominence, and Qogita's team envisions a future that involves contextual search, meaningful search suggestions, and personalization. Meilisearch's upcoming [vector search release](/blog/vector-search-announcement/ marks an exciting venture into the AI space, aiming to fulfill these requirements and provide fast and relevant search for AI-powered applications.

Qogita simplifies B2B trade with Meilisearch Cloud

Today we’ll learn how to promote results by pinning given documents at the top of search results. This can be used in ecommerce to give more visibility to featured products or in-app search to prioritize user-specific content. We’ll use Node.js and React to implement promoted documents.

<Callout>
  Need promoted documents inside the Meilisearch engine? [Give your feedback](https://portal.productboard.com/meilisearch/1-meilisearch/c/98-promoting-documents?utm_medium=social&utm_source=portal_share)!
</Callout>

This tutorial will use a sample movie dataset. Our objective will be to have “Finding Nemo” at the top of the results when typing “funny”, “children”, or “fish”.

Folder structure
----------------

In this guide, we’ll separate the frontend code from the backend code used to configure Meilisearch.

We’ll organize our project using the following folder structure:

```json
my-app/
├─ node_modules/
├─ data/ <-- Backend code goes there
│  ├─ setup.js
│  ├─ movies.json
│  ├─ promoted-movies.json
├─ src/ <-- React code will go there
│  ├─ App.jsx
│  ├─ main.js
│  ├─ ...
├─ package.json
```

First, create a directory for our application (eg. `my-app`) or use an existing one. In our application directory, let’s create a `data` folder for the backend code.

Download the JSON dataset and store them in our new `data` folder. Download the datasets here:

[Movies](/blog/content/files/2023/07/movies.json) ("Download")

[Promoted movies](/blog/content/files/2023/07/promoted_movies.json) ("Download")

Alright. Let’s code!

Meilisearch setup
-----------------

First, we need to launch a Meilisearch instance. You can do so by following instructions in the [local installation guide](https://www.meilisearch.com/docs/learn/getting_started/installation#local-installation) or creating an account on [Meilisearch Cloud](https://www.meilisearch.com/pricing). We’ll write a setup script to configure our Meilisearch instance and seed the data.

<Callout>
  This setup script uses the JavaScript SDK, but you can use any of [Meilisearch SDKs](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks).
</Callout>

In this tutorial, we're using [Node 18](https://nodejs.dev/en/download/). Let's install the JavaScript SDK:

```bash
npm install meilisearch
```

Then, let's create a `setup.js` file:

```javascript
// data/setup.js

import { MeiliSearch } from 'meilisearch'

// Load the datasets
import movies from './movies.json' assert { type: 'json' }
import promotedMovies from './promoted_movies.json' assert { type: 'json' }

// Load credentials from environment
const credentials = {
  host: 'your Meilisearch host',
  apiKey: 'your Meilisearch Admin API key' 
}

// Configuration
const INDEX_NAME = 'movies'
const PROMOTED_INDEX_NAME = `promoted-${INDEX_NAME}`

const searchableAttributes = ['title', 'overview', 'genre', 'release_date']
const displayedAttributes = ['id', 'title', 'overview', 'genre', 'poster', 'release_date']

const setup = async () => {
	console.log('🚀 Seeding your Meilisearch instance')
	
	const client = new MeiliSearch(credentials)
	
	// Adding searchable attributes to movies index
client.index(INDEX_NAME).updateSearchableAttributes(searchableAttributes)
	// In the promoted movies index, only `keywords` is searchable
client.index(PROMOTED_INDEX_NAME).updateSearchableAttributes(['keywords'])
	
	// Both indexes have the same visible attributes
	// `keywords` is hidden in the promoted movies index
	client
		.index(INDEX_NAME)
		.updateDisplayedAttributes(displayedAttributes)
	client
		.index(PROMOTED_INDEX_NAME)
		.updateDisplayedAttributes(displayedAttributes)
	
	// Adding documents
	await client.index(INDEX_NAME).addDocuments(movies)
	await client.index(PROMOTED_INDEX_NAME).addDocuments(promotedMovies)
	
	// Wait for tasks completion
	await watchTasks(client, INDEX_NAME)
	await watchTasks(client, PROMOTED_INDEX_NAME)
}

// Helper to watch tasks
const watchTasks = (client, uid) {
	console.log(`Watching tasks on index ${uid}`)
	const tasks = await client.index(uid).getTasks()
	console.log(`${uid} index: waiting for tasks`)
	await client.index(uid).waitForTasks(tasks)
	console.log(`${uid} index: tasks finished`)
}

await setup()
```

This script creates our two indexes. Let’s recap what it does:

*   It sets the same [displayed attributes](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#displayed-fields) for both `movies` and `promoted-movies`.
*   It sets the [searchable attributes](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#searchable-fields); only `keywords` is searchable in the `promoted-movies` index.
*   It adds documents to our Meilisearch indexes.

<Callout>
  To [optimize indexing speed](/blog/best-practices-for-faster-indexing/ always add documents _after_ configuring settings.
</Callout>

We can now run the script with Node.js:

```bash
node setup.js
```

Our Meilisearch instance is now configured and seeded. 🥳 It’s time to implement promoted results using React!

Displaying promoted results
---------------------------

First, we’ll navigate back to the root directory of our application (eg. my-app). We’ll use Vite to create the React app in this folder.

```bash
npm create vite@latest . --template react
```

If the CLI asks to remove existing files from the current directory, answer 'No'.

Then, let’s install the additional dependencies for [integrating InstantSearch with Meilisearch](/blog/instant-meilisearch/):

```bash
npm install @meilisearch/instant-meilisearch react-instantsearch-hooks-web
```

Let’s edit our App component to display search results. We want the promoted results to show up before the rest. Let’s replace the `App.jsx` boilerplate code with our own:

```javascript
// src/App.jsx

import { instantMeiliSearch } from '@meilisearch/instant-meilisearch'
import { InstantSearch, SearchBox, Hits, Index } from 'react-instantsearch-hooks-web';
   
const MEILI_HOST = 'your Meilisearch host'
const MEILI_SEARCH_KEY = 'your Meilisearch Search API key'

const meilisearchClient = instantMeiliSearch(MEILI_HOST, MEILI_SEARCH_KEY)

const App = () => (
	<div>
		<h1>Hello React</h1>
		<InstantSearch searchClient={meilisearchClient}>
			<SearchBox />
			<h2>Results</h2>
			<Index indexName='promoted-movies'>
				<Hits />
			</Index>
			<h2>More results</h2>
			<Index indexName='movies'>
				<Hits />
			</Index>
		</InstantSearch>
	</div>
);

export default App;

```

We can now run the development server using `npm run dev`. In the browser, our app should display something like this:

![HTML output of the code above](https://unable-actionable-car.media.strapiapp.com/Screenshot_2023_07_20_at_12_50_49_f0c190da8d.png)

Displaying promoted results before other results

Congrats. We’ve successfully displayed our promoted results before the rest of the results. 🎉

<Callout>
  Got stuck? Don’t hesitate to ask for help in our [Discord community](https://discord.meilisearch.com/).
</Callout>

Going further
-------------

This tutorial explored one approach for implementing promoted results. An alternative technique would be implementing document pinning in the backend—using one of [Meilisearch SDKs](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks). This approach has the benefit of allowing to merge results in a single response (as if they were coming from a single index).

Both techniques can achieve similar results. We also plan to integrate [promoted documents](https://portal.productboard.com/meilisearch/1-meilisearch/c/98-promoting-documents?utm_medium=social&utm_source=portal_share) in the Meilisearch engine. Give your feedback on the previous link to help us prioritize it.

For more things Meilisearch, you can subscribe to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e). You can learn more about our product by checking out the [roadmap](https://roadmap.meilisearch.com/) and participating in our [product discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers' community on [Discord](https://discord.meilisearch.com/).

I’ll see you there.

Implementing promoted search results with React InstantSearch

Today, we’re very happy to be sitting down with another of our Japanese language experts whose help was absolutely invaluable to reach the point that we have in support of the language: Miiton!

Getting to know Miiton
----------------------

As usual, we wanted to start the interview by getting to know a bit more about Miiton:

> “Hello, I'm @miiton, and I live in Japan. While I don't have a specific area of expertise, I've been involved in a variety of fields from networking to application development. Before I got into the kind of development work I'm doing now, I spent a lot of time setting up and configuring server racks. Recently, my work has been centered around developing and maintaining an EC site, primarily for B2B transactions, using React and Go mainly.”

We were impressed with the variety of his background, which prompted us to ask how he got into the field in the first place:

> **“**I used to work setting up and configuring server racks, configuring network, server machines, storage, etc. However, with the advent of VMware and the growth of AWS and GCP in the background, I shifted to software engineering while learning programming on my own. I originally liked things like automating deployments with PowerShell, so the transition was smooth.”

Milton also shares that he initially found Meilisearch on GitHub while trying to add an Algolia-like search feature to his support documents.  
  
Is his experience similar to any of our other contributors or users? Do let us know if it’s the case!

A constant work for the improvement of Japanese support
-------------------------------------------------------

  
We then asked what, according to him, could be improved about Meilisearch at this point in time:  

> “Japanese language support and analysis functions. Japanese language support has [progressed quickly starting with PR](https://github.com/meilisearch/charabia/pull/70), but it is still not enough to handle Japanese. [](https://github.com/meilisearch/charabia/pull/70)I am impressed that @ManyTheFish actively listens to me in Discussions and on Twitter. I respect him because he understands Japanese even though he doesn't speak Japanese. [](https://github.com/meilisearch/product/discussions/532)The [analysis function](https://github.com/meilisearch/product/discussions/532) is an important indicator for us to know what users are looking for and what content we need to provide.”

  
Indeed, Many, search engine engineer at Meilisearch, has been doing his best work to tame verious languages! The least we can say about it is that it’s far from a walk in the park. If you’re curious and wish to read more on that topic, you can [find Many’s interview here](/blog/improving-meilisearchs-language-support/  
  
It is not the first time we heard that Japanese is a particularly tricky language to tackle, and we wondered if Miiton would be able to explain to us why it is the case:

> “This is because it is difficult to determine whether a "Kanji-only strings" is Japanese or Chinese. The current implementation of Meilisearch recognizes "Kanji-only strings" as Chinese, so 東京 is normalized to 东京, making it impossible for Japanese users to search for Tokyo. This has been worked around because @ManyTheFish private-release a prototype that forces Japanese, but I hope it will somehow be resolved and incorporated into the main branch.”

  
We can only imagine how bothersome the current situation is for our Japanese users, which is also why we’re so grateful for contributors such as Miiton or [Mosuka](/blog/meilistar-interview-minoru-osuka/ for their continued support in improving the language.

Milton then goes on to explain that Meilisearch is the only open-source project to which he contributed in recent years! Naturally, we were intrigued to learn what motivated him to take that first step and start contributing:

> “It made me think, "I want to use this." Because I was shocked that I could start it up without thinking, just throw in some JSON, and use it right away.”

We were genuinely curious to hear about his firsthand experience contributing to Meilisearch, especially considering it was his first time contributing to an open-source project in a very long time:

> “Contributing to Meilisearch has allowed me to reconsider the multitude of factors involved in implementing a search system. It also gave me the option of using Meilisearch for now. I am convinced that this experience will definitely be useful in my future work. It was also a good opportunity to get in touch with Rust ”

And what a surprise to learn Miiton wasn’t familiar with Rust before contributing to Meilisearch!

A further insight into Miiton’s projects  

-------------------------------------------

Finally, we asked Miiton to share more about the projects he’s been using Meilisearch with:

![Image](https://lh4.googleusercontent.com/qEIsc8BYgC7K5i5TUuLlqU3oGP4tHgWNDOuw8iGtKE4oTJeApun7gutJBUaJ63q0lR0-i1H0hOR3LxzTT2ee_-O0RVYN5RcQZ_eaKLLqjNcYk-8ULyMGRCwpxO-gDpot79Igrz5pH0bHDHz_Nz6x-y4)

> “(I'll refrain from disclosing the specific site, as I can't.) An interesting recent effort is the integration into Shopify. Shopify has a standard & easy-to-use search function, but it only supports up to 1,000 products, and beyond that, you need to consider signing up for a third-party Shopify App. Furthermore, few of these apps are optimized for Japan, and even if they were, they were expensive. Using self-hosted Meilisearch has been a good fit so far. (I'm using the Prototype version with force Japanese language support, so this is not a implementation we can recommend to everyone at this time.) I'm operating a site with roughly 10,000 products on a 2GB memory instance, and I have plenty of resources to spare. I'm also using Prometheus' Metrics (experimental implementation) to create and monitor Grafana screens like this one.”

But that’s not all! As a hobby, Milton also created a public site where he showcases the search experience of Meilisearch in Japanese. You can find the website here, so don’t hesitate to give it a look and lots of love: [https://meilisearch-example-jp.miiton.dev/](https://meilisearch-example-jp.miiton.dev/).

And aside from language support, did any feature catch his attention in particular?

> “Of course response speed and UI development experience. I like that it is InstantSearch.js compatible because of the good development experience. It was very easy to create the above demo site. I don't have to tell you how fast the response time is.”

We couldn’t be more appreciative of Miiton’s time for this interview, and for all his continued support for the improvement of the Japanese language in Meilisearch.  
  
If like Miiton you’re also fluent in a non-european language and are interested in seeing your language improved in Meilisearch, our [Charabia repo is always open to you](https://github.com/meilisearch/charabia)!

Miiton: a master of Japanese has entered the fight!

Welcome to your bi-monthly dose of Meilisearch updates.

🤩 Meilisearch v1.2 is live
---------------------------

[![](https://unable-actionable-car.media.strapiapp.com/81602999_c47b_e299_27e9_41c0dee01e68_6a600f9d34.png)](/blog/v1-2-release/)

Meilisearch v1.2 is here everybody! Don’t forget to update your current version of Meilisearch to the latest one available to enjoy all the news features such as:

*   **Enhanced document management** via filters ⚒️
*   New IS NULL and IS EMPTY **filter operators** ✨
*   **Engine improvements 🚀**
*   And more!

Check out Laurent’s blog post for an in-depth look at [what is new in v1.2](/blog/v1-2-release/?utm_source=release-post&utm_id=v1-announcement&utm_content=whats-new-in-v1.2#new-update

* * *

🐾 What to expect from v1.3
---------------------------

With all that being said, we want to pump you up for the start of a new week! Here's the list of things we're working on for v1.3:

*   Search for **facet values**
*   Displaying **ranking details** at search
*   Sort **facet values** by alphanumerical or value count order
*   Customizing **text separators**
*   Define **fields to search on** at search-time

<Callout>
  The release candidate for v1.3 is out—[try it now](https://github.com/meilisearch/meilisearch/releases/tag/v1.3.0-rc.0)!
</Callout>

We hope to have caught your attention now…

* * *

🤖 Vector search for Meilisearch
--------------------------------

Last but not least, v1.3 will also bring **Vector Search**! We’ve started experimenting with it for the first time this year and are excited to take our first steps into AI. [Read the announcement](/blog/vector-search-announcement/  
  
If you’re as curious as us about this topic, we’re delighted to announce [we’ve released a prototype](https://github.com/meilisearch/product/discussions/621#discussioncomment-6183647) that allows Meilisearch to serve as a vector store. With this prototype, Meilisearch accepts vectors in the documents, stores them, and lets users retrieve the nearest documents using the search endpoint.  
  
We’d be grateful if you wanted to try our prototype and [share your feedback with us in this Discord conversation](https://discord.com/channels/1006923006964154428/1118827065890844733/1118827065890844733).

* * *

📰 In the news
--------------

  
**Community**

🎁 [June giveaway!](https://twitter.com/meilisearch/status/1666032750187429889)

This June, we’re hosting a giveaway! To join it, submit a PR in any of our repos, have it merged by a member of our team, and share the link to it in the ⁠[giveaway channel](https://discord.com/channels/1006923006964154428/1111273670657200198) of [our Discord](https://discord.gg/meilisearch). You’ll automatically be participating in the lottery for the chance to win some exclusive Meiliswag, similar to the one you can see below!

[![](https://unable-actionable-car.media.strapiapp.com/0f8251b0_f453_1509_8ed4_4f986132fc44_6510326af9.png)](https://twitter.com/meilisearch/status/1666032750187429889)

**Product**

☁️ [Search-as-a-service Meilisearch Cloud is open to the public](/blog/meilisearch-cloud-general-availability/?utm_source=release-post&utm_id=cloud-announcement&utm_content=earch-as-a-service#meilisearch-cloud)

Now any business can benefit from Meilisearch’s exceptional developer and search experience without dealing with server management!

🧠 [Squeezing millions of documents in 128 TB of virtual memory](/blog/dynamic-virtual-address-management/?utm_source=tech-post&utm_id=virtual-memory&utm_content=Squeezing-millions-of-documents-in-128-TB-of-virtual-memory)

Curious to know more about how dynamic management of virtual memory enabled us to remove limitations in Meilisearch indexing policy? Check out this blog post!

**Success stories**

👶 [Minipouce.fr provides a top-grade customer experience with Meilisearch's open-source offering](/blog/minipouce-top-grade-customer-experience/?utm_source=customer-experience&utm_id=minipouce-customer-experience&utm_content=Minipouce-fr-provides-a-top-grade-customer-experience-with-Meilisearchs-open-source-offering)

With Meilisearch, Minipouce.fr can maintain its commitment to delivering the best possible user experience. Read about Minipouce’s efforts to make parents’ lives easier.

📚 [Bookshop.org increases search-based purchases by 43% with Meilisearch](/blog/bookshop-increases-search-based-purchases/??utm_source=customer-experience&utm_id=bookshop-customer-experience&utm_content=Bookshop-org-increases-search-based-purchases-by-43%-with-Meilisearch)

How Meilisearch helped Bookshop.org boost their sales and how we made it happen.

* * *

🌾 Made with Meilisearch
------------------------

Our featured project of the month is [1jeune1solution.gouv.fr](https://www.1jeune1solution.gouv.fr/) submitted by our Meilistar **[LiquidIT](https://github.com/LiquidITGuy)**. This platform is an French government’s initiative to help young people get into the job market. Meilisearch allows users to search through a comprehensive list of jobs to quickly find their ideal offer.

[![](https://unable-actionable-car.media.strapiapp.com/0b6c7c06_926e_4a0f_bc3d_55cabc6dfc28_0a959bbdcf.gif)](https://www.1jeune1solution.gouv.fr/stages)

The website is built with **Next.js** and the repository is [available on Github](https://github.com/DNUM-SocialGouv/1j1s-front/). The codebase gives numerous examples of how to implement Meilisearch with **React InstantSearch Hooks** using facets, custom components, and URL synchronization.

We’re always looking to promote new projects from the community. Post yours in our [#built-with-meilisearch](https://discord.gg/meilisearch) Discord channel!

* * *

👯‍♂️ Meilisearch life
----------------------

The HR team presents a rundown of our best practices for a successful offsite with a remote-first team.

[![](https://unable-actionable-car.media.strapiapp.com/9347396d_ae70_588d_0d88_567674b7bec1_85da0966b1.jpg)](/blog/organizing-a-people-first-offsite/?utm_source=hr-post&utm_id=hr-organization&utm_content=organizing-a-people-first-offsite)

We are recruiting!

*   [Account Executive](https://jobs.lever.co/meili/92e2ebdc-3023-4fbe-8dd2-c3cfc9ecd7bf)

Wishing that you pass all your CI pipelines,

_The Meilisearch team_

Meilisearch July updates

Adding documents to Meilisearch is taking longer than expected? The following tips will help you speed up the indexing process.

<Callout>
  This is a very short guide. For a more comprehensive understanding, you can read our article on [indexing best practices](/blog/best-practices-for-faster-indexing/)
</Callout>

Define searchable attributes
----------------------------

To speed up indexing, review the [searchable attributes](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#searchable-fields) list and ensure it includes only the fields you want to be checked for query word matches. This improves relevance and search speed by eliminating irrelevant data and preventing unnecessary database growth.

Utilize filterable and sortable attributes
------------------------------------------

Some fields may not contain text but can still be vital for [filtering](https://www.meilisearch.com/docs/learn/fine_tuning_results/filtering) and [sorting](https://www.meilisearch.com/docs/learn/fine_tuning_results/sorting) results. Evaluate if numeric and boolean field values can be part of the filterable or sortable attributes lists.

Review ranking rules
--------------------

Modifying [ranking rules](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#ranking-rules) may trigger a reindexing process. Consider the impact and plan accordingly.

Optimize document size
----------------------

Smaller documents are processed faster. Compress your data using methods such as `br`, `deflate`, or `gzip` supported by Meilisearch. Refer to the [documentation](https://www.meilisearch.com/docs/reference/api/overview#content-encoding) for more details.

Keep up with the latest stable version
--------------------------------------

Regularly update to the [latest Meilisearch release](https://github.com/meilisearch/meilisearch) to take advantage of performance improvements that can significantly enhance indexing speed.

Conclusion
----------

By following these recommendations, you can optimize the performance of Meilisearch and enhance the overall search experience for your users. If you have any question, you can join us on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com).

Boost productivity and streamline development with [**Meilisearch Cloud**](https://www.meilisearch.com/pricing). No more server deployment or manual updates. Try it **free** for 14 days, no credit card needed.

_Happy coding!_

For more things Meilisearch, subscribe to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about our product by checking out our [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in our [product discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

Indexing optimization guide

Vector search enables efficient retrieval of objects sharing similar characteristics. This AI-powered search technique uses embedding vectors. These vectors are mathematical representations of objects generated by machine learning models (like LLMs). Starting with the 1.3 release, Meilisearch supports storing and searching vectors.

<Callout>
  Meilisearch v1.3 is out! Read the [release notes](/blog/v1-3-release/?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement)
</Callout>

Powering the Future of Search
-----------------------------

Vector search might be the dawn of a new era for search. Its use cases are numerous. In [ecommerce](https://ecommerce.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement), it enables offering recommendations for similar products. It also allows building multi-modal search, like image or audio search. Building upon conversational AI technologies enables the creation of Q&A applications. Combining vector search with user-provided information like geolocation and search history can power even more contextual search experiences.

Vector search is also the foundation for [semantic search](https://en.wikipedia.org/wiki/Semantic_search), which aims to understand the query’s meaning. Conversely, traditional lexical search only matches keywords. With semantic search, a `warm clothes` query could give results like `gloves`, `coat`, and more results related to winter clothing.

Vector search unlocks a world of new capabilities for search. Take a look at what some users have already implemented:

*   [similar videos recommendations](https://github.com/adrienpoly/rubyvideo/pull/19) on a site listing Ruby conferences
*   a [documentation chatbot](https://blazy-chat.vercel.app/) proof of concept using [GPT3.5](https://platform.openai.com/docs/models/gpt-3-5) and [LangChain](https://github.com/hwchase17/langchain)

<Callout>
  Meilisearch is available as a [LangChain vector store](https://python.langchain.com/docs/integrations/vectorstores/meilisearch).
</Callout>

Getting Started with Meilisearch Vector Store
---------------------------------------------

  
Starting with v1.3, you can use Meilisearch as a vector store. Meilisearch allows you to store vector embeddings alongside your documents conveniently. You will need to create the vector embeddings using your third-party tool of choice ([Hugging Face](https://huggingface.co/), [OpenAI](https://openai.com/)).

First, spin up a Meilisearch instance. You can [install Meilisearch locally](https://www.meilisearch.com/docs/learn/getting_started/installation?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement#local-installation) or create a [Meilisearch Cloud](https://www.meilisearch.com/cloud?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement) account.

Then, enable the _vector store_ experimental feature:

```Bash
curl \
  -X PATCH 'http://localhost:7700/experimental-features/' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "vectorStore": true
  }'
```
<Callout>
  This guide uses \`curl\` to make HTTP requests to communicate with Meilisearch. In practice, we recommend using our [SDKs](https://github.com/meilisearch/integration-guides) instead.
</Callout>

Meilisearch now accepts a `_vector` field in your documents. Use it to store the vector embeddings corresponding to your document in this field.

```Bash
curl -X POST -H 'content-type: application/json' \
     'localhost:7700/indexes/songs/documents' \
     --data-binary '[
         { "id": 0, "_vectors": [0, 0.8, -0.2], "title": "Across The Universe" },
         { "id": 1, "_vectors": [1, -0.2, 0], "title": "All Things Must Pass" },
         { "id": 2, "_vectors": [0.5, 3, 1], "title": "And Your Bird Can Sing" }
     ]'
```

After storing your vectorized documents in Meilisearch, you can query them using the `search` or `multi-search` route. To do this, you need to compute the vector of your query (using a third-party tool) and send it in the `vector` field.

```Bash
curl -X POST -H 'content-type: application/json' \
   'localhost:7700/indexes/songs/search' \
   --data-binary '{ "vector": [0, 1, 2] }'
```

When using vector search, returned documents include a `semanticScore` field:

```Bash
{
  "hits": [
    { "id": 0, "_vectors": [0, 0.8, -0.2], "title": "Across The Universe", "_semanticScore": 0.6754 },
    { "id": 1, "_vectors": [1, -0.2, 0], "title": "All Things Must Pass", "_semanticScore": 0.7546 },
    { "id": 2, "_vectors": [0.5, 3, 1], "title": "And Your Bird Can Sing", "_semanticScore": 0.78 }
  ],
  "query": "",
  "vector": [0, 1, 2],
  "processingTimeMs": 0,
  "limit": 20,
  "offset": 0,
  "estimatedTotalHits": 2
}

```
<Callout>
  This API is experimental. You can help us improve it by sharing your feedback in [this Github discussion](https://github.com/meilisearch/product/discussions/677?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink).
</Callout>

A First Step Toward Semantic Search
-----------------------------------

Vector search is our first step towards semantic search. But our long-term goal is to provide hybrid search—combining the benefits of full-text and semantic search to provide the most relevant search experience. Clément Renault, founder and CTO of Meilisearch, shared his thoughts on Github about [exploring semantic search](https://github.com/meilisearch/meilisearch/issues/3838?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink) — read for a founder’s perspective. We can’t wait to share more with you!

Drop your email below to learn more about our progress with AI-powered search. We’ll keep you posted on all updates regarding vector search & semantic search.

Get Meilisearch AI updates straight to your inbox 💌
----------------------------------------------------

\* indicates required

Email Address \*

Marketing Permissions

Please select all the ways you would like to hear from Meilisearch:

Email

You can unsubscribe at any time.

We use [Mailchimp](https://mailchimp.com/legal/terms) to send emails.

We’re excited to walk our first steps toward semantic search. We can’t wait to hear your thoughts on integrating Meilisearch as a vector store. You can give your feedback in [this Github discussion](https://github.com/meilisearch/product/discussions/677?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink).

You can stay in the loop by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&utm_campaign=vector-search&utm_source=blog&utm_medium=textlink). To learn more about Meilisearch's future and help shape it, look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink) and participate in our [Product Discussions](https://github.com/meilisearch/product/discussions/?utm_campaign=vector-search&utm_source=blog&utm_medium=textlink).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=announcement).

I’ll see you there.

Vector storage is coming to Meilisearch to empower search through AI

Today, we’re joined by Gregory, or you may know him as GregoryCongad on Github, for yet again one of our Meilistars interviews!

Meet Greg!
----------

Let’s start by learning a little bit more about our latest Star: just last spring, Greg finished his undergrad in computer science at [WPI](https://www.wpi.edu/) and started contributing to Meilisearch at the end of 2022.  
He actually got introduced to Computer Science all the way back in 5th grade. That’s 10 years old for all non-Americans out there! And that’s all thanks to him stumbling across [Scratch, the famous MIT website](https://scratch.mit.edu/). It was love at first sight and Greg has been passionate about programming ever since.

But he didn’t stop there!

> “In middle school, I self-taught Python, C++, and Java by reading some books and by coding up some miscellaneous side projects. I've dabbled in many languages and projects over the years, but have finally picked up Rust after hearing so many good things about it (and the hype is there for a reason!)”

Truly a young prodigy! And that’s not even half of it, for the past two summers, Greg has been interning at Amazon under the Alexa project, focusing on internal tooling, creating products for internal consumption across the organisation. Not much more can be shared on this matter of course due to his NDA, but this sure looks like high profile work.

A serendipitous discovery
-------------------------

With all this experience and knowledge under his belt, we couldn’t help but wonder how he’d discovered Meilisearch. And why change a successful method? Just as with Scratch, Greg confesses he simply stumbled upon Meilisearch:  

> “I was looking for a full text search engine to embed in a Flutter library and was going to settle on TypeSense. TypeSense wasn’t exactly what I wanted though, and eventually I found Meilisearch through one of TypeSense’s comparison pages in their docs. I realized Meilisearch was perfect for my use case, especially since I saw it was written in Rust and I wanted to pick up Rust at the time.”

We really got lucky to have caught his attention! We wondered what had convinced him to give Meilisearch a try as opposed to TypeSense, and Greg shared two main reasons:  

> “1. Persistence on disk. Since I was embedding it, having persistence across application open/close was pretty important  
> 2.The license. Meilisearch is licensed under MIT, whereas TypeSense is GPL. GPL is pretty restrictive, so I try to stay away from it whenever possible”

Our curiosity got the best of us once again, and we couldn’t help being nosey by asking which features of Meilisearch he preferred, and which one could do with a bit more work:  

> “My favorite aspect is how well everything is architected—it’s super easy to embed Meilisearch in other Rust applications and only grab the features you need. Of course, the typo tolerance and relevance that are builtin/require zero configuration are probably my favorite aspects of the product! As far as what I’d like to see, I’d love to have the engine auto-update/have compatibility across versions. Currently, you need to make dumps between versions, but it’d be very nice for that process to be automated (and some work is being done toward that end, like updating how Settings are stored internally)”

A lot of food for thought for all, which we’re very grateful for since hearing our users’ feedback is the best way for us to stay relevant and truly offer you all the best possible search experience.

And what of Greg's personal projects?
-------------------------------------

  
Seeing Greg’s impressive experience despite his young age, we asked whether he’s had the chance to use Meilisearch in a professional context or for a personal project:  

> “I’ve used/embedded Meilisearch to create [Mimir](https://github.com/GregoryConrad/mimir), which is a Flutter library that gives developers an on-device NoSQL database with typo-tolerant & relevant full text search. I originally created Mimir with an application of my own in mind, but I’ve since put that application on hold. Now I just work on Mimir and contribute back to Meilisearch as a hobby project, since I enjoy working on both! The Meilisearch team is really welcoming and nice, which is a great change when compared to other open source projects. Really easy to land your first contribution!”

Go give [Mimir](https://github.com/GregoryConrad/mimir) some love if you have the chance! And we’re very touched to hear Greg’s first experience with us was a positive one, that’s definitely something we aim for!

Embedding Meilisearch to Mimir was actually how Greg first noticed that there were some points of improvement he could contribute to in order to make Meilisearch better! His [first contributions](https://github.com/meilisearch/milli/pull/706) were actually created as a way to solve the issues he ran into when embedding Meilisearch to Mimir! That PR was even featured in one of our release blog posts. Clearly, Greg and Meili were a match made in heaven.

A big thank you to Greg for his time and his answers, we were delighted to be able to spend a moment with him!

Gregory Conrad: POV of a raising young star

We knew it was only the beginning of our 4-day offsite when we boarded the train to Brittany. But little did we know it was only the beginning of our foray into strange and delightful vehicles.

Our hotel was on the coastal path, offering a beautiful sea view from the restaurant lounge. It could have been enough—but would it have been worth it had the offsite been too demanding on our team members? At Meili, we believe in prioritizing the well-being of our employees. As a [mainly remote company](/blog/remote-work-is-the-future-of-work/)he offsite is more than a break in our daily routine. It’s also the chance to meet all the team members who could make it in real life. However, we believe it’s our responsibility as a company to provide a safe environment to everyone, especially during offsites. Our core values are carefully reflected each step of the way, structurally and from workshops to team-building activities.

Upholding our values during an offsite
--------------------------------------

At Meilisearch, [we are committed to upholding our values](/blog/advice-on-answering-interview-questions-about-values/)e are constantly striving to improve on an individual and organizational level. An offsite is a perfect opportunity to educate ourselves collectively on skills that are important to us.

On our second day during this offsite, we attended a workshop about unconscious bias. We believe all companies must be watchful of these stereotypes; they can affect how we recruit, include, and support team members. The practical questions and situations were very relevant to see how quickly we can rely on stereotypes to make decisions and how we could mitigate these effects. Some of the actions are as easy as involving other people! We are more committed than ever to ensuring our decision process involves feedback from team members from different jobs at Meilisearch and different backgrounds.

During this 4-day week, we attended another workshop: the [Climate Fresk](https://climatefresk.org/). The goal was also to identify some actions we could take, individually and as a company in the tech industry. One of the first actions we have to take is to educate ourselves about the fundamental science behind climate change—that workshop was a beginning, and we now have to pursue our efforts, for example, with our servers.

Throughout the offsite, we cater to different preferences and needs. This means being comfortable with a loser schedule and allowing our team members time and space to go at their rhythm. For example, we choose to allow separate rooms for each person. We’ve found it makes a huge difference in our energy level, and our team members get to the end of the offsite with less mental fatigue.

We also give people free time each day to spend as they wish, whether exploring the local area, going to the hotel spa, chatting with people, or simply recharging their batteries.

![](https://unable-actionable-car.media.strapiapp.com/Bretagne2_80409eaefe.jpg)

Picture by Ingrid Hazan

One of our core values is to learn from feedback. The offsite is no exception: we ask questions after each offsite to see what we did well and what to work on for the following offsites. Measuring how we do is our way to ensure that everyone's needs are met. This is a plea to all hotels: we need more veggie and vegan qualitative options during meals!

Supporting a positive and safe team-building
--------------------------------------------

At Meili, we believe in the power of teamwork. Our [collaborative approach](/blog/how-meilisearch-built-upon-42-collaborative-approach/)s at the heart of our company. At the question “What was your favorite thing about the offsite?”, most people say it was chatting with other team members.

This is why we organize low-stakes team-building activities. A loose structure helps people to bond and get to know each other better without taking away the spontaneity of the connection. We want people to have fun without feeling pressured to perform. As such, they can opt out of activities before the offsite.

We asked every team member to write a short anecdote about themselves before the offsite, printed them, and distributed them randomly to other team members. It served as a great ice-breaker and conversation starter on Tuesday evening!

On our first day, we went on a pedestrian treasure hunt in the Corsair city of Saint-Malo. Though we had to rush to make all the spots, it was a great way to explore the city with a squad of people we don’t usually work with. It’s a shame to go to a beautiful place only to spend all our time in a meeting room.

![Photo of rocks in the sea, with a tiny island in which there’s a house, surrounded by fortification walls.](https://unable-actionable-car.media.strapiapp.com/Bretagne3_3bb3447be8.jpg)

Picture by Ingrid Hazan

We had a blast building our cardboard karts on Wednesday afternoon. We had the blueprints, sheets, and cardboard tubes, and there we went! Each team did their best to build, decorate and name their cars. Each kart had its personality and backstory. The presentation speeches were a highlight of our offsite! We had a race to determine which was the fastest kart. I’m afraid I must report we suffered heavy losses and some cars didn’t make it—but they are always in our hearts!

![A white kart made all in cardboard on the road in front of trees. The kart sides are decorated with bone shapes.](https://unable-actionable-car.media.strapiapp.com/kart_379654d107.jpg)

Picture by Roxane Beaufils

Ultimately, we see ourselves as the vessel for a fantastic team to come together and create something special. Like the cars we build during our offsite activities, we provide the structure and the elements, but our team makes the jazz and soul of Meilisearch. We wish to thank the team of [Gloria](https://www.gloriaforbusiness.com/) with whom we worked on the workshops and the team of [Kymono Life](https://www.kymonolife.co/), with a special thanks to Anaïs who organized the offsite for us.

How to organize a people-first offsite

Search bars are an integral part of browsing the web. Whether we are shopping for clothes, enjoying music, or planning a vacation, there is always a dedicated search engine at work to simplify our lives. Have you ever wondered how these search engines actually work? In this article, we will delve into the inner workings of one of them: [Meilisearch](https://meilisearch.com).

Indexing time: from data to documents
-------------------------------------

Indexing is a fundamental process that encompasses the collection, parsing, and storage of data for subsequent retrieval. It plays a crucial role in enabling search engines to deliver fast and relevant results.

### A performant storage engine

Meilisearch stores data in the form of discrete records called "[documents](https://docs.meilisearch.com/learn/core_concepts/documents.html)," which are grouped together into collections known as "[indexes](https://docs.meilisearch.com/learn/core_concepts/indexes.html)." Under the hood, Meilisearch uses the LMDB ([Lightning Memory-Mapped Database](http://www.lmdb.tech/doc/)) key-value store which has proven stability and extensive usage in database applications. As the name implies, key-value stores are storage systems that organize data as a **collection of key-value pairs**.

![Illustration of a key-value store with 'engine' set to 'meilisearch' and 'language' set to 'rust'](https://unable-actionable-car.media.strapiapp.com/key_value_store_d2a8b6ff55.png)

Example of key-value entries

Key-value stores offer flexibility, fast access times, and efficient performance, contributing to quick and responsive interactions for users. By permitting only a single writing process at a time, LMDB avoids synchronization-related issues. As a result, it enables unlimited concurrent readers to have fast access to up-to-date, consistent data.

Before storing data in its key-value store, Meilisearch must preprocess it thoroughly. That’s where tokenization comes in.

### From words to tokens

**Tokenization** involves two main processes: segmentation and normalization.

**Segmentation** consists in splitting a sentence or phrase into smaller units called tokens.

Meilisearch uses (and maintains) the open-source tokenizer named [Charabia](https://github.com/meilisearch/charabia). The tokenizer is responsible for retrieving all the words (or tokens) from document fields.

<Callout>
  In practice, Meilisearch users can configure which fields should be [searchable](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#searchable-fields), allowing Charabia to know which fields should be tokenized.
</Callout>

Then comes **normalization**. Words are normalized based on each language’s particularities. For a romance language like French, this process includes setting the word in lowercase letters and removing diacritical marks such as accents. For instance, a sentence like “Le café de Nicolas” is transformed into “le cafe de nicolas”.

![The sentence 'Le café de Nicolas,' with a capital 'L' and 'N,' and an accent on the 'é' of 'café,' is first split into words during the segmentation phase. Then, during normalization, each word is lowercased, and accents are removed.](https://unable-actionable-car.media.strapiapp.com/Tokenisation_3_ea1333d3a3.png)

Example of the segmentation and normalization stages in the tokenization process

After segmenting and normalizing words, the engine needs to classify them. Using the appropriate data structure to organize tokens is crucial for performance. This will later allow swift retrieval of the most relevant search results.

### Storing tokens

Modern full-text search engines like Meilisearch come with features such as prefix search, typo tolerance, and geo search. Each data structure has its own advantages and disadvantages. When designing the search engine, our team carefully selected the ones best suited to enable these features without compromising on speed.

Throughout this section, we will explore which data structures powers Meilisearch.

#### Inverted index

First and foremost, we need to talk about inverted indexes. Without a doubt, it’s the data structure that stands out by its importance. Meilisearch uses it to return documents fast upon search.

An **inverted index** maps each word in a document to the set of documents where that term appears along with additional information like their position in the document.

![Given the following documents: `{   “id”: 1   "title": "Hello world!" }, {     “id”: 2     "title": "Hello Alice!" }.  The inverted index would look like this:  "alice" -> [2], "hello" -> [1,2],  "world" -> [1]` ](https://unable-actionable-car.media.strapiapp.com/Inverted_index_3_edcd06ad15.png)

Illustration of an inverted index generated from given documents, showcasing the mapping of keywords to document IDs

By storing words once and associating them with the documents where they appear, inverted indexes leverage the redundancy of terms across documents. Thanks to this, Meilisearch does not need to browse all documents in order to find a given word, resulting in faster searches.

Meilisearch creates approximately 20 inverted indexes per document index, making it one of the data structures with the highest number of instances. Indeed, to provide a search-as-you-type experience, the engine needs to precompute a lot at indexing time: word prefixes, documents containing filterable attributes, and more. For a better understanding of how inverted indexes work read more on [indexing best practices](/blog/best-practices-for-faster-indexing/

<Callout>
  You can find the complete list of Meilisearch inverted indexes [on GitHub](https://github.com/meilisearch/meilisearch/blob/78e611f282d726a604cfbc231666265a37c4d46b/milli/src/index.rs#L103-L157).
</Callout>

#### Roaring bitmap

**Roaring bitmaps** are compressed data structures designed for efficiently representing and manipulating sets of integers.

Meilisearch uses roaring bitmaps widely throughout its inverted indexes to represent document IDs. Roaring bitmaps offer a space-efficient way to store large sets of integers and perform set operations like [union, intersection, and difference](https://en.wikipedia.org/wiki/De_Morgan%27s_laws). These operations play a crucial role in refining search results by allowing the inclusion or exclusion of specific documents based on their relationship with others.

#### Finite-state transducer

A finite-state transducer (FST) is a structured data representation ideal for performing string-matching operations. It represents a sequence of states ordered from smallest to largest, with words arranged in lexicographic order.

<Callout>
  [Lexicographic order](https://en.wikipedia.org/wiki/Lexicographic_order) is a method of arranging or sorting items, such as words or symbols, based on their alphabetical or numerical sequence.
</Callout>

A finite-state transducer is also called a _word dictionary_ because it contains all words present in an index. Meilisearch relies on the utilization of two FSTs: one for storing all words of the dataset, and one for storing the most recurrent prefixes.

FSTs are useful because they support compression and lazy decompression techniques, optimizing memory usage and storage. Additionally, by using automata such as regular expressions, they can retrieve subsets of words that match specific rules or patterns. Moreover, this enables the retrieval of all words that begin with a specific prefix, empowering fast, comprehensive search capabilities.

The FST, with its compact design, presents the added advantage of being a smaller data structure than the inverted index. As we will see later, this contributes to faster and more efficient reading operations. You can learn more about the topic in this [comprehensive article on finite-state reducers](https://blog.burntsushi.net/transducers/).

#### R-tree

R-trees are a type of [tree](https://en.wikipedia.org/wiki/Tree_$data_structure$) used for indexing multi-dimensional or spatial information. Meilisearch leverages R-trees to power its geo search functionality.

The **R-tree** associates geographical coordinates with the identifiers of the document to which it belongs. By organizing coordinates this way, it enables Meilisearch to perform spatial queries with remarkable efficiency. These queries allow users to find nearby points, points within a specific area, or points that intersect with other spatial objects.

Search time: query processing
-----------------------------

Modern search experiences only require you to start typing to get results. To achieve this search-as-you-type experience, Meilisearch precomputes a list of the most frequent prefixes.

To be tolerant to typos, Meilisearch uses finite-state transducers in conjunction with the [Levenshtein algorithm](https://www.meilisearch.com/docs/learn/configuration/typo_tolerance#understanding-typo-calculations). This algorithm calculates the Levenshtein distance which can be thought of as the cost of transforming a string into another. In other words, it quantifies the number of transformations required for a word to be converted into another word.

In the context of the Levenshtein algorithm, possible transformations are:

*   insertions, e.g. hat -> chat
*   deletions, e.g. tiger -> tier
*   substitutions, e.g. cat -> hat
*   transpositions or swaps, e.g. scared -> sacred

FSTs generate all possible variations of a word within a specified edit distance, enabling the engine to calculate the Levenshtein distance and accurately detect typos by comparing user input against a dictionary of valid words.

It becomes apparent that there are many factors to consider when processing a search request. Has the user finished typing? Are there typos in the query? Which documents should appear first in the search results?

Let’s discuss how Meilisearch processes search queries and how it refines and sorts search results.

### Query graph

Each time it receives a search query, Meilisearch creates a query graph representing the query and its possible variations.

To compute these variations, Meilisearch applies [concatenation](https://www.meilisearch.com/docs/learn/advanced/concat#concatenated-queries) and [splitting](https://www.meilisearch.com/docs/learn/advanced/concat#split-queries) algorithms to the query terms. The variations considered also include potential typos and synonyms–if the user set them. As an example, let’s examine the query: `the sun flower`. Meilisearch will also search for the following queries:

*   `the sunflower`: concatenation
*   `the sun flowe**d**`: substitution
*   `the sun flower**s**`: addition

Consider now a more complex query: `the sun flower is facing the su`, the query graph should look like this:

![Graph representing interpretations of the search query 'the sun flower is facing the su'. The engine precomputes for each query term word variations, like 'flowed' or 'flowers' and the associated Levenshtein cost ](https://unable-actionable-car.media.strapiapp.com/debuggong_image_e0a6ce757c.png)

Visual generated using our internal debugging tool, powered by [D2](https://github.com/terrastruct/d2), showcasing the execution process of search requests

As illustrated above, the graph represents different interpretations of the search query. The engine precomputes the word variations (and their cost) for each query term. Moreover, it detects whether the last query term is a prefix (not followed by a space), thus signaling the need to consult the prefix database.

So how does the engine utilize a query graph?

As we saw earlier, during the indexing process, Meilisearch identifies all documents that have filterable attributes, such as `genre`. Subsequently, it generates a list of document IDs associated with each attribute value, like `comedy` or `horror`. First of all, when a filter is applied, Meilisearch narrows down the potential results to the document IDs that fulfill the filter criteria.

Next, it uses the query words and the variations generated in the query graph and searches for matching words in the finite-state transducer (that is, our word dictionary). If the word is considered a prefix, it will also look it up in the prefix FST.

When encountering words in a FST, it searches them within the inverted index—which contains a mapping of words to document IDs— to retrieve the corresponding document IDs.

Finally, the engine performs an intersection to identify the documents that contain the words in the query graph and meet the filter criteria.

Let's take an example to better understand query processing: suppose you have a dataset of songs. The search query is `John Lennon`. The user wants to retrieve only songs released between 1957 and 1975.

First, Meilisearch retrieves the document IDs of songs within that time frame. Then, after checking in the FST that the words in the query graph exist, Meilisearch retrieves the IDs of the documents that contain either `John`, `Lennon`, or both. It also retrieves the possible variations, but we are not including them in this example for the sake of simplicity.

![Three Venn diagrams illustrating the process of retrieving songs released based on a user query (John Lennon) with a date filter (between 1057 and 1975).  Diagram 1: Represents songs released between 1957 and 1975. Diagram 2: Represents songs with "John" and/or "Lennon" in the title. Diagram 3: Represents the intersection between the two previous diagrams.](https://unable-actionable-car.media.strapiapp.com/diagrams_2_ac89ed0fa9.png)

Diagrams illustrating the operations involved in retrieving documents based on a user query.

Finally, the intersection of the two sets of document IDs is taken (the purple area in the diagram.) This means only document IDs that appear in both sets are kept. In other words, Meilisearch retains the document IDs of songs released between 1957 and 1975 that contain either `John`, `Lennon`, or both.

What happens when several documents match the search query, how does the engine decide which one to return first? Which one is more relevant? Let's explore the rules allowing Meilisearch to determine how to rank results.

### Relevancy

As we discussed earlier, the query graph includes possible variations of words. As a result, Meilisearch will also return documents with an artist, such as _John_ _Lebon_, as part of the search results.

Fortunately, the query graph not only accounts for word variations but also assigns them the Levenshtein cost, which, among other factors, helps Meilisearch determine the relevancy of the search results.

Meilisearch sorts documents in the search results using [bucket sort](https://en.wikipedia.org/wiki/Bucket_sort). This algorithm allows ranking documents based on a set of consecutive rules. By default, Meilisearch prioritizes rules as follows:

1.  Words: sorts documents based on the number of matched query terms, with documents containing all query terms ranked first
2.  Typo: sorts documents based on the number of typos, with documents matching query terms with fewer typos ranked first
3.  Proximity: sorts documents based on the distance between matched query terms. Documents where query terms occur close together and in the same order as the query string are ranked first
4.  Attribute: sorts documents based on the ranking order of attributes. Documents containing query terms in more important attributes are ranked first. Documents with query words at the beginning of attributes are considered more relevant than those with query words at the end
5.  Sort: sorts documents based on the user-defined parameters set at query time, if active
6.  Exactness: sorts documents based on the similarity of matched words with the query words

Meilisearch applies these rules sequentially, sorting results step by step. If two documents are tied after one rule, it uses the next rule to break the tie.

<Callout>
  Note that these rules are fully customizable, meaning you can add, delete, and reorder them as needed. Read more in the [relevancy documentation](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#ranking-rules).
</Callout>

By default, Meilisearch returns up to 1000 documents per search, prioritizing delivery of the most relevant results rather than all matching results. In other words, Meilisearch prioritizes efficiency and relevance over exhaustive results to ensure an optimized search experience.

Conclusion
----------

Search engines are complex systems that encompass multiple interconnected components, all working together to deliver a seamless search experience. While the inner workings may differ among search engines, we have explored the underlying mechanisms that power a modern search engine.

Meilisearch is currently exploring the vast realm of [**semantic search**](https://github.com/meilisearch/meilisearch/issues/3838). **AI-powered search** is reshaping the way we understand queries and documents, opening up new possibilities and use cases. If you’re eager to experience it firsthand, we invite you to try our [vector search prototype](https://github.com/meilisearch/product/discussions/621#discussioncomment-6183647)—your feedback would be greatly valuable!

Meilisearch is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience. Do you want to see Meilisearch in action? Try our [demo](https://where2watch.meilisearch.com/?ref=blog.meilisearch.com). You can also [run it locally](https://www.meilisearch.com/docs/learn/getting_started/quick_start?ref=blog.meilisearch.com) or create an account [on Meilisearch Cloud](https://cloud.meilisearch.com/login?ref=blog.meilisearch.com).

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

Meilisearch latest news and company updates