In today's interconnected world, many applications need to handle content in multiple languages. Meilisearch offers powerful capabilities for multilingual search, combining intelligent language handling with specialized models for optimal relevance. This guide will walk you through setting up multilingual search with Meilisearch, covering both full-text and semantic search approaches.

Why multilingual search matters
===============================

Before we dive into the technical details, let's consider why multilingual search is crucial:

1.  Global reach: Expand your audience by catering to users in their native languages.
2.  Improved user experience: Provide relevant results regardless of the language used in queries or content.
3.  Increased engagement: Users are more likely to interact with content they can easily find and understand.

Now, let's explore how Meilisearch makes multilingual search a breeze.

Full-text search: intelligent language handling
-----------------------------------------------

At its core, Meilisearch excels at handling diverse languages out of the box. Our [advanced tokenizer](https://www.meilisearch.com/docs/learn/indexing/tokenization?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) is designed to work with:

*   Languages that use whitespace to separate words (like English, Spanish, and French)
*   Languages with different writing systems (such as Chinese, Japanese, and Thai)

### Automatic language detection

When you add documents to your Meilisearch index, the engine automatically detects the language of each field. This means you don't need to specify languages manually in most cases - Meilisearch will handle it for you.

For example, if your documents contain a mix of English and Spanish content:

```json
[
  {
    "id": 1,
    "title": "The quick brown fox",
    "description": "El zorro marrón rápido salta sobre el perro perezoso"
  },
  {
    "id": 2,
    "title": "La luna y las estrellas",
    "description": "A beautiful night sky filled with twinkling stars"
  }
]
```

Meilisearch will automatically detect and properly handle the different languages in each field, ensuring accurate search results across your multilingual content.

### Specifying languages for specific fields

While automatic detection works well in most cases, you might want more control over language handling for certain fields. Meilisearch allows you to [specify languages](https://www.meilisearch.com/docs/reference/api/settings#localized-attributes?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) for particular attributes using the `localizedAttributes` setting.

Here's how you can set it up:

```json
{
  "localizedAttributes": [
    {
      "locales": ["eng"],
      "attributePatterns": ["title_en", "*_english"]
    },
    {
      "locales": ["spa"],
      "attributePatterns": ["description_es", "*_spanish"]
    }
  ]
}
```

This configuration tells Meilisearch to treat fields ending with `_en` or `_english` as English, and fields ending with `_es` or `_spanish` as Spanish. This approach is particularly useful when you have a consistent naming convention for your multilingual fields or when you want to ensure specific treatment for certain attributes.

Semantic search: Enhancing multilingual relevance
=================================================

While full-text search works well for many multilingual scenarios, semantic search can provide even better results, especially for complex queries or when dealing with nuanced meanings across languages.

Meilisearch supports various embedding models for semantic search, offering powerful auto-embedding capabilities. This means you can easily set up semantic search without having to manage the embedding process yourself. Meilisearch handles the generation and storage of embeddings automatically, seamlessly integrating them into your search experience.

For multilingual applications, we recommend using specialized multilingual models rather than relying on general-purpose models. These models are designed to understand and represent meaning across multiple languages, leading to more relevant search results in multilingual contexts.

One excellent option for multilingual semantic search is the [Cohere Multilingual embedding model](https://docs.cohere.com/docs/multilingual-language-models). It offers a great balance of speed and relevance across a wide range of languages. Cohere's model is particularly adept at understanding context and nuance in different languages, making it an ideal choice for diverse, multilingual content. Follow [the guide](https://www.meilisearch.com/docs/guides/ai/cohere?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search) if you want to do it now.

To explore different models and their performance, you can use the [Meilisearch playground](https://playground.meilisearch.com). This interactive tool allows you to test various embedding models and see how they perform with your specific multilingual queries and content.

Conclusion
----------

We encourage you to experiment with different settings and models to find the optimal configuration for your specific use case. And remember, the Meilisearch community is always here to help - join us on [Discord](https://discord.meilisearch.com/) or check out our documentation for more detailed information and support.

Ready to get started with multilingual search? Try Meilisearch today [with a 14 day trial](https://cloud.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=multilingual_search)!

Searching across multiple languages

Searching across disparate data sources can be a real headache. But aggregating information from various databases or indices enables the creation of more relevant search experiences for users. In this guide, we'll explore what federated search is and the use cases it unlocks.

What is federated search?
-------------------------

Federated search empowers you to conduct searches across multiple systems simultaneously. In the context of a search engine, federated search allows querying numerous indices and retrieving the results in a single, consolidated list.

This differs from having a monolithic index that stores various types of resources where documents could be filtered by type. Leveraging multiple indices enables the implementation of specific ranking rules for each index.

Federated search extends the capabilities of [multi-index search](__GHOST_URL__/introducing-multi-search/?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search), merging results into a unified, cohesive response. This allows surfacing the most relevant results across the entire dataset, rather than just the top hits for each individual index.

Federated search is a crucial feature for SaaS applications, where users frequently need to search through all available resources regardless of type. An example you might be familiar with is searching in Slack—a single search bar to find users, messages, and shared documents.

What are the benefits of federated search?
------------------------------------------

Integrating federated search into your application can facilitate more convenient search experiences. By supporting searches across multiple sources, you can build one search bar to query all the data. This eliminates the need for multiple menus or lists of results, resulting in a more intuitive and user-friendly interface.

The search experience is further enhanced by the increased relevance of the results. The comprehensiveness of the returned resources will match items that the user may not have initially considered including in their search (e.g., discovering a prospect's email in meeting notes instead of just contact details).

Uncovering the most pertinent result can be transformative for your users and your business. In an internal tool, relevant results enable users to make more informed decisions. However, in customer-facing applications, it can be the pivotal factor between converting a sale or not.

Locating information effortlessly within your application or e-commerce platform leads to higher customer satisfaction and improved retention. This allows users to extract more value from the service, ultimately elevating the overall experience quality.

How federated search improves developer experience
--------------------------------------------------

Implementing federated search leads to a more streamlined search implementation process. With federated search, results are delivered in a single, unified list, simplifying front-end development. This goes beyond [multi-index search](__GHOST_URL__/introducing-multi-search/?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search) because it comes without the added complexity of implementing front-end logic to merge, sort, and paginate the results.

In this setup, the relevance of documents can be further fine-tuned thanks to granular, per-index configuration. This allows tailoring the [relevancy settings](https://www.meilisearch.com/docs/learn/relevancy/relevancy?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search) to specific data types (a specific index) rather than the entire dataset (all indices). As a federated search request comes, the search engine can ensure that the most important results rise to the top.

This also simplifies extending the search functionality if new types of documents need to be included in the future. You'll only need to focus on configuring the relevancy settings for the new document type, instead of revisiting the relevancy for the entire dataset.

Conclusion
----------

Federated search enables performing searches against multiple indices of a search engine and returns a single, consolidated list of results. With a search engine that supports federated search, developers can build more relevant user search experiences without incurring additional complexity.

Federated search is available in [Meilisearch 1.10](__GHOST_URL__/meilisearch-1-10/?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search) and higher. Learn more about implementing federated search with Meilisearch by [reading the documentation](https://www.meilisearch.com/docs/reference/api/multi_search?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search#federated-multi-search). Check out which search engines support federated search and other key features via our [comparison of search engines](https://www.meilisearch.com/docs/learn/resources/comparison_to_alternatives?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search).

* * *

[Meilisearch](https://meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) is an open-source AI-powered search engine enabling developers to build state-of-the-art experiences while enjoying simple, intuitive DX.

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search). You can learn more about Meilisearch by looking at the [roadmap](https://roadmap.meilisearch.com/?utm_campaign=federated-search&utm_source=blog&utm_medium=what-is-federated-search) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?ref=blog.meilisearch.com).

What is federated search?

  
[Semantic search](__GHOST_URL__/what-is-a-vector-database/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison#what-is-similarity-search) is transforming search technology by providing more accurate and relevant results. However, with many embedding models available, choosing the right one can be challenging. This guide will help you understand the key factors to consider when selecting a model to build semantic search.

Overview
--------

In this guide, we will use the open-source search engine [Meilisearch](https://www.meilisearch.com?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) to perform the semantic searches. For the purpose of the tests, we're using the entry tier of [Meilisearch Cloud](https://www.meilisearch.com/cloud?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) (i.e., the [Build plan](https://www.meilisearch.com/pricing?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison).)

This guide will cover the following models:

| Model/Service | Dimensions | Context Length |
| --- | --- | --- |
| [Cohere embed-english-v3.0](https://docs.cohere.com/docs/embeddings) | 1024 | 512 |
| [Cohere embed-english-light-v3.0](https://docs.cohere.com/docs/embeddings) | 384 | 512 |
| [Cohere embed-multilingual-v3.0](https://docs.cohere.com/docs/embeddings) | 1024 | 512 |
| [Cohere embed-multilingual-light-v3.0](https://docs.cohere.com/docs/embeddings) | 384 | 512 |
| [OpenAI text-embedding-3-small](https://platform.openai.com/docs/guides/embeddings) | 1536 | 8192 |
| [OpenAI text-embedding-3-large](https://platform.openai.com/docs/guides/embeddings) | 3072 | 8192 |
| [Mistral](https://mistral.ai/product/embeddings/) | 1024 | 8192 |
| [VoyageAI voyage-2](https://voyageai.com/) | 1024 | 4000 |
| [VoyageAI voyage-large-2](https://voyageai.com/) | 1536 | 16000 |
| [VoyageAI voyage-multilingual-2](https://voyageai.com/) | 1024 | 32000 |
| [Jina Colbert v2](https://jina.ai/news/jina-colbert-v2-multilingual-late-interaction-retriever-for-embedding-and-reranking/) | 128, 96, or 64 | 8192 |
| [OSS all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 384 | 512 |
| [OSS bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | 1024 | 512 |
| [OSS bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | 1536 | 512 |

Factors to consider
-------------------

### 1\. Results relevancy

[Relevancy](https://www.meilisearch.com/docs/learn/relevancy/relevancy?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) is crucial for effective search, as it ensures that users find the most pertinent results quickly. In the realm of semantic search, achieving a balance between relevancy and speed is essential to provide a seamless user experience. It's important to consider the tradeoffs of [vector search vs full-text search](__GHOST_URL__/full-text-search-vs-vector-search/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison).

When selecting a model, consider your specific use case, such as the need for multilingual support, handling multi-modal data, or addressing domain-specific requirements. If you have a highly specialized use case or need to support a particular language, it may be beneficial to explore models that can be trained on your data or opt for multilingual models.

The performance difference between a very small model and a large model is not always substantial. **Smaller models are generally less expensive and faster**, making them a practical choice in many scenarios. Therefore, it is often worth considering smaller models for their cost-effectiveness and speed.

Additionally, you should always consider the context you're providing to the model. In Meilisearch, this comes in the form of a [document template](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison#embedders-object). The more accurately the template describes the data, the better the search results will be, leading to a more satisfying user experience.

### 2\. Search performance

You read everywhere now. Time is money. And the web is no different. Nowadays, [search-as-you-type](https://www.meilisearch.com/docs/learn/getting_started/what_is_meilisearch?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison#features) is the baseline for customer-facing applications. Saving users time greatly enhances their satisfaction and keeps them engaged with your platform.

To achieve lightning-fast search performance, **consider using a local model** to minimize latency by eliminating the need for round trips to the embedding service. If you need to use a remote model, then hosting your search service (e.g., your Meilisearch database) in close proximity to the embedding service can significantly reduce latency.

The table below showcases latency benchmarks for various local embedding models and embedding APIs. All requests are originated from a Meilisearch instance hosted on AWS (London datacenter.)

| Model/Service | Latency |
| --- | --- |
| [Cloudflare bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | ±800ms |
| [Cloudflare bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | ±500ms |
| [Cohere embed-english-v3.0](https://docs.cohere.com/docs/embeddings) | ±170ms |
| [Cohere embed-english-light-v3.0](https://docs.cohere.com/docs/embeddings) | ±160ms |
| [Local gte-small](https://huggingface.co/sentence-transformers/gte-small) | ±20ms |
| [Local all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | ±10ms |
| [Local bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | ±20ms |
| [Local bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | ±60ms |
| [Mistral](https://mistral.ai/product/embeddings/) | ±200ms |
| [Jina colbert](https://huggingface.co/jinac/cobert-v1) | ±400ms |
| [OpenAI text-embedding-3-small](https://platform.openai.com/docs/guides/embeddings) | ±460ms |
| [OpenAI text-embedding-3-large](https://platform.openai.com/docs/guides/embeddings) | ±750ms |
| [VoyageAI voyage-2](https://voyageai.com/) | ±350ms |
| [VoyageAI voyage-large-2](https://voyageai.com/) | ±400ms |

Here you can see that there are some clear winners in terms of latency. Unfortunately, latency is not the same as throughput, so we also need to take a close look at the indexing time.

### 3\. Indexing performance

Indexing performance is another critical aspect when [comparing search solutions](https://www.meilisearch.com/docs/learn/resources/comparison_to_alternatives?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison). The embedding model performance will directly impact the indexing speed of your search solution. And the speed at which your data can be indexed directly impacts the overall efficiency and scalability of your search solution.

Local **models without GPUs may have slower indexing** due to limited processing power. In contrast, third-party services offer varying speeds and limitations based on their infrastructure and service agreements. It is essential to evaluate these factors to ensure that your chosen model and service can meet your requirements effectively.

Several factors come into play when optimizing indexing. Again, the latency plays a big role: reducing the time taken for data to travel between your application and the model is always going to improve your experience. Additionally, the **maximum size of API calls** the API accepts, the provider's **rate limiting**, and the model's supported **number of dimensions** can all influence the efficiency and scalability of the indexing process.

The benchmark below compares the indexing of a 10k e-commerce documents (with [automatic embedding generation](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison#generate-auto-embeddings)):

| Model/Service | Indexation Time |
| --- | --- |
| [Cohere embed-english-v3.0](https://docs.cohere.com/docs/embeddings) | 43s |
| [Cohere embed-english-light-v3.0](https://docs.cohere.com/docs/embeddings) | 16s |
| [OpenAI text-embedding-3-small](https://platform.openai.com/docs/guides/embeddings) | 95s |
| [OpenAI text-embedding-3-large](https://platform.openai.com/docs/guides/embeddings) | 151s |
| [Cloudflare bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | 152s |
| [Cloudflare bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | 159s |
| [Jina Colbert V2](https://huggingface.co/jinac/cobert-v1) | 375s |
| [VoyageAI voyage-large-2](https://voyageai.com/) | 409s |
| [Mistral](https://mistral.ai/product/embeddings/) | 409s |
| [Local all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 880s |
| [Local bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | 3379s |
| [Local bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | 9132s |

### 4\. Pricing

While local embedders are free, most services charge per million of tokens. Here's a breakdown of the pricing for each platform:

*   [Cohere](https://docs.cohere.com/docs/pricing):
    *   $0.10 per million tokens
*   [OpenAI](https://openai.com/pricing):
    *   $0.13 per million tokens for text-embedding-3-large
    *   $0.02 per million tokens for text-embedding-3-small
*   [Cloudflare](https://developers.cloudflare.com/workers-ai/platform/pricing/):
    *   $0.011/1,000 Neurons
*   [Jina](https://jina.ai/pricing/):
    *   $0.18 per million tokens
*   [Mistral](https://mistral.ai/product/embeddings/):
    *   $0.10 per million tokens
*   [VoyageAI](https://voyageai.com/pricing):
    *   $0.10 per million tokens for voyage-2
    *   $0.12 per million tokens for voyage-large-2
    *   $0.12 per million tokens for voyage-multilingual-2
*   Local model: Free

As your search needs grow and scale, it may become more cost-effective to invest in your own GPU machine. By having your own hardware, you can have greater control over the performance and scalability of your search solution and potentially reduce costs in the long run.

It is often best to start with a well-known model from the list provided. They are generally easy to setup and you will easily find community resources to help you.. As the need arises, you can consider migrating the model to a cloud provider like AWS. Many services offer this option, allowing you to leverage their infrastructure for improved performance and scalability.

Alternatively, you can choose an equivalent open-source model to self-host, giving you even more flexibility and control over your search solution in the long term. Please note that optimizing local models for performance or high volume may require to scale your infrastructure accordingly.

* * *

Ready to elevate your search experience?

[Talk to a search expert](https://dub.sh/meili-sales?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison)

* * *

Going further
-------------

While this article provides a comprehensive overview, we did not delve deeply into optimization techniques. There are several additional optimizations that can be explored to further enhance the performance of semantic search.

Here is a list of additional areas to investigate when choosing a model for your search experience:

*   Experiment with different presets (query vs. document) for models that offer this option to potentially improve relevancy
*   Evaluate specialized models for specific applications to assess their performance and suitability for your use case
*   Explore models that provide a reranking function to further refine search results
*   Test higher-tier accounts on each platform to check for improved performance and reduced rate limiting
*   Investigate parameters for receiving quantized data directly from the API to optimize data transfer and processing

Conclusion
----------

| Model/Service | Dimensions | Context Length | Latency | Indexation Time | Pricing (per million tokens) |
| --- | --- | --- | --- | --- | --- |
| [Cohere embed-english-v3.0](https://docs.cohere.com/docs/embeddings) | 1024 | 512 | ±170ms | 43s | $0.10 |
| [Cohere embed-english-light-v3.0](https://docs.cohere.com/docs/embeddings) | 384 | 512 | ±160ms | 16s | $0.10 |
| [OpenAI text-embedding-3-small](https://platform.openai.com/docs/guides/embeddings) | 1536 | 8192 | ±460ms | 95s | $0.02 |
| [OpenAI text-embedding-3-large](https://platform.openai.com/docs/guides/embeddings) | 3072 | 8192 | ±750ms | 151s | $0.13 |
| [Mistral](https://mistral.ai/product/embeddings/) | 1024 | 8192 | ±200ms | 409s | $0.10 |
| [VoyageAI voyage-2](https://voyageai.com/) | 1024 | 4000 | ±350ms | 330s | $0.10 |
| [VoyageAI voyage-large-2](https://voyageai.com/) | 1536 | 16000 | ±400ms | 409s | $0.12 |
| [Jina Colbert v2](https://jina.ai/news/jina-colbert-v2-multilingual-late-interaction-retriever-for-embedding-and-reranking/) | 128, 96, or 64 | 8192 | ±400ms | 375s | $0.18 |
| [OSS all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 384 | 512 | ±10ms | 880s | Free |
| [OSS bge-small-en-v1.5](https://huggingface.co/Cloudflare/bge-small-en-v1.5) | 1024 | 512 | ±20ms | 3379s | Free |
| [OSS bge-large-en-v1.5](https://huggingface.co/Cloudflare/bge-large-en-v1.5) | 1536 | 512 | ±60ms | 9132s | Free |

Choosing the right model and service for semantic search involves carefully balancing several key factors: relevancy, search performance, indexation performance, and cost.

Each option presents its own set of trade-offs:

*   Cloud-based services like Cohere and OpenAI offer excellent relevancy and reasonable latency, with Cohere's embed-english-light-v3.0 standing out for its balance of speed and performance.
*   Local models provide the fastest search latency but may struggle with indexation speed on limited hardware.
*   Emerging services like Mistral and VoyageAI show promise with competitive pricing and performance.
*   Open-source models offer cost-effective solutions for those willing to manage their own infrastructure.

Ultimately, the best choice depends on your specific use case, budget, and performance requirements. For many applications, starting with a cloud-based service like Cohere or OpenAI provides a good balance of ease of use, performance, and cost. As your needs grow, consider exploring local or specialized models, or contact Meilisearch's sales team for tailored solutions.

* * *

[Meilisearch](https://www.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) is an open-source search engine enabling developers to build state-of-the-art experiences while enjoying simple, intuitive DX.

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison). You can learn more about the product by checking out its [roadmap](https://roadmap.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=models-comparison) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions).

Choosing the best model for semantic search

Deciding between Algolia and Meilisearch for your search solution? We're here to help! In this article, we'll compare their features, capabilities, pricing models, and overall strengths and weaknesses.

What is Algolia?
----------------

Founded in 2012, Algolia is a hosted search engine that provides full-text and AI-powered search capabilities. It offers an API that integrates fast, customizable search experiences into websites and mobile applications.

Algolia is known for its remarkable speed and scalability, making it a go-to choice for enterprises looking for sophisticated search functionalities. However, these premium features come with a higher cost, Algolia tends to be more suitable for larger businesses with bigger budgets.

What is Meilisearch?
--------------------

Launched in 2018, [Meilisearch](https://meilisearch.com/?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia) is an open-source search engine that features full-text and AI-powered search capabilities Designed to be developer-friendly and plug-and-play, it addresses most common use cases straight out of the box. Built with Rust, it ensures exceptional performance, security, and flexibility.

Known for its lightning-fast search speeds and modern development environment, Meilisearch is an excellent choice for small-to-medium growing businesses seeking high-performance search functionalities.

Unlike Algolia, Meilisearch is [open-source](https://github.com/meilisearch/meilisearch). You can self-host it or use the [managed Meilisearch Cloud service](https://meilisearch.com/cloud?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia), allowing for flexibility and cost-efficiency.

Algolia vs Meilisearch - Key features compared
----------------------------------------------

### Developer experience

Algolia and Meilisearch both prioritize ease of use, offering straightforward setup processes and comprehensive documentation. These search engines work out-of-the-box, providing great relevancy in most cases without the need for additional configurations. They feature intuitive REST APIs and place a strong emphasis on the developer experience. Meilisearch's open-source nature is a significant asset, as it enables better feedback collection and faster implementations.

When comparing Meilisearch Cloud and Algolia dashboards, Algolia’s interface stands out for its advanced and comprehensive UI. This is understandable, given that Algolia is a fully-hosted search engine that has been in the market for over a decade. In contrast, Meilisearch, which launched its cloud offering much more recently, is still developing its dashboard.

Overall, the Algolia dashboard feels more beginner-friendly and integrated, guiding users smoothly from data addition to frontend integration. Meilisearch, on the other hand, partially relies on its [documentation](https://www.meilisearch.com/docs/learn/getting_started/cloud_quick_start?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia) to guide users through the process. However, it’s important to note that Meilisearch’s dashboard is continually evolving based on user feedback, showing significant potential for improvement.

### SDKs and integrations

Both Algolia and Meilisearch provide [API clients](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia) for a variety of popular programming languages, including JavaScript, .NET, Python, PHP, and more, ensuring developers can easily integrate search functionalities into their applications.

Algolia offers a significantly broader selection of SDKs and integrations, particularly in the realm of specialized e-commerce platforms such as Shopify and Adobe Commerce (formerly Magento). This extensive range of integrations makes Algolia a versatile choice for businesses with specific platform requirements.

### Relevance and typo tolerance

Meilisearch and Algolia both deliver excellent relevancy and typo tolerance right out of the box, with options for customization to fine-tune search results. They offer similar features, including synonyms, deduplication, sorting, filtering, faceting, and geosearch.

However, configuring some of these features in Meilisearch can feel more straightforward. Certain functionalities, such as [sorting](https://www.meilisearch.com/docs/learn/fine_tuning_results/sorting#sorting?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia), require fewer configuration steps compared to Algolia. This simplicity makes Meilisearch an attractive option for developers looking for ease of setup and use.

Moreover, unlike Algolia, Meilisearch does not require replicating or copying indexes for each sort criterion, making cost management much more manageable and fair for customers.

Algolia, on the other hand, provides a more extensive feature set with advanced configuration options. Features like filter weighting offer more customization, but this can also add complexity. While Algolia's features can sometimes offer more advanced configuration, Meilisearch is rapidly closing the gap, [regularly launching new features](https://github.com/meilisearch/meilisearch/releases?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia) that enhance its capabilities.

### AI-powered search

Both Algolia and Meilisearch offer advanced semantic search capabilities, but with notable differences in accessibility and customization.

Algolia provides NeuralSearch, its approach to hybrid search, which combines neural search and full-text search results. Algolia leverages AI at multiple levels, including search, ranking, and query suggestions. This feature allows you to choose between keyword search and neural search modes. When using Algolia, text embedding is managed by their proprietary model, which is automatically applied to the documents. However, NeuralSearch is not widely available; it is reserved for the highest plan (Elevate) and requires event data from the front end. In practice, many users have yet to see a running instance of NeuralSearch. 

Meilisearch, on the other hand, offers a more customizable [AI-powered search](https://www.meilisearch.com/solutions/vector-search?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia) experience. It uses state-of-the-art large language model (LLM) providers such as OpenAI and HuggingFace to generate vector embeddings. Additionally, Meilisearch allows you to configure multiple embedders for an index and supports custom embeddings, ensuring flexibility and adaptability. Its [hybrid search](https://www.meilisearch.com/solutions/hybrid-search?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia) approach is highly customizable, enabling you to adjust the semantic ratio for each hybrid search query, ranging [from full semantic to full-text search](__GHOST_URL__/full-text-search-vs-vector-search/?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia). Meilisearch's AI-powered search is available both when self-hosting and on Meilisearch Cloud, making it accessible to a wide range of users.

### Analytics & data visualization

Both Meilisearch and Algolia provide robust analytics and data visualization tools to help you monitor and optimize search performance, but they offer these insights in slightly different ways.

Meilisearch offers a [comprehensive set of analytics](https://www.meilisearch.com/docs/learn/cloud/analytics?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia), including metrics such as total searches, user activity, and key search effectiveness indicators like no result rate and click-through rate. You can visualize search trends over time and gain insights into user behavior, including the most searched queries and geographic distribution of searches.

Algolia, on the other hand, provides a more extensive suite of analytics and data visualization tools. Its overview dashboard includes similar key metrics like total searches, user activity, and conversion rates. However, Algolia goes further by offering advanced analytics providing deeper insights into popular filter values and top-performing results.

A notable feature that sets Algolia apart is its A/B testing capability. This allows you to compare an index and its replica with different settings, evaluating metrics like click-through rates and conversion rates. 

In summary, while both Meilisearch and Algolia offer essential analytics tools, Algolia includes more detailed and advanced options, including A/B testing, which can be particularly beneficial for optimizing user interactions in ecommerce use cases. Meilisearch focuses on core metrics and delivers a straightforward visualization of search trends and user behavior.

### Community and support

Algolia offers a dedicated Discord community with 507 members, accessible to all users, providing a platform for discussions, questions, and support. For more personalized assistance, email and website support are available starting from the Grow plan. Additionally, Algolia provides Standard and Extended Support options, though pricing is not disclosed and is only available through sales inquiries.

Meilisearch, being an open-source search engine, has fostered a robust community. Its [Discord community](https://discord.meilisearch.com/?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia) boasts 2,441 members, offering a vibrant space for open-source enthusiasts and users to engage, share knowledge, and seek help. For Cloud users, Meilisearch ensures comprehensive support tailored to each plan: the Build plan includes Discord support, the Pro plan offers chat and email support, and the Custom plan provides priority support with dedicated channels. This tiered support structure reflects Meilisearch's commitment to accessible and responsive customer service.

### Pricing and commercial offerings

Algolia offers a mix of free, pay-as-you-go, and custom pricing plans. The Free Plan includes up to 10,000 search requests per month and 1 million records, while the Grow Plan features a clear pay-as-you-go structure with specific costs for additional search requests and records. However, their Premium and Elevate Plans do not have disclosed pricing and require users to request details from sales.  

In contrast, Meilisearch Cloud has fixed monthly pricing for its Build and Pro plans, priced at $30 and $300 per month, respectively, with specified features and resources included. For larger projects, Meilisearch has a Custom Plan that requires a custom quote.

While Meilisearch does not offer a free tier, it starts at a relatively low monthly cost with substantial resources and support. Additionally, it includes a 14-day free trial without requiring a credit card, making it easy for potential users to explore its features. Moreover, Meilisearch being open-source, it can also be [self-hosted for free](https://www.meilisearch.com/docs/learn/getting_started/installation#local-installation?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia).

Although Algolia might save some costs in specific scenarios, its complexity and potential hidden costs can lead to higher expenses in practice. Meilisearch, with its clear and predictable [pricing model](https://www.meilisearch.com/pricing?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia), is often more attractive for businesses that prefer straightforward budgeting and cost transparency.

### Global coverage

Meilisearch Cloud has server regions in major cities such as San Francisco, New York, London, Frankfurt, and Singapore, strategically positioning itself in key commercial hubs across North America, Europe, and Asia. This coverage allows Meilisearch to provide reliable performance and data residency options in these important regions.

Algolia has a much broader network with clusters deployed in 16 regions globally. This extensive coverage spans the US, Europe, Asia, and additional regions like Brazil, Japan, and Australia. Algolia’s expansive network allows for greater flexibility in data localization, enhancing performance and compliance with local regulations.

Looking ahead, Meilisearch plans to expand its reach by adding more server regions to its network. This expansion will broaden its global coverage, further enhancing its service offerings.

Algolia challenges
------------------

Algolia faces several notable challenges, one of which is its high cost. While Algolia does offer a free tier, it is limited, and scaling up can quickly become expensive. Algolia's powerful features are well-suited for large enterprises, but smaller businesses might find these advanced features and the associated costs overwhelming. The lack of transparency for its Premium and Elevate plans further complicates budgeting, as potential customers must engage with the sales team to get detailed pricing information, which can deter smaller businesses that need to understand costs upfront. 

Additionally, fully leveraging Algolia’s extensive features and capabilities often requires significant customization and technical expertise. This can be a barrier for businesses without dedicated technical teams. 

Furthermore, Algolia's reliance on its proprietary models for AI features means users are dependent on Algolia's technology stack. This dependency can limit businesses looking for more flexibility or integration with other AI solutions.

Meilisearch challenges
----------------------

Meilisearch, despite its strengths, also faces its own set of challenges. One notable challenge is the absence of a completely free tier in the managed cloud version, which might deter very small businesses or startups looking to minimize costs. While Meilisearch's pricing is transparent and reasonable, the lack of a free option in the managed cloud version can be a disadvantage compared to competitors like Algolia. However, being open-source, Meilisearch provides a free, self-hosted option that ensures cost savings for those willing to manage their own infrastructure.

While Meilisearch offers essential and robust features, it might lack some of the advanced features and integrations that Algolia provides, particularly for specialized use cases such as advanced e-commerce applications.

When should you consider Algolia over Meilisearch?
--------------------------------------------------

If your business requires sophisticated e-commerce search functionalities, such as merchandising tools, product recommendations, and advanced filtering options, Algolia is currently better equipped to handle these needs. The platform's features are designed to improve product discovery and conversion rates.

For businesses that want to optimize their search configurations through A/B testing, Algolia provides robust tools to compare different search setups. This is especially beneficial in e-commerce, where fine-tuning search performance can significantly impact sales and customer engagement. 

Algolia’s infrastructure is also optimized for global reach, offering high availability and low latency across different regions. If your business operates internationally and requires consistent performance worldwide, Algolia is a strong contender.

In summary, Algolia excels in delivering advanced e-commerce functionalities, comprehensive analytics, and a globally optimized infrastructure. While it may come with higher costs and a steeper learning curve, these features make it particularly suitable for large enterprises and businesses with complex, high-demand search requirements.

When should you consider Meilisearch over Algolia?
--------------------------------------------------

Meilisearch is an excellent choice for businesses that prioritize simplicity, transparency, and flexibility. Its straightforward setup process and developer-friendly design make it ideal for teams seeking a hassle-free implementation. With robust core features that work out of the box, Meilisearch ensures high performance, fast response times, and excellent relevancy without requiring extensive customization or technical expertise.

For organizations that value open-source solutions, Meilisearch is particularly appealing. It benefits from active community contributions and maintenance, fostering a collaborative environment that accelerates innovation. This allows businesses to adapt and extend the search engine according to their specific requirements, providing greater flexibility and control over their search infrastructure.

The open-source model also ensures the search engine remains aligned with user needs, adding a layer of transparency and flexibility. As a result, customers gain better confidence in their ability to shape the product and steer it in the direction they need, enhancing their overall control and adaptability.

Regular updates every two months ensure that Meilisearch users can benefit from the latest advancements and enhancements, keeping the search engine up-to-date with evolving needs and technologies. This continuous improvement cycle makes Meilisearch a strong contender for businesses looking for a dynamic and easy-to-use search solution.

In summary, choose Meilisearch over Algolia if you need a flexible, developer-friendly search engine with transparent pricing and the benefits of an open-source community. Meilisearch’s focus on simplicity, high performance, and continuous innovation makes it a compelling option for businesses of all sizes.

* * *

Learn more about what Meilisearch can bring to your business.

[Talk to a search expert](https://dub.sh/meili-sales?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia)

* * *

Conclusion
----------

Both Algolia and Meilisearch offer robust search solutions with a developer-friendly experience and high performance.

Algolia is particularly strong in the e-commerce space, offering advanced analytics and features tailored to that use case. Its extensive feature set and flexible scalability options make it a great choice for larger enterprises looking for comprehensive search capabilities.

Meilisearch excels with its advanced AI capabilities, fast development cycle, and transparent pricing, making it an excellent choice for tech teams and app builders who want to be efficient and achieve fast ROI while benefiting from a secure, scalable search solution.

If you're considering making the switch, check out our [Migrating from Algolia to Meilisearch](https://www.meilisearch.com/docs/learn/update_and_migration/algolia_migration?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia) guide for a smooth transition.

You can see Meilisearch in action using this online [demo](https://where2watch.meilisearch.com/?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-algolia). Also, feel free to check out its [public roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and [GitHub community](https://github.com/orgs/meilisearch/discussions?ref=blog.meilisearch.com) to see what new and upcoming features you can look forward to.

Meilisearch vs Algolia

Navigating today’s ocean of information can be both a marvel and a maze. Full-text and vector search enable us to build search experiences empowering users to find relevant products, content, and more. As our quest for precision and context in search evolves, a question arises: can we balance the lexical flexibility of full-text search with the semantic depth of vector search?

Let’s explore together the pros and cons of each solution and discover the synergies that are redefining modern search & discovery experiences.

Full-text search
----------------

[Full-text search](https://www.meilisearch.com/docs/learn/what_is_meilisearch/overview?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector#features) refers to matching some or all of a text query with documents stored in a database. Compared to traditional database queries, full-text search provides results even in case of partial matches. It allows building more flexible search interfaces for users, thus enabling them to find accurate results more quickly.

In practice, an efficient full-text search solution comes with tolerance to typos, synonyms, prefix search, and fuzzy matching. These features enable end users to find what they’re looking for even when the information at their disposal is limited. Allowing searches with incomplete information makes discovery faster.

Although modern SQL and NoSQL databases come with some full-text search capabilities, they often fall short compared to search-focused databases. For a deeper dive on this topic, consider reading on areas where [Postgres falls short for full-text search](__GHOST_URL__/postgres-full-text-search-limitations/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector).

<Callout>
  Learn about the [inner workings of a search engine](__GHOST_URL__/how-full-text-search-engines-work/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) in our dedicated article.
</Callout>

Vector search
-------------

[Vector search](https://www.meilisearch.com/solutions/vector-search?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) is the AI-powered approach to search. Instead of looking for documents matching a text query, it allows finding documents that have a similar _semantic_ meaning. This is achieved by building upon the semantic understanding of text that [Large Language Models](https://en.wikipedia.org/wiki/Large_language_model) (LLMs) provide.

LLMs can process database records and generate [vector embeddings](__GHOST_URL__/what-are-vector-embeddings/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector)—the numeric representation of the semantic meaning of documents. These embeddings are then stored in [vector databases](__GHOST_URL__/what-is-a-vector-database/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector). As the name suggests, vector search works by comparing similar vectors.

By performing vector comparisons, vector search effectively enables [semantic search](__GHOST_URL__/what-is-a-vector-database/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector#what-is-similarity-search). Despite relying on different underlying technologies, it also solves a similar purpose as full-text search: empowering users by improving the discovery aspect of search. It notably enables multi-modal search—searching with non-textual input, like in Google Images.

Comparative analysis: full-text search vs vector search
-------------------------------------------------------

Full-text search works by finding words in documents. That makes it good at understanding the **lexical intent** behind a user query. But it falls short when it has to work with vague queries that cannot be fuzzy-matched. The lack of semantic understanding of documents makes it unable to understand context and make generalizations.

To summarize, **full-text search**:

*   ✅ Corrects typos — e.g. `reutrn of the jedi`
*   ✅ Handles precise queries — e.g. `an exact product name`
*   ✅ Handles incomplete queries — e.g. `return of the j`
*   ❌ Cannot handle vague queries — e.g. `folks fighting with lightsabers`
*   ❌ Cannot understand context — e.g. `winter clothes`

On the other hand, vector search works by finding documents with similar meanings. That makes it good at understanding the **semantic intent** behind a user query. It’s great at understanding synonyms and descriptions of documents. It’s also adapted to the discovery of similar documents.

To summarize, **vector search**:

*   ✅ Handles vague queries — e.g. first released star wars movie
*   ✅ Understands context — e.g. winter clothes
*   ✅ Can suggest similar documents
*   ❌ Cannot handle precise queries
*   👎 Requires more resources

As we can see, neither solution is perfect. They each come with clear advantages for given use cases. They complement each other. Building state-of-the-art search experiences would require combining the benefits of both full-text and vector search.

![“Why don't we have both?“ meme](https://unable-actionable-car.media.strapiapp.com/why_dont_we_have_both_1e31f67a5c.gif)

Indeed, can we have both?

The case for hybrid search
--------------------------

Hybrid search combines the strengths of full-text search and vector search. It builds upon the accessible, search-as-you-type experience of full-text search and integrates the enhanced discovery capabilities that AI search enables.

Solutions like Meilisearch’s [hybrid search](https://www.meilisearch.com/solutions/hybrid-search?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) bring the best of both worlds without compromise. This allows developers to use a unified API for all types of searches. The code sample below demonstrates a usage of such API.

```js
// Hybrid search with Meilisearch

const results = await client.multiSearch({
	queries: [{ 
		indexUid: 'movies', 
		q: 'batman',
		hybrid: { embedder: 'default', semanticRatio: 0.5 }
	}]
})

```
<Callout>
  Not using JavaScript? Meilisearch supports [multiple SDKs](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector).
</Callout>

This code uses a Meilisearch client to search for the `batman` query in the movies index. It uses hybrid search, which is configured via the `hybrid` parameter. Let’s review the fields in the `hybrid` object and see what they enable.

The `embedder` field allows choosing which embedder will process the query (Meilisearch allows [configuring multiple embedders](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector#generate-vector-embeddings) for a given index.)

The `semanticRatio` field allows us to balance the importance of results coming from semantic search. A higher value will allow documents that are further distanced in the [semantic space](__GHOST_URL__/what-are-vector-embeddings/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector#what-is-the-semantic-space) to make it to the final search results.

The `semanticRatio` value works as follows:

*   Semantic ratio = 0 — it’s a full-text search
*   Semantic ratio = 1 — it’s a vector search
*   0 < semantic ratio < 1 — it’s a hybrid search ✨

As semantic search will always return _some_ documents, it’s important to strike a good balance to keep the overall search results relevant. The demo below allows you to search for movies and play with the semantic ratio to see how it affects results.

[![Searching for movies using hybrid search](https://unable-actionable-car.media.strapiapp.com/hybrid_search_gif_cfa6283c31.gif)](https://where2watch.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector)

Searching for ”green ogre in fairy tale parody” ([Live demo](https://where2watch.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector))

* * *

Hybrid search is coming to Meilisearch Cloud, join the waitlist:

[Get early access](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector)

Conclusion
----------

As engineers will tell you, the question of “What is the best tool for X?” is always: _it depends_. On their own, neither full-text search nor vector search can meet all the criteria to build a fast, relevant search experience. Hybrid search merges the accessibility of full-text search with improved discovery that AI enables.

**Hybrid search** is the modern approach to search, unifying behind a single API the most advanced search capabilities. This hybrid approach blends full-text search with semantic search, enhancing both the accuracy and comprehensiveness of search results. It is a modern, flexible solution adapted to today’s search use cases.

* * *

[Meilisearch](https://meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) is an open-source search engine enabling developers to build state-of-the-art experiences while enjoying simple, intuitive DX. 

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector). You can learn more about the product by checking out its [roadmap](https://roadmap.meilisearch.com/?utm_campaign=vector-search&utm_source=blog&utm_medium=full-text-vs-vector) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions).

Full-text search vs vector search

What is a vector?
-----------------

Vectors are mathematical entities that possess both **magnitude** and **direction**. This ability to not only convey how much (magnitude) but also in which direction, makes them powerful tools for representing and manipulating different types of complex data. 

Vectors are often pictured as **arrows** pointing in a certain direction **within a space**, but they can also be translated into numbers. Representing vectors as **numerical values** makes it easier to work with them mathematically. 

Vectors are a broad mathematical concept used for various purposes across multiple fields.

Vector embeddings are a particular application of vectors in the domain of [machine learning](https://en.wikipedia.org/wiki/Machine_learning) and AI.

What is the purpose of vector embeddings?
-----------------------------------------

Machine learning models are designed to uncover patterns and relationships. They **map complex entities**, like text or images, **to points within a** [**vector space**](https://en.wikipedia.org/wiki/Vector_space). We call these vector embeddings. 

Vector embeddings, also called just embeddings, represent non-numeric data in a numerical format while **preserving the semantic meaning and relationships** of these non-numeric entities. Their goal is to allow computational models in machine learning and natural language processing (NLP) to **"understand" similarities and differences** between entities.

What is vector space embedding?
-------------------------------

In a vector space, similar entities are **positioned closely together**, indicating their semantic or contextual similarity. For instance, in the context of word embeddings, words with similar meanings are embedded near each other in the vector space. 

This spatial configuration is what enables embeddings to effectively capture and organize the **semantic relationships** between entities, a concept known as the semantic space.

What is the semantic space?
---------------------------

The semantic space is like a **virtual landscape** that each [large language model](https://en.wikipedia.org/wiki/Large_language_model) (LLM) builds as it trains. During this training phase, the model analyzes huge amounts of data to represent and understand language. The diversity and nature of the information it absorbs play a significant role in **shaping this semantic space**, which in turn influences how the LLM interprets and generates language.

Given the intricate nature of the data to be transformed, vector embeddings require **multidimensional spaces** to encompass the depth of these relationships and nuances. Depending on the **complexity of the features** they are meant to capture and the size of the dataset involved, these spaces can span a wide range, from dozens to thousands of dimensions.

This multi-dimensional space allows algorithms to interpret and process complex data in a way that mirrors human intuition and understanding.

### Illustrating the semantic space

Let’s illustrate the semantic space with a very simple example. Consider a diagram with three axes corresponding to the following semantic properties: `feline`, `juvenile`, and `canine`.

![](https://unable-actionable-car.media.strapiapp.com/semantic_space_56a01e7496.png)

*   On the `feline` axis, we have the `cat`
*   On the `juvenile` axis, the `baby`
*   On the `canine` axis, we have the `dog`

By combining these axes, we can find intersections that give us more specific entities:

*   `Feline` and `juvenile` combined give us the `kitten`
*   `Juvenile` and `canine` combined give us the `puppy`

![](https://unable-actionable-car.media.strapiapp.com/semantic_pets_9d93f8fecc.png)

By assigning numerical values to these properties, we can construct a simple semantic space:

| word   | canine | feline | juvenile |
|--------|--------|--------|----------|
| dog    | 1      | 0      | 0        |
| cat    | 0      | 1      | 0        |
| baby   | 0      | 0      | 1        |
| kitten | 0      | 1      | 1        |
| puppy  | 1      | 0      | 1        |

### Embedding vectors in the semantic space

In reality, **the semantic space is more complex**, and properties are not always clearly defined. We don’t know if this is actually the `canine` property, but it’s correlated to something `canine`, and the `dog` ranks very high on this property. The numbers are not `1` or `0` but some real numbers. This complexity allows for a nuanced understanding of how words and concepts relate to each other. The actual semantic space could look like the following:

| word   | ??canine?? | ??feline?? | ??juvenile?? |
|--------|------------|------------|--------------|
| dog    | 0.95973    | 0.12       | 0.22         |
| cat    | 0.05       | 0.99954    | 0.08         |
| baby   | 0.001      | 0.002      | 0.8973       |
| kitten | 0.02       | 0.9212     | 0.8647       |
| puppy  | 0.897      | 0.04       | 0.8834       |


From these detailed values, vector embeddings are created, capturing the essence of each word in a multidimensional vector, such as `[0.95973, 0.12, 0.22]` for `dog`. These vectors do more than just position words in a space; they build a **detailed network of meanings**, with each aspect designed to reveal a bit of what the word really means. The specific dimensions and what they represent can **vary from one model to another**, reflecting the complexity of the semantic meanings they encapsulate. 

Conclusion
----------

Vector embeddings are numerical representations of complex, non-numerical data. They are generated by machine learning models using vectors of real numbers. Embedded in a high-dimensional space, these vectors encapsulate the original data’s relationships and characteristics in a format suitable for computation. 

[**Vector databases**](__GHOST_URL__/what-is-a-vector-database/?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search), like Meilisearch, are the go-to for working with embeddings, as they facilitate **similarity searches**, also known as semantic searches, which allow identification of items that are semantically similar based on their vector representations.

AI Search is coming to Meilisearch Cloud, join the waitlist:

[Join now](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search)

* * *

[Meilisearch](https://meilisearch.com/?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience. 

A long-time actor in keyword search, Meilisearch enables users to address search use cases by building upon AI-powered solutions, not only supporting vector search as a vector store but also by providing hybrid search. This hybrid approach blends full-text search with semantic search, enhancing both the accuracy and comprehensiveness of search results.

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?utm_source=blog&utm_medium=embeddings-content&utm_campaign=vector-search).

What are vector embeddings?

Understanding vector databases
------------------------------

Vector databases are the go-to for performing searches based on similarity, which plays a key role in **AI-driven applications** like recommending your next favorite movie, identifying someone in a photo, or digging up texts that resonate with your search. At the core of these applications are **vector embeddings**, complex data forms that exceed the storage and retrieval capabilities of traditional databases. 

### The role of vector embeddings

[Vector embeddings](__GHOST_URL__/what-are-vector-embeddings/?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search) are a way to **transform complex non-numeric data**, such as words, sentences, or even images, **into a numerical format** while preserving their semantic **meaning** and relationships. 

Embeddings are **multidimensional objects** generated by machine learning models where each dimension represents a different feature or aspect of the data. To properly capture the data complexity, vectors can range from dozens to thousands of dimensions, depending on the size and nature of the data. 

### Vector databases vs traditional databases

This complexity makes **traditional databases**–designed to store structured data in tables– unfit to handle embeddings. The volume and complexity of these vectors, each potentially containing thousands of dimensions, challenge the row-and-column format. This mismatch necessitates alternative storage and retrieval solutions tailored to the requirements of vector data.

This is where **vector databases** like [Meilisearch](https://www.meilisearch.com/?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search) come into play. They are designed to address the unique demands of vector embeddings, facilitating efficient storage and retrieval of the information they contain. In particular, they enable performing **similarity searches**, also called semantic searches, which are central to leveraging embeddings effectively. 

<Callout>
  Learn more about [how Meilisearch built Arroy in Rust](__GHOST_URL__/spotify-inspired-hybrid-search-and-rust/?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search), an open-source vector store.
</Callout>

In other words, vector databases allow us to interact easily and efficiently with vector embeddings, making them essential for applications that require semantic understanding and similarity matching.

What is similarity search?
--------------------------

If we think of **vector embeddings** as stars in vast cosmic constellations, similarity search, or **vector search**, would be like trying to **find the nearest** stars to your current position in space. In practical terms, this means finding the most relevant documents, images, or products based on your search query.

To do so, you need to **measure the distance** between the **query** **vector** and other **vectors in the database**, typically using methods like [cosine similarity](https://en.wikipedia.org/wiki/Cosine_similarity) or [Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_distance). These are just different techniques for determining how close or distant other data points are from your query, much like gauging the **proximity** of stars in the night sky. 

### The role of machine learning models

However, the **success** of this search isn't just about mathematical calculations; it's highly dependent on the **machine learning model** used to generate and query the vectors. Each vector's meaning is intrinsically tied to the semantic space of the model that created it. Consistency here is crucial, ensuring that all vectors 'speak the same language' and adhere to the same contextual rules, making searches **meaningful and accurate**. That is to say, to achieve relevant search results, it's essential to **use the same model** for generating and querying the embeddings.

**Similarity search** is where vector databases like **Meilisearch** truly shine, as they allow for a wide array of applications such as face recognition, movie recommendations, and personalized content discovery. By allowing users to store vector embeddings alongside their documents, Meilisearch not only facilitates similarity searches but also introduces **hybrid search** capabilities, expanding its potential applications. Through the integration of models from various AI solution providers, Meilisearch enables users to **refine vector embeddings** to better suit their specific needs.

In summary, the ability of these databases to analyze and compare complex data patterns allows for highly relevant and accurate results across diverse fields, enhancing user experiences and operational efficiency.

AI Search is coming to Meilisearch Cloud, join the waitlist:

[Join now](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search)

* * *

[Meilisearch](https://meilisearch.com/?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience. 

A long-time actor in keyword search, Meilisearch enables users to address search use cases building upon AI-powered solutions, not only supporting vector search as a vector store but also by providing hybrid search. This hybrid approach blends full-text search with semantic search, enhancing both the accuracy and comprehensiveness of search results.

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/) or subscribe to the [newsletter](https://dub.sh/meili-newsletter?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search) and participating in [product discussions](https://github.com/orgs/meilisearch/discussions?utm_source=blog&utm_medium=vector-db-content&utm_campaign=vector-search).

What is a vector database?

Meilisearch vs Typesense

Search bars are an integral part of browsing the web. Whether we are shopping for clothes, enjoying music, or planning a vacation, there is always a dedicated search engine at work to simplify our lives. Have you ever wondered how these search engines actually work? In this article, we will delve into the inner workings of one of them: [Meilisearch](https://meilisearch.com).

Indexing time: from data to documents
-------------------------------------

Indexing is a fundamental process that encompasses the collection, parsing, and storage of data for subsequent retrieval. It plays a crucial role in enabling search engines to deliver fast and relevant results.

### A performant storage engine

Meilisearch stores data in the form of discrete records called "[documents](https://docs.meilisearch.com/learn/core_concepts/documents.html)," which are grouped together into collections known as "[indexes](https://docs.meilisearch.com/learn/core_concepts/indexes.html)." Under the hood, Meilisearch uses the LMDB ([Lightning Memory-Mapped Database](http://www.lmdb.tech/doc/)) key-value store which has proven stability and extensive usage in database applications. As the name implies, key-value stores are storage systems that organize data as a **collection of key-value pairs**.

![Illustration of a key-value store with 'engine' set to 'meilisearch' and 'language' set to 'rust'](https://unable-actionable-car.media.strapiapp.com/key_value_store_d2a8b6ff55.png)

Example of key-value entries

Key-value stores offer flexibility, fast access times, and efficient performance, contributing to quick and responsive interactions for users. By permitting only a single writing process at a time, LMDB avoids synchronization-related issues. As a result, it enables unlimited concurrent readers to have fast access to up-to-date, consistent data.

Before storing data in its key-value store, Meilisearch must preprocess it thoroughly. That’s where tokenization comes in.

### From words to tokens

**Tokenization** involves two main processes: segmentation and normalization.

**Segmentation** consists in splitting a sentence or phrase into smaller units called tokens.

Meilisearch uses (and maintains) the open-source tokenizer named [Charabia](https://github.com/meilisearch/charabia). The tokenizer is responsible for retrieving all the words (or tokens) from document fields.

<Callout>
  In practice, Meilisearch users can configure which fields should be [searchable](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#searchable-fields), allowing Charabia to know which fields should be tokenized.
</Callout>

Then comes **normalization**. Words are normalized based on each language’s particularities. For a romance language like French, this process includes setting the word in lowercase letters and removing diacritical marks such as accents. For instance, a sentence like “Le café de Nicolas” is transformed into “le cafe de nicolas”.

![The sentence 'Le café de Nicolas,' with a capital 'L' and 'N,' and an accent on the 'é' of 'café,' is first split into words during the segmentation phase. Then, during normalization, each word is lowercased, and accents are removed.](https://unable-actionable-car.media.strapiapp.com/Tokenisation_3_ea1333d3a3.png)

Example of the segmentation and normalization stages in the tokenization process

After segmenting and normalizing words, the engine needs to classify them. Using the appropriate data structure to organize tokens is crucial for performance. This will later allow swift retrieval of the most relevant search results.

### Storing tokens

Modern full-text search engines like Meilisearch come with features such as prefix search, typo tolerance, and geo search. Each data structure has its own advantages and disadvantages. When designing the search engine, our team carefully selected the ones best suited to enable these features without compromising on speed.

Throughout this section, we will explore which data structures powers Meilisearch.

#### Inverted index

First and foremost, we need to talk about inverted indexes. Without a doubt, it’s the data structure that stands out by its importance. Meilisearch uses it to return documents fast upon search.

An **inverted index** maps each word in a document to the set of documents where that term appears along with additional information like their position in the document.

![Given the following documents: `{   “id”: 1   "title": "Hello world!" }, {     “id”: 2     "title": "Hello Alice!" }.  The inverted index would look like this:  "alice" -> [2], "hello" -> [1,2],  "world" -> [1]` ](https://unable-actionable-car.media.strapiapp.com/Inverted_index_3_edcd06ad15.png)

Illustration of an inverted index generated from given documents, showcasing the mapping of keywords to document IDs

By storing words once and associating them with the documents where they appear, inverted indexes leverage the redundancy of terms across documents. Thanks to this, Meilisearch does not need to browse all documents in order to find a given word, resulting in faster searches.

Meilisearch creates approximately 20 inverted indexes per document index, making it one of the data structures with the highest number of instances. Indeed, to provide a search-as-you-type experience, the engine needs to precompute a lot at indexing time: word prefixes, documents containing filterable attributes, and more. For a better understanding of how inverted indexes work read more on [indexing best practices](__GHOST_URL__/best-practices-for-faster-indexing/).

<Callout>
  You can find the complete list of Meilisearch inverted indexes [on GitHub](https://github.com/meilisearch/meilisearch/blob/78e611f282d726a604cfbc231666265a37c4d46b/milli/src/index.rs#L103-L157).
</Callout>

#### Roaring bitmap

**Roaring bitmaps** are compressed data structures designed for efficiently representing and manipulating sets of integers.

Meilisearch uses roaring bitmaps widely throughout its inverted indexes to represent document IDs. Roaring bitmaps offer a space-efficient way to store large sets of integers and perform set operations like [union, intersection, and difference](https://en.wikipedia.org/wiki/De_Morgan%27s_laws). These operations play a crucial role in refining search results by allowing the inclusion or exclusion of specific documents based on their relationship with others.

#### Finite-state transducer

A finite-state transducer (FST) is a structured data representation ideal for performing string-matching operations. It represents a sequence of states ordered from smallest to largest, with words arranged in lexicographic order.

<Callout>
  [Lexicographic order](https://en.wikipedia.org/wiki/Lexicographic_order) is a method of arranging or sorting items, such as words or symbols, based on their alphabetical or numerical sequence.
</Callout>

A finite-state transducer is also called a _word dictionary_ because it contains all words present in an index. Meilisearch relies on the utilization of two FSTs: one for storing all words of the dataset, and one for storing the most recurrent prefixes.

FSTs are useful because they support compression and lazy decompression techniques, optimizing memory usage and storage. Additionally, by using automata such as regular expressions, they can retrieve subsets of words that match specific rules or patterns. Moreover, this enables the retrieval of all words that begin with a specific prefix, empowering fast, comprehensive search capabilities.

The FST, with its compact design, presents the added advantage of being a smaller data structure than the inverted index. As we will see later, this contributes to faster and more efficient reading operations. You can learn more about the topic in this [comprehensive article on finite-state reducers](https://blog.burntsushi.net/transducers/).

#### R-tree

R-trees are a type of [tree](https://en.wikipedia.org/wiki/Tree_$data_structure$) used for indexing multi-dimensional or spatial information. Meilisearch leverages R-trees to power its geo search functionality.

The **R-tree** associates geographical coordinates with the identifiers of the document to which it belongs. By organizing coordinates this way, it enables Meilisearch to perform spatial queries with remarkable efficiency. These queries allow users to find nearby points, points within a specific area, or points that intersect with other spatial objects.

Search time: query processing
-----------------------------

Modern search experiences only require you to start typing to get results. To achieve this search-as-you-type experience, Meilisearch precomputes a list of the most frequent prefixes.

To be tolerant to typos, Meilisearch uses finite-state transducers in conjunction with the [Levenshtein algorithm](https://www.meilisearch.com/docs/learn/configuration/typo_tolerance#understanding-typo-calculations). This algorithm calculates the Levenshtein distance which can be thought of as the cost of transforming a string into another. In other words, it quantifies the number of transformations required for a word to be converted into another word.

In the context of the Levenshtein algorithm, possible transformations are:

*   insertions, e.g. hat -> chat
*   deletions, e.g. tiger -> tier
*   substitutions, e.g. cat -> hat
*   transpositions or swaps, e.g. scared -> sacred

FSTs generate all possible variations of a word within a specified edit distance, enabling the engine to calculate the Levenshtein distance and accurately detect typos by comparing user input against a dictionary of valid words.

It becomes apparent that there are many factors to consider when processing a search request. Has the user finished typing? Are there typos in the query? Which documents should appear first in the search results?

Let’s discuss how Meilisearch processes search queries and how it refines and sorts search results.

### Query graph

Each time it receives a search query, Meilisearch creates a query graph representing the query and its possible variations.

To compute these variations, Meilisearch applies [concatenation](https://www.meilisearch.com/docs/learn/advanced/concat#concatenated-queries) and [splitting](https://www.meilisearch.com/docs/learn/advanced/concat#split-queries) algorithms to the query terms. The variations considered also include potential typos and synonyms–if the user set them. As an example, let’s examine the query: `the sun flower`. Meilisearch will also search for the following queries:

*   `the sunflower`: concatenation
*   `the sun flowe**d**`: substitution
*   `the sun flower**s**`: addition

Consider now a more complex query: `the sun flower is facing the su`, the query graph should look like this:

![Graph representing interpretations of the search query 'the sun flower is facing the su'. The engine precomputes for each query term word variations, like 'flowed' or 'flowers' and the associated Levenshtein cost ](https://unable-actionable-car.media.strapiapp.com/debuggong_image_e0a6ce757c.png)

Visual generated using our internal debugging tool, powered by [D2](https://github.com/terrastruct/d2), showcasing the execution process of search requests

As illustrated above, the graph represents different interpretations of the search query. The engine precomputes the word variations (and their cost) for each query term. Moreover, it detects whether the last query term is a prefix (not followed by a space), thus signaling the need to consult the prefix database.

So how does the engine utilize a query graph?

As we saw earlier, during the indexing process, Meilisearch identifies all documents that have filterable attributes, such as `genre`. Subsequently, it generates a list of document IDs associated with each attribute value, like `comedy` or `horror`. First of all, when a filter is applied, Meilisearch narrows down the potential results to the document IDs that fulfill the filter criteria.

Next, it uses the query words and the variations generated in the query graph and searches for matching words in the finite-state transducer (that is, our word dictionary). If the word is considered a prefix, it will also look it up in the prefix FST.

When encountering words in a FST, it searches them within the inverted index—which contains a mapping of words to document IDs— to retrieve the corresponding document IDs.

Finally, the engine performs an intersection to identify the documents that contain the words in the query graph and meet the filter criteria.

Let's take an example to better understand query processing: suppose you have a dataset of songs. The search query is `John Lennon`. The user wants to retrieve only songs released between 1957 and 1975.

First, Meilisearch retrieves the document IDs of songs within that time frame. Then, after checking in the FST that the words in the query graph exist, Meilisearch retrieves the IDs of the documents that contain either `John`, `Lennon`, or both. It also retrieves the possible variations, but we are not including them in this example for the sake of simplicity.

![Three Venn diagrams illustrating the process of retrieving songs released based on a user query (John Lennon) with a date filter (between 1057 and 1975).  Diagram 1: Represents songs released between 1957 and 1975. Diagram 2: Represents songs with "John" and/or "Lennon" in the title. Diagram 3: Represents the intersection between the two previous diagrams.](https://unable-actionable-car.media.strapiapp.com/diagrams_2_ac89ed0fa9.png)

Diagrams illustrating the operations involved in retrieving documents based on a user query.

Finally, the intersection of the two sets of document IDs is taken (the purple area in the diagram.) This means only document IDs that appear in both sets are kept. In other words, Meilisearch retains the document IDs of songs released between 1957 and 1975 that contain either `John`, `Lennon`, or both.

What happens when several documents match the search query, how does the engine decide which one to return first? Which one is more relevant? Let's explore the rules allowing Meilisearch to determine how to rank results.

### Relevancy

As we discussed earlier, the query graph includes possible variations of words. As a result, Meilisearch will also return documents with an artist, such as _John_ _Lebon_, as part of the search results.

Fortunately, the query graph not only accounts for word variations but also assigns them the Levenshtein cost, which, among other factors, helps Meilisearch determine the relevancy of the search results.

Meilisearch sorts documents in the search results using [bucket sort](https://en.wikipedia.org/wiki/Bucket_sort). This algorithm allows ranking documents based on a set of consecutive rules. By default, Meilisearch prioritizes rules as follows:

1.  Words: sorts documents based on the number of matched query terms, with documents containing all query terms ranked first
2.  Typo: sorts documents based on the number of typos, with documents matching query terms with fewer typos ranked first
3.  Proximity: sorts documents based on the distance between matched query terms. Documents where query terms occur close together and in the same order as the query string are ranked first
4.  Attribute: sorts documents based on the ranking order of attributes. Documents containing query terms in more important attributes are ranked first. Documents with query words at the beginning of attributes are considered more relevant than those with query words at the end
5.  Sort: sorts documents based on the user-defined parameters set at query time, if active
6.  Exactness: sorts documents based on the similarity of matched words with the query words

Meilisearch applies these rules sequentially, sorting results step by step. If two documents are tied after one rule, it uses the next rule to break the tie.

<Callout>
  Note that these rules are fully customizable, meaning you can add, delete, and reorder them as needed. Read more in the [relevancy documentation](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#ranking-rules).
</Callout>

By default, Meilisearch returns up to 1000 documents per search, prioritizing delivery of the most relevant results rather than all matching results. In other words, Meilisearch prioritizes efficiency and relevance over exhaustive results to ensure an optimized search experience.

Conclusion
----------

Search engines are complex systems that encompass multiple interconnected components, all working together to deliver a seamless search experience. While the inner workings may differ among search engines, we have explored the underlying mechanisms that power a modern search engine.

Meilisearch is currently exploring the vast realm of [**semantic search**](https://github.com/meilisearch/meilisearch/issues/3838). **AI-powered search** is reshaping the way we understand queries and documents, opening up new possibilities and use cases. If you’re eager to experience it firsthand, we invite you to try our [vector search prototype](https://github.com/meilisearch/product/discussions/621#discussioncomment-6183647)—your feedback would be greatly valuable!

Meilisearch is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience. Do you want to see Meilisearch in action? Try our [demo](https://where2watch.meilisearch.com/?ref=blog.meilisearch.com). You can also [run it locally](https://www.meilisearch.com/docs/learn/getting_started/quick_start?ref=blog.meilisearch.com) or create an account [on Meilisearch Cloud](https://cloud.meilisearch.com/login?ref=blog.meilisearch.com).

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/?ref=blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

How to deliver the best search results: inside a full text search engine

Full text search refers to matching some or all of a text query with documents stored in a database. Compared to traditional database queries, full text search provides results even in case of partial matches. It allows building more flexible search interfaces for users, thus enabling them to find accurate results more quickly.

From simple in-app search to browsing huge ecommerce catalogs, full text search use cases are numerous. It is so common that Postgres and other relational databases include dedicated APIs for full text search. Unfortunately, Postgres falls short of search-focused databases in multiple areas.

1\. Complex setup
-----------------

To provide relevant results, full text search should tolerate typos, allow synonyms, and allow partial matches. Additionally, results ranking needs to be highly customizable to adapt to a business’s specific needs. Configuring full text search on Postgres comes at the cost of [comprehensive configuration](https://rachbelaid.com/postgres-full-text-search-is-good-enough/) and often requires extensions that can’t be used when using managed Cloud services.

Creating database indexes, writing queries, and ranking algorithms quickly overstep domain knowledge and demand expertise in searching, indexing, and linguistics. Optimizing for performance gets even harder when working around constraints from mixed-and-matched extensions aimed at coping with Postgres full text search limitations.

Conversely, a search-focused database comes with [state-of-the-art features](https://www.meilisearch.com/docs/learn/what_is_meilisearch/overview#features) like typo tolerance, prefix search, fuzzy matching, synonyms, and customizable rankings out of the box.

2\. Faceted search
------------------

[Faceted search](https://www.meilisearch.com/docs/learn/fine_tuning_results/faceted_search) allows users to refine search results by broad categories. It’s often used in ecommerce applications. For example, a clothing shop can implement filtering by facets such as brand, size, or rating range.

[![Faceted search on an ecommerce website](https://unable-actionable-car.media.strapiapp.com/ecommerce_demo_4_final_1_48b9d742ac.gif)](https://ecommerce.meilisearch.com/)

Faceted search on an ecommerce website ([see demo](https://ecommerce.meilisearch.com/))

Implementing filtering is tricky enough for a single facet. But facets can take many shapes: category tags, price ranges, or minimum rating. Implementing filtering for all types is very challenging. By all means, the trickiest query to implement is aggregating results to build facet counts. This becomes very resource intensive on large datasets.

The complexity of implementing faceted search with Postgres grows exponentially with the number of facets. Faceted search alone makes a strong selling point for search engines like [Elasticsearch or Meilisearch](__GHOST_URL__/why-should-you-use-meilisearch-over-elasticsearch/). They come with optimized, first-class APIs to handle facets filtering and counting.

3\. Typo tolerance
------------------

By default, Postgres full text search cannot handle misspellings. Users commonly install the `pg_trgm` extension to work around this limitation. (Again, this solution is not always available in managed Postgres.) This extension notably introduces new operators to compare similarity between strings along with search-optimized GIN and GIST indexes.

The new indexes allow more configuration for full text search, but choosing between GIN and GIST indexes is not always trivial. Moreover, the new operators do not take into account word proximity, space separators, or word size. This, in particular, makes it hard to achieve real fuzzy matching with Postgres.

Ideally, a search-focused database should allow configuring different rules for single versus multiple words queries. It is the case of Meilisearch, which allows disabling typos entirely for specific fields. This enables users to search via unique identifiers like a book’s _International Standard Book Number_ (_ISBN_).

[![Example of typo tolerance](https://unable-actionable-car.media.strapiapp.com/typo_tolerance_2_gif_4409419422.gif)](https://typo-tolerance.meilisearch.com/)

Typo tolerance is disabled for the ISBN field ([see demo](https://typo-tolerance.meilisearch.com/))

4\. Language support
--------------------

Linguistic specificities vary a lot between languages using the Latin alphabet and others like Arabic or Chinese. As of Postgres 15, full text search dictionaries are unavailable for Simplified and Traditional Chinese, Korean, and Japanese, among others. This means resorting to specific implementations for different languages.

<Callout>
  To see the [list of languages supported by Postgres full text search](https://gist.github.com/Strift/3155b9238bac6495d4cd57b83d78415f), you can run the \`\\dFd\` command in Postgres.
</Callout>

Language support constraints are amplified in managed environments like Amazon RDS where users are unable to access the filesystem. This restricted access prevents them from implementing custom dictionaries, stemmers, synonyms, and more.

Meilisearch comes with [optimized language support](https://www.meilisearch.com/docs/learn/what_is_meilisearch/language) for Chinese, Japanese, Korean, Hebrew and more, on top of all languages using whitespace to separate words.

<Callout>
  Thanks to its vibrant open-source community, Meilisearch benefits from native speakers' contributions to [improve language-specific features](__GHOST_URL__/improving-meilisearchs-language-support/).
</Callout>

5\. Paying the backend price
----------------------------

Postgres is a database intended to communicate with server-side languages. When building a public-facing client application, it means building an API on top of the database to communicate with clients. On top of additional development time, creating such a proxy entails further problems.

First comes the issue of latency: making requests to an API that queries the database before returning results is bound to take some time. This does not affect dedicated search engines, as they come with a public API intended to deliver data to end users.

Now comes the second issue: security. Search engines APIs are designed, from the start, for public consumption. And security’s baked in for that use case. By default, API keys restrict search requests, while advanced features like [tenant tokens enable multi-tenancy](https://www.meilisearch.com/docs/learn/security/tenant_tokens#tenant-token-payload-reference).

[![Example of tenant tokens usage](https://unable-actionable-car.media.strapiapp.com/meili_create_token_c6fb7330a6.gif)](https://tenant-token.meilisearch.com/)

Using tenant tokens to restrict access to documents ([see demo](https://tenant-token.meilisearch.com/))

6\. Scaling limitations
-----------------------

There is a legitimate motive for wanting to keep all data in a single database. But having search-related data in the main database comes with great technical consequences. Full text search queries on large dataset become costly with Postgres, especially when ranking results and computing facet counts.

Monolithic databases often become the bottleneck of applications that need to scale. Don’t add unnecessary search-related costs to this resource when you can avoid it. These costs are only multiplied when building user-facing applications with high traffic.

Unlike relational databases, full text search engines like [Meilisearch use inverted indexes](__GHOST_URL__/best-practices-for-faster-indexing/). This data structure creates information redundancy to allow quicker information retrieval. It is designed to perform search operations, and will naturally outperform relational databases on large datasets. And, when search usage spikes, there is a single service that needs scaling.

7\. Relevancy
-------------

As we mentioned earlier, relevant search requires typo tolerance, custom rankings, and synonyms. In modern applications, users expect results to update on each keystroke, which requires [prefix searching](https://www.meilisearch.com/docs/learn/advanced/prefix). But Postgres full text search `ts_rank` function only allows attribute weighting. When using the `pg_trgm` extension, developers are left to implement their own sorting based on similarity.

In search-focused databases, the concepts of result ranking, attribute priority, numbers of word matched, and exactness of the query are first-class concepts. They match high-level APIs that allow explicit fine-tuning of search behavior. This makes it easier to make these concepts available to non-tech, business stakeholders. This was cited as a key reason for [why Bookshop chose Meilisearch](__GHOST_URL__/bookshop-increases-search-based-purchases/) for their ecommerce search.

8\. Missing out on InstantSearch libraries
------------------------------------------

When it comes to search experience, websites and applications often implement the same user interface patterns: a textual search bar, checkbox lists of facets, range sliders, sorting menu, page navigation, etc. The open-source InstantSearch library comes with implementations of all these features in the form of widgets available via SDKs in JavaScript, iOS, and Android.

When time to market is paramount, it’s hard to pass on such niceties. Supported by Algolia, the InstantSearch libraries enjoy wide adoption and several search engine databases come with [InstantSearch-compatible APIs](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch). Read our Nuxt guide for ecommerce search to learn [how to implement InstantSearch widgets with Vue](__GHOST_URL__/nuxt-ecommerce-search-guide/).

<Callout>
  Prefer React? Read our tutorial on [using InstantSearch with React](__GHOST_URL__/instant-meilisearch/).
</Callout>

9\. Limited Cloud support
-------------------------

Outsourcing the provisioning, maintenance, and scaling of servers is a common strategy in the Cloud era. Instead of managing servers, teams can focus on delivering value to their users. Postgres, like other databases, is available in a wide range of Cloud offerings for managed services. Unfortunately, managed services often come with limitations.

In the case of Postgres, implementing state-of-the-art full text search requires installing extensions. Moreover, fine-tuning language dictionaries and more configuration require access to the file system. Unfortunately, this means that many features are unavailable in Cloud environments.

To enable delegation of infrastructure, search engines often come with [dedicated cloud services](https://meilisearch.com/cloud). Those bespoke platforms do not compromise and allow using the full palette of search features. Moreover, customers can benefit from premium SLAs, support, and other enterprise services tailored to their searching use cases.

* * *

Postgres is a great, flexible database that allows the implementation of many custom, all-in-one solutions. Its full text search features might be sufficient for basic search, but it falls short when it comes to real-time search with relevance concerns. These limitations become worse on large datasets. It’s only natural, because Postgres is a database, not a search engine.

Meilisearch is an open-source search engine to build fast and relevant search experiences. It aims at providing state of the art experiences for end users while providing a simple and intuitive developer experience. You can try it by [running Meilisearch locally](https://www.meilisearch.com/docs/learn/getting_started/quick_start) or creating an account [for free on Meilisearch Cloud](https://cloud.meilisearch.com/login).

* * *

Learn more about what Meilisearch can bring to your business

[Talk to a search expert](https://dub.sh/meili-sales?utm_campaign=comparisons&utm_source=blog&utm_medium=vs-postgres)

* * *

For more things Meilisearch, you can join the community on [Discord](https://discord.meilisearch.com/) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e). You can learn more about the product by checking the [roadmap](https://roadmap.meilisearch.com/) and participating in [product discussions](https://github.com/meilisearch/product/discussions).

Meilisearch latest news and company updates