We've been busy working on things you've asked for, and the team is excited to share what's new in Meilisearch this quarter.

## A revamped search preview 🔍

The [search preview](https://www.meilisearch.com/docs/learn/getting_started/search_preview) has been one of our most-used features, but it didn't let you test everything Meilisearch can do. We've now rebuilt our search preview to make it more useful to assess test search result relevancy when you are tweaking an index's settings.

What's included:

- Show/hide the results' [ranking score](https://www.meilisearch.com/docs/learn/relevancy/ranking_score):

![ranking score.png](https://unable-actionable-car.media.strapiapp.com/image_38_9583f10fe8.png)

- Check and customize search options
- Configure filterable attributes as facets
- Test [AI-powered searches](https://www.meilisearch.com/docs/learn/ai_powered_search/getting_started_with_ai_search) if this functionality has been enabled for your project.

An updated preview allows you to test everything in real-time, making it easier to get your search working exactly how you want.

## New index settings in the UI ⚙️

We've fixed one of your biggest frustrations - you and your team no longer need to rely on JSON input to [change your index settings](https://www.meilisearch.com/docs/learn/configuration/configuring_index_settings). Instead of typing JSON code, you'll now see straightforward controls in the UI for adjusting settings like:

- Add and delete searchable attributes:
![searchable attributes/index settings.png](https://unable-actionable-car.media.strapiapp.com/image_37_5bea152778.png)

- Change the ranks of your searchable attributes
- Reset to default

Now anyone on your team can get access to user-friendly modification of the index settings without needing to understand JSON syntax.

## Speed up your search  with feature opt-out 🚀

We've got some updates for those of you working with massive datasets. To **reduce disk usage** and **speed up indexing by up to 15%**, you can now optimize Meilisearch's performance: 

- Choose to opt-out of prefix search and facet search features
- Reduce indexing time and save disk space

This has been one of our most requested features from users dealing with large-scale deployments, and we're excited to see how you'll use it.

## What else is cooking? 🎯
#### New Japan region is coming in Q4! 🇯🇵
Soon, you’ll be able to host your Meilisearch projects in Japan. This means much faster searches for users in Asia-Pacific - instead of connecting to Singapore, they'll connect directly to servers in Japan. The difference? A search that used to take 80+ milliseconds will happen almost instantly.

To use it, you’ll just need to pick Japan when you create a new project in the [cloud](https://cloud.meilisearch.com) dashboard. Keep an eye on our updates to know exactly when it’s released!


We are not stopping here. Our team is already working on our early 2025 releases. Here's a quick spoiler:

- AI-powered hybrid search general availability
- New Cloud dashboard features
- Even better performance optimizations

## Try it out!

Want to test these features? [Sign up for Meilisearch Cloud](https://cloud.meilisearch.com/) to experience these new features firsthand.

---

We built these changes based on your feedback, and we'd love to know if they help. Try them out and tell us what you think.

For  a more in-depth look at these updates, visit our [documentation](https://docs.meilisearch.com/).

Meilisearch Q4 2024 release wrap-up

We're excited to introduce Meilisearch v1.11. In this article, we'll cover the most significant changes. For a full list of updates, visit the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.11.0) on GitHub.

<Callout>
  Meilisearch 1.11 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=release-v1-11&utm_source=blog), too—upgrade now!
</Callout>

## Experimental: vector database indexing performance improvements

Meilisearch v1.11 introduces a [new embedder option](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=release-v1-11&utm_source=blog#binaryquantized): `binaryQuantized`.

Binary quantization compresses vectors by encoding each dimension with 1-bit values, significantly boosting performance, especially in datasets with over 1M documents and models with more than 1024 dimensions.

Smaller embeddings (under 1024 dimensions) tend to give worse results. With fewer dimensions, the binary vector doesn't retain enough information for good relevancy.


<Callout>
Enabling this feature permanently updates all vectors to contain only 1s and -1s, which can greatly affect search relevancy. This change is irreversible.
</Callout>

Before enabling this feature, we recommend testing it in a smaller or duplicate index in a development environment to evaluate its impact on your use case.

## Experimental: AI-powered search improvements

### Breaking changes

This release introduces several breaking changes for feature stabilization. 

####  `embedder` parameter becomes mandatory

In previous versions, the default embedder was automatically used for search and similar requests when no embedder was specified. Starting with v1.11, it’s now mandatory to specify the `embedder` parameter when performing:
- Semantic or hybrid search via [`GET`](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-11&utm_source=blog#search-in-an-index-with-get) and [`POST`](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-11&utm_source=blog#search-in-an-index-with-post) where the embedder must be included as part of the `hybrid` object.
- Similarity search via [`GET`](https://www.meilisearch.com/docs/reference/api/similar?utm_campaign=release-v1-11&utm_source=blog#get-similar-documents-with-get) and [`POST`](https://www.meilisearch.com/docs/reference/api/similar?utm_campaign=release-v1-11&utm_source=blog#get-similar-documents-with-post), where the `embedder` is required as a top-level parameter.


```javascript
// Semantic and hybrid search
client.index('INDEX_NAME').search(
  query, {
    hybrid: {
      embedder: 'EMBEDDER_NAME' // mandatory as of v1.11
   }
  }
)

// Similarity search
client.index('INDEX_NAME').searchSimilarDocuments({ 
  id: 'TARGET_DOCUMENT_ID',
  embedder: 'EMBEDDER_NAME' // mandatory as of v1.11
})
```

Additionally, even for full-vector searches (when using `vector` without `q`), you must include the `hybrid` object with the `embedder`.

```javascript
client.index('INDEX_NAME').search({
  vector: [0, 1, 2],
  hybrid: {
      embedder: 'EMBEDDER_NAME'// mandatory as of v1.11
   }
})
```

#### Non-zero `semanticRatio` now ignored when passing `vector` without `q`

Before Meilisearch v1.11, when performing a search with the [`vector`](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-11&utm_source=blog#vector-experimental) and [`hybrid`](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-11&utm_source=blog#hybrid-search-experimental) parameters but without [`q`](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-11&utm_source=blog#query-q), a [placeholder keyword search](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-11&utm_source=blog#placeholder-search) occurred unless `hybrid.semanticRatio` was set to `1.0`. In v1.11, a semantic search is always performed unless `hybrid.semanticRatio` is set to `0.0`.

### Additions and improvements

#### Default OpenAI model update

The default [OpenAI model](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=release-v1-11&utm_source=blog#model) has been updated from `text-embedding-ada-002` to the new `text-embedding-3-small`. This new model is both more cost-effective and more accurate than the legacy `ada` model, offering better performance at a lower cost.

#### New embedder parameter

Meilisearch 1.11's new `documentTemplateMaxBytes` [parameter](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=release-v1-11&utm_source=blog#documenttemplatemaxbytes) allows truncating the document's template text when it exceeds the specified byte limit. It is set, by default, to 400 bytes. 

This change is backward compatible. Existing embedders will retain the 400-byte default, with no need to recompute embeddings during the upgrade. The new limit will only apply when a document is modified and the template is re-rendered.

#### Improved document template

Fields in `documentTemplate` now include the `field.is_searchable` property. The default [document template](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=release-v1-11&utm_source=blog#documenttemplate) will filter out both empty fields and fields not in the searchable attributes list. The new default template only applies to embedders created starting with v1.11.


```liquid
// v1.11 default document template
{% for field in fields %}
  {% if field.is_searchable and not field.value == nil %}
    {{ field.name }}: {{ field.value }}
  {% endif %}
{% endfor %}

// v1.10 default document template
{% for field in fields %}
  {{ field.name }}: {{ field.value }}
{% endfor %}
```

## Federated search improvements: facet distribution and stats

Meilisearch v1.11 introduces two new options for federated searches: `facetsByIndex` and `mergeFacets`. These options allow you to request facet [distribution](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-11&utm_source=blog#facetdistribution) and [stats](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-11&utm_source=blog#facetstats) for federated search results.

### Facet Information by index

To retrieve facet distribution and stats for each individual index, use the `facetsByIndex` [option](https://www.meilisearch.com/docs/reference/api/multi_search?utm_campaign=release-v1-11&utm_source=blog#facetsbyindex). 


```json
{
  "federation": {
    "facetsByIndex": {
      "movies": ["title", "id"],
      "comics": ["title"]
    }
  },
  "queries": [
    { "q": "Batman", "indexUid": "movies" },
    { "q": "Batman", "indexUid": "comics" }
  ]
}
```

The response will include facet distribution and stats for each index:


```json
{
  "facetsByIndex": {
    "movies": {
      // Facet distribution for queries on the `movies` index
      "distribution": {
        "title": {
          "Batman returns": 1
        },
        "id": {
          "42": 1
        }
      },
      // Facet stats for queries on the `movies` index
      "stats": {
        "id": {
          "min": 42,
          "max": 42
        }
      }
    },
    "comics": {
      // Facet distribution for queries on the `comics` index
      "distribution": {
        "title": {
          "Batman: the killing joke": 1
        }
      },
      // Facet stats for queries on the `comics` index (empty because only numeric values are considered)
      "stats": {}
    }
  }
}
```

### Merged facet information

To get facet distribution and stats from all indexes in a single result, use both [`facetsByIndex`](https://www.meilisearch.com/docs/reference/api/multi_search?utm_campaign=release-v1-11&utm_source=blog#facetsbyindex) and [`mergeFacets`](https://www.meilisearch.com/docs/reference/api/multi_search?utm_campaign=release-v1-11&utm_source=blog#mergefacets):


```json
POST /multi-search
{
  "federation": {
    "facetsByIndex": {
      "movies": ["title", "id"],
      "comics": ["title"]
    },
    "mergeFacets": { "maxValuesPerFacet": 10 }
  },
  "queries": [
    { "q": "Batman", "indexUid": "movies" },
    { "q": "Batman", "indexUid": "comics" }
  ]
}
```

The response will include merged facet data:


```json
{
  "facetDistribution": {
    "title": {
      "Batman returns": 1,
      "Batman: the killing joke": 1
    },
    "id": {
      "42": 1
    }
  },
  "facetStats": {
    "id": {
      "min": 42,
      "max": 42
    }
  }
}
```

<Callout>
Learn more about [federated search and its use cases](https://www.meilisearch.com/blog/what-is-federated-search?utm_campaign=release-v1-11&utm_source=blog).
</Callout>

## Experimental: new `STARTS_WITH` filter operator

The new `STARTS_WITH` [filter operator](https://www.meilisearch.com/docs/learn/filtering_and_sorting/filter_expression_reference?utm_campaign=release-v1-11&utm_source=blog#starts-with) matches any field values that start with the specified string pattern:


```json
// returns all heroes whose name starts with 'super'
"filter": "hero STARTS_WITH super"
```

This is an experimental feature. To enable it, use the [experimental features](https://www.meilisearch.com/docs/reference/api/experimental_features?utm_campaign=release-v1-11&utm_source=blog) endpoint and set the `containsFilter` option to `true`. Note that this will also activate the `CONTAINS` filter operator.

<Callout>
Share your feedback via this [GitHub discussion](https://github.com/orgs/meilisearch/discussions/763)
</Callout>

## Contributors shout-out

We want to give a massive thank you to the contributors who made this release possible: [@iornstein](https://github.com/iornstein) for their help with [Meilisearch](https://github.com/meilisearch/meilisearch), and [@luflow](https://github.com/luflow), [@tkhshtsh0917](https://github.com/tkhshtsh0917) for their work on [charabia](https://github.com/meilisearch/charabia).

We also want to extend special thanks to our SDK maintainers working hard to make Meilisearch available across many [languages and frameworks](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks?utm_campaign=release-v1-11&utm_source=blog).

* * *

And that’s a wrap for v1.11! These release notes only highlights the most significant updates. For an exhaustive listing, read the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.11.0) on Github.

Upgrade with ease using Meilisearch Cloud.

[Open Meilisearch Cloud](https://cloud.meilisearch.com/login?utm_campaign=release-v1-11&utm_source=blog)

* * *

Stay in the loop of everything Meilisearch by subscribing to our [monthly newsletter](https://dub.sh/meili-newsletter?utm_campaign=release-v1-11&utm_source=blog). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-11&utm_source=blog) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-11&utm_source=blog).

Meilisearch 1.11

Today we're announcing Meilisearch v1.10. In this article, we’ll review the most impactful changes. For an exhaustive listing, check out the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.10.0) on GitHub.

<Callout>
  Meilisearch 1.10 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=release-v1-10&utm_source=blog), too—upgrade now!
</Callout>

New: federated search
---------------------

We’re excited to announce that [federated search](/blog/what-is-federated-search/?utm_campaign=release-v1-10&utm_source=blog is finally coming to Meilisearch. This feature extends the [multi-search API](https://www.meilisearch.com/docs/reference/api/multi_search?utm_campaign=release-v1-10&utm_source=blog#perform-a-multi-search) to allow you to merge results from multiple queries into a single, unified response. You can now search different indexes and get the most relevant results across the entire dataset, not only the most relevant for each index.

For SaaS platforms that need to search across various types of resources or ecommerce sites with diverse product categories, federated search is a game-changer.

Here is an example payload for making a [federated search request](https://www.meilisearch.com/docs/reference/api/multi_search?utm_campaign=release-v1-10&utm_source=blog#federated-multi-search) against the `movies` and `comics` indexes:

```json
{
  "federation": {
    "offset": 5,
    "limit": 10
  },
  "queries": [
    {
      "q": "Batman",
      "indexUid": "movies"
    },
    {
      "q": "Batman",
      "indexUid": "comics"
    }
  ]
}
```

In this example, the [federated search response](https://www.meilisearch.com/docs/reference/api/multi_search?utm_campaign=release-v1-10&utm_source=blog#response-200-ok-1) will look like this:

```json
{
  "hits": [
    {
      "id": 42,
      "title": "Batman returns",
      "overview": "..",
      "_federation": {
        "indexUid": "movies",
        "queriesPosition": 0
      }
    },
    {
      "comicsId": "batman-killing-joke",
      "description": "..",
      "title": "Batman: the killing joke",
      "_federation": {
        "indexUid": "comics",
        "queriesPosition": 1
      }
    },
 ],
  "processingTimeMs": 0,
  "limit": 2,
  "offset": 0,
  "estimatedTotalHits": 2,
  "semanticHitCount": 0
}
```

### Query weighting

Federated search isn't just about combining results; it's about making those results more meaningful and relevant. Thanks to [query weighting](https://www.meilisearch.com/docs/reference/api/multi_search?utm_campaign=release-v1-10&utm_source=blog#federationoptions), you can prioritize certain queries over others, ensuring that the most important data rises to the top.

The example payload below uses **query weighting** in a federated search to give more importance to documents where the `boosted` field is `true`:

```json
{
  "federation": {},
  "queries": [
    {
      "q": "iphone case",
      "indexUid": "products",
      "filter": "BOOSTED = true",
      "federationOptions": {
        "weight": 3.0
      }
    },
    {
      "q": "iphone case",
      "indexUid": "products"
      // federationOptions.weight defaults to 1
    }
  ]
}
```
<Callout>
  Learn more about [federated search and its use cases](/blog/what-is-federated-search/?utm_campaign=release-v1-10&utm_source=blog)
</Callout>

New: language settings for queries and documents
------------------------------------------------

By default, Meilisearch automatically detects the language used in documents and queries. This comes with some limitations for edge cases. Meilisearch 1.10 introduces new language settings that allow users to manually customize their language preferences.

There are two ways to state the language used in a search:

*   Set the language used in a query
*   Set the language used in a document field

### Setting the search query locale

You can define which language is used in a query in the [search parameters](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-10&utm_source=blog#query-locales).

**Example payload for searching with query `locales`:**

```json
{
   "q": "進撃の巨人",
   "locales": ["jpn"]
}
```

### Setting the document attributes’ locales

You can define which language is used in your dataset via the [index settings](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=release-v1-10&utm_source=blog#localized-attributes).

**Example payload for settings with `localizedAttributes`:**

```json
{
  "localizedAttributes": [
    {
      "locales": ["jpn"], 
      "attributePatterns": ["*_japanese"]
    }
  ]
}
```
<Callout>
  Meilisearch support all [ISO-639-2B](https://en.wikipedia.org/wiki/List_of_ISO_639-2_codes) locales.
</Callout>

Self-hosting: Ubuntu v20 is now required
----------------------------------------

If you’re self-hosting Meilisearch, take note: Meilisearch 1.10 now requires **Ubuntu version 20**. This update ensures compatibility with the latest features and maintains optimal performance and security. We advise reading the documentation on [how to upgrade your Ubuntu release](https://documentation.ubuntu.com/server/how-to/software/upgrade-your-release/).

<Callout>
  This change does not affect [Meilisearch Cloud](https://meilisearch.com/cloud?utm_campaign=release-v1-10&utm_source=blog) users.
</Callout>

Experimental: AI-powered search DX improvements
-----------------------------------------------

### New: custom headers to REST API-based embedders

Meilisearch 1.10 supports sending custom headers to your API-based embedding service. When using the `rest` source for your [embedder settings](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=release-v1-10&utm_source=blog#embedders-experimental), Meilisearch now accepts an optional `headers` object. These headers will be added to all requests made to the embedder API.

**Example payload for embedder settings:**

```json
{
  "source": "rest",
  "apiKey": "•••••••••••••••••",
  "headers": {
    "Custom-Header": "value"
  }
}
```

### New: quality of life improvements

Additionally, we added some minor changes to make working with Meilisearch AI-powered search API easier:

*   Embedder settings now accept a `url` parameter for OpenAI embedder to access OpenAI through a proxy
*   Embedder settings now accept a `dimensions` parameter for ollama embedder
*   Meilisearch now includes `_vectors` in the response for documents without vectors (only when `retrieveVectors` is enabled)

### ⚠️ Breaking: REST API-based embedders breaking changes

The API for [configuring an embedder based on a REST API](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-10&utm_source=blog#generate-vector-embeddings) (e.g., OpenAI, Mistral, etc.) has quickly evolved and to avoid confusion, we have decided to reorganize the parameters in dedicated `request` and `response` fields.

In practice, this change:

*   removes `query` , `inputField`, `inputType`, `pathToEmbeddings` and `embeddingObject` fields
*   adds `request` and `response` fields

Take a look at the example below to learn how to migrate to the new syntax.

**Before** — With Meilisearch 1.9

```json
{
   "source": "rest",
   "url": "https://localhost:10006",
   "query": {
     "model": "minillm",
   },
   "inputField": ["prompt"],
   "inputType": "text",
   "embeddingObject": ["embedding"]
}
```

**After** — Starting with Meilisearch 1.10

```json
{
   "source": "rest",
   "url": "https://localhost:10006",
   "request": {
     "model": "minillm",
     "prompt": "{{text}}"
   },
   "response": {
     "embedding": "{{embedding}}"
   }
}
```

#### Upgrade procedure

This procedure is only necessary if you're using REST API-based embedder. To upgrade to Meilisearch 1.10, follow these steps:

1.  Remove embedders with source `"rest"`
2.  Update your [Meilisearch Cloud project](https://www.meilisearch.com/docs/learn/update_and_migration/updating?utm_campaign=release-v1-10&utm_source=blog#updating-meilisearch-cloud) or [self-hosted Meilisearch instance](https://www.meilisearch.com/docs/learn/update_and_migration/updating?utm_campaign=release-v1-10&utm_source=blog#updating-a-self-hosted-meilisearch-instance) as usual

Experimental: `CONTAINS` filter operator
----------------------------------------

Meilisearch 1.10 introduces the `CONTAINS` operator. This is still in the experimental stage due to some performance issues we’re working to resolve. This new [filter operator](https://www.meilisearch.com/docs/learn/filtering_and_sorting/filter_expression_reference?utm_campaign=release-v1-10&utm_source=blog#contains) allows to check if a substring is included in a larger string.

Given the document `{ id: 1, name: "Keffir" }`, you can use the filter as follows:

*   `name CONTAINS kef` — search will match the document
*   `name CONTAINS clifford` — search will not match the document

<Callout>
  Remember all filters used lowercase, normalized strings (all accents are removed).
</Callout><Callout>
  Share your feedback via this [Github discussion](https://github.com/orgs/meilisearch/discussions/763).
</Callout>

Experimental: update documents with a function
----------------------------------------------

Meilisearch 1.10 allows you to edit documents by executing [a Rhai function](https://rhai.rs/). This allows you to update only a part of your dataset using filters. Additionally, using a function allows you to implement dynamic logic based on your documents’ data.

The following payload updates the title of all documents with `id` above 3000:

```json
{
  "filter": "id > 3000",
  "function": "doc.title = `✨ ${doc.title.to_upper()} ✨`"
}
```

This also enables more complex use cases, like [implementing a decaying ranking strategy](https://felx.me/2021/08/29/improving-the-hacker-news-ranking-algorithm.html). Depending on your needs, you might implement a function such as the following that will be run regularly by a Cron job.

```json
{
  "context": { "now": 1715423249 },
  "function": "
    // `posted_at` and `now` are Unix Epoch timestamps in seconds
    // they must be converted to hours.
    let age_hours = (context.now - doc.posted_at) / 60 / 60;
    doc.ranking_score = doc.upvotes ** 0.8 / (age_hours + 2) ** 1.8;
  "
}
```
<Callout>
  Share your feedback via this [Github discussion](https://github.com/orgs/meilisearch/discussions/762).
</Callout>

* * *

And that’s a wrap for v1.10! These release notes only highlights the most significant updates. For an exhaustive listing, read the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.10.0) on Github.

Upgrade with ease using Meilisearch Cloud.

[Open Meilisearch Cloud](https://cloud.meilisearch.com/login?utm_campaign=release-v1-10&utm_source=blog)

* * *

Contributors shout-out
----------------------

This release wouldn’t be possible without [Meilisearch](https://github.com/meilisearch/meilisearch) and [Charabia](https://github.com/meilisearch/charabia) contributors [@Karribalu](https://github.com/Karribalu), [@hanbings](https://github.com/hanbings), [@junhochoi](https://github.com/hunhochoi), [@JWSong](https://github.com/JWSong), [@PeterDaveHello](https://github.com/PeterDaveHello), [@LukasKalbertodt](https://github.com/LukasKalbertodt), and [@phillitrOSU](https://github.com/phillitrOSU).

We also want to extend special thanks to our SDK maintainers working hard to make Meilisearch available across many [languages and frameworks](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks). We’re excited for [@Sherlouk](https://github.com/Sherlouk) and [@Ja7ad](https://github.com/Ja7ad) to join our open-source maintainers. 💪

* * *

Stay in the loop of everything Meilisearch by subscribing to our [monthly newsletter](https://dub.sh/meili-newsletter?utm_campaign=release-v1-10&utm_source=blog). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-10&utm_source=blog) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-10&utm_source=blog).

Meilisearch 1.10

We're excited to unveil Meilisearch v1.9. In this article, we’ll review the most impactful changes. For an exhaustive listing, check out the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.9.0) on GitHub.

<Callout>
  Meilisearch 1.9 is available on [Meilisearch](https://cloud.meilisearch.com/?utm_campaign=release-v1-9&utm_source=blog) Cloud, too—upgrade now!
</Callout>

New: ranking score threshold
----------------------------

Meilisearch 1.9 allows excluding search results with low ranking scores. When using the new `rankingScoreThreshold` option, Meilisearch will not return any documents below the defined threshold.

```bash
curl \
 -X POST 'http://localhost:7700/indexes/movies/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
    "q": "green ogre living in a swamp",
    "hybrid": { 
	    "semanticRatio": 0.9, 
	    "embedder": "default" 
	},
    "showRankingScore": true,
    "limit": 5,
    "rankingScoreThreshold": 0.2
 }'
```

Using the ranking score threshold when implementing [hybrid search](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-9&utm_source=blog#hybrid-search-experimental) removes irrelevant results and allows your [search analytics](https://www.meilisearch.com/docs/learn/analytics/analytics?utm_campaign=release-v1-9&utm_source=blog) to collect the _No search results_ metric properly.

<Callout>
  Excluded results do not count towards `estimatedTotalHits`, `totalHits`, or facet distribution.
</Callout>

New: distinct attribute at search time (group by)
-------------------------------------------------

Meilisearch 1.9 adds the ability to define a [distinct attribute](https://www.meilisearch.com/docs/learn/configuration/distinct?utm_campaign=release-v1-9&utm_source=blog) at search time. Meilisearch will only return one document with the specified attribute value when using the new `distinct` search parameter.

This feature is commonly used in ecommerce applications. Consider a `products` index containing multiple variants of the same product, e.g. `Blue iPhone 15` and `Red iPhone 15` documents that share the same `product_id`. The API call below will return _a single_ iPhone 15:

```bash
curl \
 -X POST 'http://localhost:7700/indexes/products/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
    "q": "iphone",
    "distinct": "product_id"
 }'
```
<Callout>
  When `distinct` is provided, Meilisearch ignores the index’s [distinct attribute](https://www.meilisearch.com/docs/learn/configuration/distinct?utm_campaign=release-v1-9&utm_source=blog).
</Callout>

New: `frequency` matching strategy
----------------------------------

Meilisearch 1.9 introduces a new [matching strategy](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-9&utm_source=blog#matching-strategy) to prioritize results that contain occurrences of the least frequent query terms. When using the `frequency` matching strategy, Meilisearch will deprioritize very common words.

Let’s take the example of the `"the little prince"` query. In our indexed documents, the words `"the"` and `"little"` likely have a lot of occurrences. Consequently, the matching strategy will prioritize documents containing `"prince"` instead.

Experimental: new similar documents API
---------------------------------------

Meilisearch 1.9 introduces a new [AI-powered search](https://www.meilisearch.com/docs/learn/ai-powered-search/getting_started_with_ai_search?utm_campaign=release-v1-9&utm_source=blog) feature that allows searching for documents similar to an existing document.

The following API call searches for documents similar to the document whose [primary key](https://www.meilisearch.com/docs/learn/core_concepts/primary_key?utm_campaign=release-v1-9&utm_source=blog) is `23` in the movies index:

```bash
curl \
  -X POST /indexes/movies/similar
  -H 'Content-Type: application/json' \
  --data-binary '{
    "id": "23",
    "embedder": "default",
  }'
```

Check out the [similar documents API](https://www.meilisearch.com/docs/reference/api/similar?utm_campaign=release-v1-9&utm_source=blog) for more information about additional parameters.

Experimental: avoid re-generating embeddings
--------------------------------------------

When importing a [dump](https://www.meilisearch.com/docs/learn/advanced/dumps?utm_campaign=release-v1-9&utm_source=blog) created with Meilisearch 1.9 or higher, Meilisearch will not re-generate the embeddings. This will avoid unnecessary computing when upgrading your Meilisearch database.

### New: `regenerate` parameter

Additionally, Meilisearch 1.9 introduces a new API to give more fine-grained control over [document embedding generation](https://www.meilisearch.com/docs/learn/ai-powered-search/getting_started_with_ai_search?utm_campaign=release-v1-9&utm_source=blog#generate-vector-embeddings-with-openai). Specifically, it enables embedding generation whenever a document is updated.

The document `_vectors` object now accepts objects in addition to arrays. The provided object accepts a `regenerate` boolean and an optional `embeddings` array.

Consider the example document below with user-provided embeddings:

```json
{
	"id": 42,
	"_vectors": {
		// Embeddings for the `default` embedder
		// Equivalent to `regenerate: true`
		"default": [0.1, 0.2 ],
		// Embeddings for the `text` embedder
		"text": {
			"embeddings": [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]],
			// Never regenerate embeddings
			"regenerate": false
		},
		"translation": {
			"embeddings": [0.1, 0.2, 0.3, 0.4],
			// Regenerate embeddings when document is updated
			"regenerate": true
		}
	}
}

```

While, in general, you might want to re-generate your embeddings whenever the document is updated, this facilitates migrating from user-provided embeddings to letting Meilisearch handle embedding without incurring unnecessary costs.

Experimental: hybrid search breaking changes
--------------------------------------------

As we move toward [stabilizing AI-powered search features](https://github.com/meilisearch/meilisearch/issues/4733), we introduced minor breaking changes to make the APIs less error-prone.

### Breaking: empty `embeddings` array

Following user feedback that the previous behavior was unexpected and unhelpful, providing an empty `embeddings` array will now tell Meilisearch that the document has no embeddings.

<Callout>
  Before Meilisearch 1.9, an empty `embeddings` array was interpreted as a single embedding of dimension 0.
</Callout>

### Breaking: removed `_vectors` in search results

Starting with Meilisearch 1.9, API responses to [vector search and hybrid search](https://www.meilisearch.com/docs/learn/ai-powered-search/getting_started_with_ai_search?utm_campaign=release-v1-9&utm_source=blog) requests will not include `_vectors` in the response.

However, you can now use the new `retrieveVectors` search parameter if you want API responses to include them:

```bash
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "star wars",
    "retrieveVectors": true
  }'
```

### Breaking: optimizing user-provided `embeddings`

Starting with Meilisearch 1.9, vector embeddings will no longer be stored as-is. Numbers will be cast to floats in canonicalized representation to save storage and optimize performance. Simply put, the vector `[3]` might be stored as `[3.0]`.

Contributors shout-out
----------------------

Thanks to all community members who participated in this release. Shout out to [@gh2k](https://github.com/gh2k), [@writegr](https://github.com/writegr), and [@yudrywet](https://github.com/yudrywet) for their contributions to [Meilisearch](https://github.com/meilisearch/meilisearch) and [@mosuka](https://github.com/mosuka), [@Soham1803](https://github.com/Soham1803), and [@tkhshtsh0917](https://github.com/tkhshtsh0917) for their contributions to [Charabia](https://github.com/meilisearch/charabia).

And, of course, many thanks to our SDK maintainers, thanks to whom Meilisearch is available across many languages. Special thanks to [@the-sinner](https://github.com/the-sinner) and [@norkunas](https://github.com/norkunas). 🫶

* * *

And that’s a wrap for v1.9! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.9.0).

Stay in the loop of everything Meilisearch by subscribing to our [monthly newsletter](https://dub.sh/meili-newsletter?utm_campaign=release-v1-9&utm_source=blog). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-0&utm_source=blog) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-9&utm_source=blog).

Meilisearch 1.9

On June 11, 2024, Meilisearch hosted a live event packed with product updates, sneak peeks, and roadmap discussions. If you couldn't make it, here's a quick recap of what went down.

**AI-powered search refresher and new embedders**
-------------------------------------------------

Our product team took the stage to share insights into developing our AI-powered search solution and outlined our vision for providing ready-to-use components for seamless AI-powered search integration. By adding semantic and AI capabilities to Meilisearch's core keyword and full-text product, we introduced a [hybrid search solution](https://www.meilisearch.com/solutions/hybrid-search) in early 2024.

To streamline the developer experience, we have since added [Ollama and REST embedders](/blog/meilisearch-1-8/ for easier integration within the v1.8 scope.

![](https://lh7-us.googleusercontent.com/docsz/AD_4nXd_QlnM1707ZYLjpOrhzxjZrWDxgdVo3TL5Dewz6dM1ZIbhZp4CcPUJdtDhV4xAA_ZZACDSv8h9JF_d2AoFSNGYNlShW_JOI_s1TrqbEtntN-7pvxW8UnTTdVMNml9oBroLhZfrVprM5dQmM0HiTP55MZPw?key=pr3OBspkNUkqXQYoKJ84OA)

You are welcome to try out the hybrid search feature using our [movie demo database](https://where2watch.meilisearch.com/). 

**New relevancy improvements and smarter search**
-------------------------------------------------

We also introduced several new features designed to _improve search relevancy_ in v1.9:

*   **Negative keyword search**: exclude unwanted results easily.
*   **New frequency matching strategy**: enhance search accuracy by removing the most common term found in the dataset.
*   **Filter by score**: set a baseline _rankingScore_ to return only results considered "good enough" in terms of relevancy.
*   **Get similar documents:** utilize AI to find similar documents within your dataset.

This new feature set is slated for launch on July 1st.

**Query suggestions sneak peek**
--------------------------------

Attendees also got a sneak peek into an upcoming release. The new _query suggestions_ feature aims to enhance search efficiency by returning real-time, relevant suggestions as users type, ordered by popularity. This feature leverages [cloud search analytics](/blog/search-analytics-release/ to generate and rank suggestions based on popularity and relevance. Designed for seamless integration, it requires no new endpoints or tools and is expected to be released in Q3. Ongoing improvements will be based on user feedback.

**Check out the recording of the event for more details:**
----------------------------------------------------------

* * *

Get these monthly updates straight to your inbox by subscribing to our [newsletter](https://dub.sh/meili-newsletter).

For more things Meilisearch, join our developers' community on [Discord](https://discord.meilisearch.com/?ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/?ref=blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

Until next time!

Here's what you missed in the latest Meilisearch live event: V1.8, V1.9, new embedders, query suggestions, and more.

<Callout>
  Meilisearch 1.8 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=release-v1-8&utm_source=blog) too—upgrade now!
</Callout>

New: negative search
--------------------

Meilisearch 1.8 allows you to exclude specific terms from your search results using the `-` operator. 

For example, to find documents that contain the word "room" but not "escape", place a `-` before "escape”:

```Bash
curl \
  -X POST 'https://edge.meilisearch.com/indexes/places/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "-escape room" }'

```

Experimental: Improved hybrid search
------------------------------------

<Callout>
  AI search is an experimental feature. Give feedback in this [GitHub discussion](https://github.com/orgs/meilisearch/discussions/677).
</Callout>

### New Ollama embedder

Meilisearch 1.8 introduces support for Ollama models. [Configure the embedder](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-8&utm_source=blog#generate-vector-embeddings) in your index settings:

```JSON
"default": {
  "source": "ollama", // 👈 Use the ollama source
  "url": "http://localhost:11434/api/embeddings",
  // ☝️ optional, fetched from MEILI_OLLAMA_URL environment variable if missing
  "apiKey": "<your-api-key>",
  // ☝️ optional
  "model": "nomic-embed-text",
  "documentTemplate": "{{doc.field_name}}" // like for openAI and huggingFace sources
}

```

### New REST Embedder

Starting with Meilisearch 1.8, you can use any embedder accessible via a REST API. Here’s how to configure an OpenAI embedder with the `rest` source:

```JSON
{
  "source": "rest",
  "apiKey": "<your-openai-api-key>",
  "dimensions": 1536,
  "url": "https://api.openai.com/v1/embeddings",
  "query": {
    "model": "text-embedding-ada-002"
  },
  "inputField": ["input"],
  "inputType": "textArray",
  "pathToEmbeddings": ["data"],
  "embeddingObject": ["embedding"]
}

```

For more details on the API and available parameters, please consult the documentation on [REST embedders](https://www.meilisearch.com/docs/learn/experimental/vector_search#generate-auto-embeddings?utm_campaign=release-v1-8&utm_source=blog).

### New embedder setting: distribution shift

<Callout>
  This is an advanced API for custom models. OpenAI and Hugging Face models don't require this setting.
</Callout>

Depending on the embedding model, [vector embeddings](/blog/what-are-vector-embeddings/?utm_campaign=release-v1-8&utm_source=blog) are not always uniformly distributed within the [vector space](/blog/what-are-vector-embeddings/?utm_campaign=release-v1-8&utm_source=blog) When vectors are overly clustered, semantic search scores can be very closely ranked. As a result of semantic scores being packed, hybrid results might have sub-optimal relevancy.

To address this issue, you can now shift the distribution of results with an affine transformation. This transformation normalizes semantic scores to prevent them from clustering too high or too tightly, making them more comparable to the [keyword scores](https://www.meilisearch.com/docs/reference/api/search#ranking-score).

```JSON
"default": {
  "source": "huggingFace",
  "model": "MODEL_NAME",
  "distribution": {  // describes the natural distribution of results
    "mean": 0.7, // mean value
    "sigma": 0.3 // standard deviation
  }
}

```

*   The **mean** value represents the central point around which you observe semantic search scores to be densely packed. In the provided code sample, the setting indicates that semantic scores from this model tend to cluster around a mean of 0.7.
*   The **sigma** value determines the spread of ranking scores from the mean (standard deviation). The sigma provided in the code sample above means that before shifting the distribution, the results tend to have an average distance of 0.3 to the mean.

This allows for a more effective comparison of `_rankingScore` between semantic and keyword search results, leading to improved results ranking.

### Breaking: trimmed API response

To speed up the search response and reduce bandwidth usage, Meilisearch v1.8 no longer returns the following fields in the search responses:

*   the `vector` field, which returned the query vector 
*   the `value` field included in the [`vectorSort`](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-8&utm_source=blog#vectorsorttarget_vector) object, which is returned when you add \`”showRankingScoreDetails”: true”\` to a search query
*   the `_semanticScore` field; its value is the same as `_rankingScore`, which you should use instead. Set [`”showRankingScore”: true”`](https://www.meilisearch.com/docs/reference/api/search?utm_source=blog&utm_campaign=release-v1-8#ranking-score) in the search parameters

Search robustness improvements
------------------------------

### New search cut-off setting

To enhance stability and performance, Meilisearch automatically terminates any search request that exceeds 1500 milliseconds. 

To adjust this timeout threshold, you can customize the `searchCutoffMs` setting:

```bash
curl \
  -X PATCH 'https://edge.meilisearch.com/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "searchCutoffMs": 150
  }'

```

### Experimental: concurrent search requests limit

This release introduces a limit on concurrent search requests to prevent Meilisearch from using excessive amounts of RAM and crashing. The default queue limit is set to 1000 requests. 

To modify this limit, restart your self-hosted instance with the `--experimental-search-queue-size` option:

```bash
./meilisearch --experimental-search-queue-size 100
```

👉 This limit does not impact the search performance. It only controls the number of search requests that can be queued simultaneously to enhance system security.

<Callout>
  Give feedback regarding concurrent search requests limit in this [GitHub discussion](https://github.com/orgs/meilisearch/discussions/729).
</Callout>

Contributors shout-out
----------------------

We are really grateful for all the community members who participated in this release. We would like to thank: [@availhang](https://github.com/availhang), [@hack3ric](https://github.com/hack3ric), [@jakobklemm](https://github.com/jakobklemm), [@mohsen-alizadeh](https://github.com/mohsen-alizadeh), [@redistay](https://github.com/redistay), [@rohankmr414](https://github.com/rohankmr414), [@shuangcui](https://github.com/shuangcui), [@Toromyx](https://github.com/Toromyx), [@6543](https://github.com/6543) for their help with Meilisearch, and [@Gusted](https://github.com/Gusted), [@mosuka](https://github.com/mosuka), [@6543](https://github.com/6543), for their work on [Charabia](https://github.com/meilisearch/charabia).

Special thanks to our SDKs maintainers, thanks to whom Meilisearch is available across so many languages. 🫶

* * *

And that’s a wrap for v1.8! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.8.0).

Stay in the loop of everything Meilisearch by subscribing to our [newsletter](https://dub.sh/meili-newsletter). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-8&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-8&utm_source=blog).

Meilisearch 1.8

We're excited to unveil Meilisearch v1.7. In this article, we’ll review the most impactful changes. For an exhaustive listing, check out the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0) on Github.

<Callout>
  Meilisearch 1.7 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog) too—upgrade now!
</Callout>

Improved hybrid & vector search
-------------------------------

<Callout>
  AI search is an experimental feature, read the docs to [enable AI search](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-7&utm_source=blog#activate-vector-search).
</Callout>

### New OpenAI embedding models

Starting with Meilisearch 1.7, you can use OpenAI’s third-generation embedding models:

*   `text-embedding-3-small`
*   `text-embedding-3-large`

Compared to the previous `text-embedding-ada-002` model, these new models offer improvements allowing to balance search results relevancy and cost efficiency.

The `small` model is more cost-effective, generating smaller vector embeddings that require less storage. The `large` focuses on delivering higher precision in search results, making it ideal to optimize relevancy.

### Configuring OpenAI model dimensions

When using the new `text-embedding-3-small` and `text-embedding-3-large` OpenAI models, you can now configure **dimensions** for your model. When not provided, the model's defaults dimensions are used.

You can do so when updating your settings:

```json
{
  // other settings...

  "embedders": {
    "small": {
      "source": "openAi",
      "model": "text-embedding-3-large",
      "dimensions": 512 // must be >0, must be <= 3072 for "text-embedding-3-large"
    },
  }
}

```

Higher dimensions typically offer more complex embeddings which can lead to more accurate search results. However, increased dimensions are more costly to compute and require more storage.

<Callout>
  You cannot customize dimensions when using OpenAI `text-embedding-ada-002`.
</Callout>

### Use AI Search on Meilisearch Cloud

Meilisearch hybrid search and vector search are coming to Meilisearch Cloud. We're rolling out invites every day. Join the beta. 👇

[Get early access](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_campaign=release-v1-7&utm_source=blog)

GPU support for computing Hugging Face embeddings
-------------------------------------------------

Meilisearch 1.7 comes with the ability to enable GPU support for [self-hosted Meilisearch](https://www.meilisearch.com/docs/learn/getting_started/installation?utm_campaign=release-v1-7&utm_source=blog) for computing Hugging Face embeddings. This can significantly speed up the embedding generation process.

Find detailed instructions in our dedicated guide on [Using GPU to compute Hugging Face embeddings](https://www.meilisearch.com/docs/learn/cookbooks/computing_hugging_face_embeddings_gpu?utm_campaign=release-v1-7&utm_source=blog).

Display ranking scores details at search
----------------------------------------

Meilisearch 1.7 stabilizes the `showRankingScoreDetails` search parameter that was introduced in [Meilisearch 1.3](/blog/v1-3-release/?utm_campaign=release-v1-7&utm_source=blog This parameter adds a `_rankingScoreDetails` field to each document in search results, enabling further inspection of the search results.

You can include the parameter when making searches:

```sh
curl \
  -X POST 'https://edge.meilisearch.com/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "batman",
    "showRankingScoreDetails": true
  }'

```

Learn more about the [`_rankingScoreDetails` object](https://www.meilisearch.com/docs/reference/api/search#ranking-score-details?utm_campaign=release-v1-7&utm_source=blog) and how to use it with your preferred SDK in the API reference.

Contributors shout-out
----------------------

We want to give a massive thank you to all contributors that made this release possible: [@capJavert](https://github.com/capJavert), [@codesmith-emmy](https://github.com/codesmith-emmy), [@Elliot67](https://github.com/Elliot67) and [@Gosti](https://github.com/Gosti) for their PRs in Meilisearch, and [@agourlay](https://github.com/agourlay), [@choznerol](https://github.com/choznerol), [@ngdbao](https://github.com/ngdbao), [@timvisee](https://github.com/timvisee), and [@xshadowlegendx](https://github.com/xshadowlegendx) for their work on [charabia](https://github.com/meilisearch/charabia).

Special thanks to our SDKs maintainers, thanks to whom Meilisearch is available across so many languages. 🫶

* * *

And that’s a wrap for v1.7! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0).

Stay in the loop of everything Meilisearch by subscribing to our [newsletter](https://dub.sh/meili-newsletter). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-7&utm_source=blog).

Meilisearch 1.7

On March 7th, we hosted a live showcase session introducing hybrid search and other Meilisearch product updates. Watch a recording and see the hybrid search demo [here](https://www.youtube.com/watch?v=Lgv8iTdWDQc&t=881s).

* * *

AI's influence on search
------------------------

The rapid advancement of AI technology is undeniably reshaping human-machine interactions. Historically, communication with machines necessitated intermediaries like programming languages and structured scripts.

Now, with the arrival of GPTs and AI-assisted bots, machine-learning models are approaching a deeper comprehension of natural human language. This evolution brings humans and machines closer, changing the way we seek and find information.

From keyword-based searches to discovery-driven inquiries, people’s approach to search evolves: we are moving from searching for known items to exploring unknown territories, searching and finding the concepts instead of focusing solely on words.

Take a streaming media company: the searches across their platform might include the inquiry for a specific movie title, such as "Back to the Future," as well as conceptually-driven searches like "Give me recommendations for a feel-good sci-fi movie from the 80s".

Consider everyday scenarios: spotting an item on the street and using an image to search for it, or conducting voice searches instead of typing in words on keyboards. These are manifestations of our changing search behaviors.

The pinnacle: hybrid search
---------------------------

While traditional search methods excel in keyword-based queries, they falter with complex, nuanced requests. This is where Meilisearch’s new hybrid search approach comes into play, enriching the efficiency and precision of our robust full-text search engine with a depth of semantic understanding, catering to a wide array of use cases driven by customer-generated data.

Integrating models from AI giants like [OpenAI](https://www.meilisearch.com/docs/learn/experimental/vector_search#generate-auto-embeddings-with-openai) (open-source embedders generation feature is set for the next release cycle), Meilisearch enables users to create and fine-tune vector embeddings, tailoring their search engine to specific business needs. Moreover, evolving AI models learn from user interactions, continuously improving search precision.

What‘s included in the new release package?
-------------------------------------------

The AI-enhanced hybrid search from Meilisearch leverages our existing robust search infrastructure, including advanced sorting, filtering, faceting, and geo-sorting capabilities, among others.

New release features
--------------------

*   **Enhanced search relevancy**: The AI-enhanced layers bring a deeper semantic understanding, significantly improving result quality.
*   **AI embedders autogeneration:** Choose between using a third-party solution like OpenAI or your own locally generated embeddings. An open-source embedders generation feature is set for the next release cycle.

![](https://unable-actionable-car.media.strapiapp.com/Group_143725866_7f8309846d.png)

Add a new AI embedder in Meilisearch Cloud.

*   **Improved prompting:** Enhanced to incorporate key information templates, aiding the system in prompt ingestion and relevant metadata interpretation.
*   **Semantic ratio control:** Customize your search to span anywhere from purely keyword-based to entirely semantic.  
    

![](https://unable-actionable-car.media.strapiapp.com/Group_143725865_1_f3910ea1ce.png)

Choose where you want your results to fall on the spectrum from pure keyword-based to fully semantic search.

*   **Multimodal search capabilities:** Expanding search functionality to include image, sound, and video content.

**Democratizing search**
------------------------

At Meilisearch, our journey has always been anchored in simplicity, performance, and out-of-the-box relevancy. Our core mission remains steadfast: make it easy to find anything anywhere by providing a go-to-search solution for every team.

> “Meilisearch's hybrid search will democratize advanced search technology, focusing on simplicity and open-source customizability”  
> \- **Quentin de Quelen, Meilisearch’s CEO and Co-founder.**

As we evolve, our methods become more sophisticated, adapting to the ever-changing needs of businesses and developers alike. In a world brimming with data, effective content discovery is essential, not just a luxury. Our vision combines innovation with accessibility, aiming to make cutting-edge search technology inclusive for all.

The launch of our hybrid search product marks a significant milestone for Meilisearch and the search technology landscape. We invite you to try the new solution and join us in shaping the future of search.

* * *

Our AI-powered hybrid search is currently in beta. Eager to try it out? Sign up for our waitlist by using the link below, and our team will be in contact soon.

[Sign up now](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq?utm_source=Blog&utm_campaign=HybridSearchRelease)

* * *

Stay in the loop by subscribing to our [newsletter](https://dub.sh/meili-newsletter?&utm_source=blog&utm_campaign=HybridSearchRelease). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_source=Blog&utm_campaign=HybridSearchRelease) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions?utm_source=Blog&utm_campaign=HybridSearchRelease).

For anything else, join our developer community on [Discord](https://discord.com/invite/hTD5hU2yWV?utm_source=Blog&utm_campaign=HybridSearchRelease).

Introducing hybrid search: combining full-text and semantic search for optimal balance

The new dashboard incorporates a set of metrics providing insights into your project’s performance. We want to help engineers move faster by making it easier to spot issues during implementation and having better visibility on your Meilisearch database health.

What metrics can you expect?
----------------------------

This first iteration comes with:

*   30 days of data retention
*   Real-time mode

**Search performance metrics**

*   Search latency
*   Maximum search queries per second

**Operations metrics**

*   Bandwidth (in/out)
*   API Calls — Succeeded/Failed

The default view combines the data from the indexes per project, but it’s also possible to filter out the insights for a specific index of your choice. The dashboard also allows you to select a custom range using a date picker or get real-time data insights:

![](https://unable-actionable-car.media.strapiapp.com/monitoring_metrics_custom_gif_gif_382681a817.jpg)

Why use monitoring**?**
-----------------------

With our brand-new dashboard, you can:

*   Ensure your users are getting lightning-fast search responses for every query
*   Monitor how your search functionality holds up under varying loads, not only during peak traffic conditions but also while indexing
*   Identify and rectify any performance bottlenecks or errors
*   Proactively maintain and optimize your project

**Availability and plans**
--------------------------

The new Cloud monitoring metrics are available for all Cloud users across all pricing tiers. Each tier has access to the same metrics and chart functionalities of the monitoring metrics dashboard.

[Try it in Meilisearch Cloud](https://cloud.meilisearch.com/?utm_campaign=monitoring-v1&utm_source=blog)

Learn how to [get started with Meilisearch Cloud](https://www.meilisearch.com/docs/learn/getting_started/cloud_quick_start?utm_campaign=monitoring-v1&utm_source=blog) and read the [Monitoring documentation](https://www.meilisearch.com/docs/learn/cloud/monitoring#requirements?utm_campaign=monitoring-v1&utm_source=blog).

* * *

Stay in the loop by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&utm_campaign=monitoring-v1&utm_source=blog). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=monitoring-v1&utm_source=blog) and participate in our [Product Discussions](https://github.com/meilisearch/product/discussions/?utm_campaign=monitoring-v1&utm_source=blog).

For anything else, join our developer community on [Discord](https://discord.meilisearch.com/?utm_campaign=teams-release&utm_source=blog).

Cloud monitoring metrics have arrived!

We’re announcing the release of Meilisearch 1.6. Let’s dive into some of the most important changes. You can also view the [full changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.6.0) on GitHub.

<Callout>
  Meilisearch 1.6 is also available on [Meilisearch Cloud](https://cloud.meilisearch.com/login?utm_campaign=release-v1-6&utm_source=blog), including all experimental features.
</Callout>

Experimental feature: hybrid search
-----------------------------------

Meilisearch introduces hybrid search. It combines full-text and semantic search to enhance the accuracy and comprehensiveness of search results. Picture a movie app like [where2watch](https://where2watch.meilisearch.com/?utm_campaign=release-v1-6&utm_source=blog). Now, your users will be able to find those movies they can't quite name but remember the story.

Furthermore, Meilisearch now streamlines the creation of vector embeddings. Choose your preferred embedders and Meilisearch will handle all interactions with external tools for you.

### Configuring embedders

You can [configure the embedders](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-6&utm_source=blog#generate-vector-embeddings) in your index settings. Select from three types of embedders for your needs:

`openAI`:

*   Uses the [OpenAI](https://openai.com/) API for computing embeddings
*   Requires an OpenAI API key for operation

`huggingFace`:

*   Enables local computation of embeddings by downloading models from the [HuggingFace Hub](https://huggingface.co/models)
*   Operates on your CPU–not your GPU–which may impact indexing performance

`userProvided`:

*   Functions similarly to Meilisearch v1.3, with a key difference: you must define a specific embedder
*   Allows you to add pre-computed embeddings into your documents. You perform searches using vectors instead of text.

To use hybrid search, define at least one embedder in the index settings:

```JSON
{
  "embedders": {
    "default": {
      "source":  "openAi",
      "apiKey": "<your-OpenAI-API-key>",
      "model": "text-embedding-ada-002",
      "documentTemplate": "A movie titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
    },
    "image": {
      "source": "userProvided",
      "dimensions": 512
    },
    "translation": {
      "source": "huggingFace",
      "model": "sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2",
      "documentTemplate": "A movie titled '{{doc.title}}' whose description starts with {{doc.overview|truncatewords: 20}}"
    }
  }
}

```

The `documentTemplate` field acts as a blueprint for creating your document's embedding. It uses the [Liquid template language](https://shopify.dev/docs/api/liquid). While its inclusion is optional, it is highly recommended, especially since embedding models are optimized for concise texts. It keeps only the necessary content, excluding non-essential data like `id`, and helps in adding context to increase relevancy.

### **Hybrid search**

To [perform a hybrid search](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-6&utm_source=blog#vector-search-with-auto-embeddings), use the hybrid field within the `POST /index/:index_uid/search` route.

```JSON
{
    "q": "Plumbers and dinosaurs",
    "hybrid": {
        "semanticRatio": 0.9,
        "embedder": "default"
    }
}

```

`embedder`: an embedder from the options configured in your index settings.

`semanticRatio`: a floating value ranging from `0` to `1`; `1` is a  fully semantic search; `0` is an exact, match-focused full-text search; the default is `0.5` which mixes both methods.

Your control over the semantic ratio directly influences how search results are ranked. A higher semantic ratio shifts the focus towards the context and meaning behind your query, ranking results that are more semantically relevant higher. 

On the other hand, a lower semantic ratio increases the weight given to keyword accuracy in the ranking process, bringing results that closely match your specific search terms to the forefront.

### **Breaking changes in the experimental vector search API**

Meilisearch v1.6 introduces some breaking changes in the vector search API.

Previously, you could send vectors without specifying a model. Now, you must define a model in the settings:

```JSON
"embedders": {
    "default": {
      "source": "userProvided",
      "dimensions": 512
    }
}

```

Because Meilisearch now supports multiple embedders, it has updated the vector submission format from arrays to JSON objects.

*   Previews format: `“_vectors”: [[0.0, 0.1]]`
*   New format: `“_vectors”: {“image2text”: [0.0, 0.1, …]}`

For detailed information on these updates, refer to the [documentation](https://www.meilisearch.com/docs/learn/experimental/vector_search?utm_campaign=release-v1-6&utm_source=blog).

For in-depth technical information, explore the [series of articles](/blog/spotify-inspired-hybrid-search-and-rust/?utm_campaign=release-v1-6&utm_source=blog on [Arroy](https://github.com/meilisearch/arroy), an open-source repository based on [Spotify’s Annoy](https://github.com/spotify/annoy) and developed in Rust. This library, created and maintained by the Meilisearch engine team, specializes in searching for vectors in a space that are near a specified query vector.

**Performance optimization**
----------------------------

### Improved indexing speed

We're thrilled to share a major enhancement in Meilisearch's indexing performance. Our recent tests, including scenarios with frequent and partial document updates, have shown impressive results: **a reduction in indexing time by up to 50%**, and in some cases, **even as much as 75%**. 

Thanks to our latest optimizations, Meilisearch now **stores and pre-computes less data**. Additionally, during document updates, it **re-indexes or deletes only the necessary data**. For instance, in an e-commerce dataset, updating the stock level of a product results in re-indexing just the 'stock' field, rather than the entire product document.

### Disk space usage reduction

Meilisearch reduces internal data storage, leading to a more **compact database size** on your disk. With a dataset of approximately 15Mb, we observed a **40% to 50% reduction in database size**.

This enhancement not only reduces the database size, but also **improves its stability**, making the space savings more evident as the number of documents increases.

**New feature: customize proximity precision**
----------------------------------------------

To further reduce indexing speed, Meilisearch now allows you to tailor the accuracy of the [proximity ranking rule](https://www.meilisearch.com/docs/learn/core_concepts/relevancy?utm_campaign=release-v1-6&utm_source=blog#3-proximity) to your specific needs.

The proximity ranking rule is computationally demanding and may lead to longer indexing times. Reducing its accuracy can greatly enhance performance, and in most scenarios, it will not substantially affect the relevancy of the results.

To adjust its impact [configure the `proximityPrecision` setting](https://www.meilisearch.com/docs/reference/api/settings?utm_campaign=release-v1-6&utm_source=blog#proximity-precision):

```JSON
curl \
  -X PATCH 'http://localhost:7700/indexes/books/settings/proximity-precision' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "proximityPrecision": "byAttribute"
  }'
```

The default `proximityPrecision` setting is `byWord`, which calculates proximity based on exact word distances. 

The `byAttribute` setting considers words in the same attribute as proximate, regardless of their exact distance.

Using `byAttribute` can boost the indexing speed, but it might slightly change how relevant the results are. This becomes more noticeable in searches where it's important for words to be close to each other.

For example, when you're looking through song lyrics or long articles, like trying to find 'world war' in a bunch of Wikipedia pages, you might end up with results that contain these words but not necessarily close together or in the desired order. This is also true for [phrase searches](https://www.meilisearch.com/docs/reference/api/search?utm_campaign=release-v1-6&utm_source=blog#phrase-search) and for searches involving [multi-word synonyms](https://www.meilisearch.com/docs/learn/configuration/synonyms?utm_campaign=release-v1-6&utm_source=blog#multi-word-synonyms), where the specific combination of words is crucial.

**New feature: task queue webhook**
-----------------------------------

Meilisearch now offers a webhook feature to notify a custom URL when an asynchronous task finishes (either succeeds, fails, or gets canceled).

This feature is particularly useful for streamlining workflows, saving you from polling the tasks route.

Set up your webhook at launch using these environment variables:

```bash
MEILI_TASK_WEBHOOK_URL=https://mywebsite.com/my-super-webhook?user=1234&number=8

MEILI_TASK_WEBHOOK_AUTHORIZATION_HEADER='Bearer 12340987546wowowlolol'
```

You can also use the respective command-line options.

Once set up, the webhook sends out a payload in [JSON Lines (ndjson) format](https://www.meilisearch.com/docs/learn/core_concepts/documents?utm_campaign=release-v1-6&utm_source=blog#ndjson) to your specified URL, containing the list of finished tasks:

```json
//POST HTTP request to https://myproject.com/mywebhook?common=people

{"uid":4,"indexUid":"movie","status":"failed","type":"indexDeletion","canceledBy":null,"details.deletedDocuments":0,"error.message":"Index `movie` not found.","error.code":"index_not_found","error.type":"invalid_request","error.link":"https://docs.meilisearch.com/errors#index_not_found","duration":"PT0.001192S","enqueuedAt":"2022-08-04T12:28:15.159167Z","startedAt":"2022-08-04T12:28:15.161996Z","finishedAt":"2022-08-04T12:28:15.163188Z"}
{"uid":5,"indexUid":"movie","status":"failed","type":"indexDeletion","canceledBy":null,"details.deletedDocuments":0,"error.message":"Index `movie` not found.","error.code":"index_not_found","error.type":"invalid_request","error.link":"https://docs.meilisearch.com/errors#index_not_found","duration":"PT0.001192S","enqueuedAt":"2022-08-04T12:28:15.159167Z","startedAt":"2022-08-04T12:28:15.161996Z","finishedAt":"2022-08-04T12:28:15.163188Z"}
```

**Experimental feature: limit the number of batched tasks**
-----------------------------------------------------------

To speed up the indexing process, Meilisearch processes similar [tasks](https://www.meilisearch.com/docs/learn/async/asynchronous_operations?utm_campaign=release-v1-6&utm_source=blog#tasks-and-asynchronous-operations) in large batches. However, excessive [queued tasks](https://www.meilisearch.com/docs/learn/async/asynchronous_operations?utm_campaign=release-v1-6&utm_source=blog#task-queue) can occasionally cause crashes or stalls.

To control the number of batched tasks, set the limit at launch using either the command-line argument `--experimental-max-number-of-batched-tasks`, the `MEILI_EXPERIMENTAL_MAX_NUMBER_OF_BATCHED_TASKS` environment variable, or the [configuration file](https://www.meilisearch.com/docs/learn/configuration/instance_options?utm_campaign=release-v1-6&utm_source=blog#configuration-file).

**Contributors shout-out**
--------------------------

We are really grateful for all the community members who participated in this release.We would like to thank [@Karribalu](https://github.com/Karribalu) and [@vivek-26](https://github.com/vivek-26) for their help with [Meilisearch](https://github.com/meilisearch/meilisearch). We also want to send a special shout-out to our SDKs maintainers 🦸

* * *

And that’s it for v1.6! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.6.0).

Stay in the loop of everything Meilisearch by subscribing to [the newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=release-v1-6&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions?ref=blog.meilisearch.com).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-6&utm_source=blog).

Meilisearch latest news and company updates