Meilisearch 1.8

New: negative search

Meilisearch 1.8 allows you to exclude specific terms from your search results using the - operator. 

For example, to find documents that contain the word "room" but not "escape", place a - before "escape”:

curl 
  -X POST 'https://edge.meilisearch.com/indexes/places/search' 
  -H 'Content-Type: application/json' 
  --data-binary '{ "q": "-escape room" }'

Experimental: Improved hybrid search

New Ollama embedder

Meilisearch 1.8 introduces support for Ollama models. Configure the embedder in your index settings:

"default": {
  "source": "ollama", // 👈 Use the ollama source
  "url": "http://localhost:11434/api/embeddings",
  // ☝️ optional, fetched from MEILI_OLLAMA_URL environment variable if missing
  "apiKey": "<your-api-key>",
  // ☝️ optional
  "model": "nomic-embed-text",
  "documentTemplate": "{{doc.field_name}}" // like for openAI and huggingFace sources
}

New REST Embedder

Starting with Meilisearch 1.8, you can use any embedder accessible via a REST API. Here’s how to configure an OpenAI embedder with the rest source:

{
  "source": "rest",
  "apiKey": "<your-openai-api-key>",
  "dimensions": 1536,
  "url": "https://api.openai.com/v1/embeddings",
  "query": {
    "model": "text-embedding-ada-002"
  },
  "inputField": ["input"],
  "inputType": "textArray",
  "pathToEmbeddings": ["data"],
  "embeddingObject": ["embedding"]
}

For more details on the API and available parameters, please consult the documentation on REST embedders.

New embedder setting: distribution shift

Depending on the embedding model, vector embeddings are not always uniformly distributed within the vector space When vectors are overly clustered, semantic search scores can be very closely ranked. As a result of semantic scores being packed, hybrid results might have sub-optimal relevancy.

To address this issue, you can now shift the distribution of results with an affine transformation. This transformation normalizes semantic scores to prevent them from clustering too high or too tightly, making them more comparable to the keyword scores.

"default": {
  "source": "huggingFace",
  "model": "MODEL_NAME",
  "distribution": {  // describes the natural distribution of results
    "mean": 0.7, // mean value
    "sigma": 0.3 // standard deviation
  }
}

• The mean value represents the central point around which you observe semantic search scores to be densely packed. In the provided code sample, the setting indicates that semantic scores from this model tend to cluster around a mean of 0.7.
• The sigma value determines the spread of ranking scores from the mean (standard deviation). The sigma provided in the code sample above means that before shifting the distribution, the results tend to have an average distance of 0.3 to the mean.

This allows for a more effective comparison of _rankingScore between semantic and keyword search results, leading to improved results ranking.

Breaking: trimmed API response

To speed up the search response and reduce bandwidth usage, Meilisearch v1.8 no longer returns the following fields in the search responses:

• the vector field, which returned the query vector 
• the value field included in the vectorSort object, which is returned when you add `”showRankingScoreDetails”: true”` to a search query
• the _semanticScore field; its value is the same as _rankingScore, which you should use instead. Set ”showRankingScore”: true” in the search parameters

Search robustness improvements

New search cut-off setting

To enhance stability and performance, Meilisearch automatically terminates any search request that exceeds 1500 milliseconds. 

To adjust this timeout threshold, you can customize the searchCutoffMs setting:

curl 
  -X PATCH 'https://edge.meilisearch.com/indexes/movies/settings' 
  -H 'Content-Type: application/json' 
  --data-binary '{
    "searchCutoffMs": 150
  }'

Experimental: concurrent search requests limit

This release introduces a limit on concurrent search requests to prevent Meilisearch from using excessive amounts of RAM and crashing. The default queue limit is set to 1000 requests. 

To modify this limit, restart your self-hosted instance with the --experimental-search-queue-size option:

./meilisearch --experimental-search-queue-size 100

👉 This limit does not impact the search performance. It only controls the number of search requests that can be queued simultaneously to enhance system security.

Contributors shout-out

We are really grateful for all the community members who participated in this release. We would like to thank: @availhang, @hack3ric, @jakobklemm, @mohsen-alizadeh, @redistay, @rohankmr414, @shuangcui, @Toromyx, @6543 for their help with Meilisearch, and @Gusted, @mosuka, @6543, for their work on Charabia.

Special thanks to our SDKs maintainers, thanks to whom Meilisearch is available across so many languages. 🫶

And that’s a wrap for v1.8! This release post highlights the most significant updates. For an exhaustive listing, read the changelog on Github.

Stay in the loop of everything Meilisearch by subscribing to our newsletter. To learn more about Meilisearch's future and help shape it, take a look at our roadmap and come participate in our Product Discussions.

For anything else, join our developers community on Discord.