In this article, let's take a look at some of the most significant changes in Meilisearch's latest update. This release brings you new features such as smart crop and deterministic API keys. With v0.28, we have stabilized our API, taking the first step towards a v1.0 🎉 This stabilization brings multiple changes. You can read [the full changelog on GitHub](https://github.com/meilisearch/meilisearch/releases/tag/v0.28.0), but we’ll go over the main ones in this article.

**New feature: smart crop**
---------------------------

Instead of considering the first search term match as the best cropping location, Meilisearch centers the crop around the largest number of unique matches, giving priority to terms that are closer to each other and follow the original query order.  Meilisearch also considers context when cropping and prioritizes keeping the sentence together.

Given the following string:

<Callout>
 “A young elephant, whose oversized ears enable him to fly, helps save a struggling circus, but when the circus plans a new venture, Dumbo and his friends discover dark secrets beneath its shiny veneer.”‌
</Callout>

If the search query is `Dumbo` and the `cropLength` is `5`, Meilisearch will now return:

`"… Dumbo and his friends discover…”`

Instead of:

`"…new venture, Dumbo and his…”`

**New feature: deterministic API keys**
---------------------------------------

A **deterministic algorithm** is an algorithm that, given a particular input, always produces the same output, with no randomness involved.

You can create a deterministic `key` value by specifying a `uid` field at creation. The `uid` value must follow the [uuid v4](https://www.sohamkamani.com/uuid-versions-explained) format. If you don't specify anything, Meilisearch automatically generates the `uid` for you.

The value of the `key` field is generated by hashing the master key with the `uid`. The same combination always results in the same `key` value.

This will allow you to have the same set of API keys across different Meilisearch instances. Henceforth, when upgrading or redeploying your Meilisearch instance, you’ll be able to keep your API keys.

<Callout>
 As a result of these modifications, keys imported from older versions of Meilisearch will have their `key` and `uid` fields regenerated. When updating your Meilisearch instance, **you will need to update your keys**.
</Callout>

We have also added a `name` field to make API key retrieval more convenient. A key object should now look like this:

```JSON
{
 "name": null,
 "description": "Manage documents: Products/Reviews API key",
 "key": "d0552b41536279a0ad88bd595327b96f01176a60c2243e906c52ac02375f9bc4",
 "uid": "6062abda-a5aa-4414-ac91-ecd7944c0f8d",
 "actions": [
 "documents.add",
 "documents.delete"
 ],
 "indexes": [
 "products",
 "reviews"
 ],
 "expiresAt": "2021-12-31T23:59:59Z",
 "createdAt": "2021-10-12T00:00:00Z",
 "updatedAt": "2021-10-13T15:00:00Z"
}
```

‌Other changes regarding API key management include:

* being able to retrieve, update, and delete a key by either the `key` or `uid` fields
* introducing new actions to manage API keys (`keys.get`, `keys.create`, `keys.update`, `keys.delete`)
* removing the possibility of updating the `actions`, `indexes`, or `expiresAt` properties of an API key after creation for security reasons

**Breaking change: search nomenclature**
----------------------------------------

We’re on the road to v1, which means defining a stable API. To improve clarity, we have made several changes to the naming of some search parameters and response fields in the `/indexes/{uid}/search` endpoint.

Search parameters formerly known as `facetsDistribution` and `matches` are now called `facets` and `showMatchesPosition`, respectively.

The response fields returned when using those parameters are now `facetDistribution` instead of `facetsDistribution` –note the suppression of the _s–_ and `_matchesPosition` instead of `_matchesInfo`.

The response field `nbHits` has been renamed `estimatedTotalHits`. This value was recurrently used to calculate the number of search result pages, which we strongly advise against. To learn how to paginate with Meilisearch without using `nbHits`, check out this [fresh new guide](https://docs.meilisearch.com/learn/advanced/pagination.html).

For the following query:

```bash
curl \
 -X POST 'http://localhost:7700/indexes/movies/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{
 "q": "Shazam",
 "facets: ["genres"],
 "showMatchesPosition": true
 }'
```

‌You’ll get the following response:

```JSON

{
 "hits": [
 {
 "id": "287947",
 "title": "Shazam!",
 "poster": "https://image.tmdb.org/t/p/w500/xnopI5Xtky18MPhK40cZAGAOVeV.jpg",
 "overview": "A boy is given the ability to become an adult superhero in times of need with a single magic word.",
 "release_date": 1553299200,
 "genres": [
 "Action",
 "Comedy",
 "Fantasy"
 ],
 "_matchesPosition": {
 "title": [
 {
 "start": 0,
 "length": 6
 }
 ]
 }
 },
 ...
 ],
 "estimatedTotalHits": 3,
 "query": "Shazam",
 "limit": 20,
 "offset": 0,
 "processingTimeMs": 4,
 "facetDistribution": {
 "genres": {
 "Action": 3,
 "Animation": 2,
 "Comedy": 1,
 "Fantasy": 1
 }
 }
}
 
```

**Breaking change: task management**
------------------------------------

### **Browsing tasks**

We have added a new pagination system to the `/tasks` endpoint.

With this change, it’s much easier to browse through the tasks, as their number can quickly grow in instances with a large number of asynchronous operations.

For each call to this endpoint, the response will return the following fields:

* `limit`: number of tasks returned (defaults to 20)
* `from`: the `uid` of the first task returned
* `next`: the `uid` of the next task

To view the next page of results, you would repeat the same query, replacing the value of `from` with the value of `next`. When the value of `next` is `null`, there are no more tasks to view.

This type of pagination system is called _keyset pagination_. As opposed to offset pagination used to browse indexes, documents, and keys, it has two main advantages: it prevents any inconsistencies and, since there is no need to scan and count records, it’s more efficient, which is a significant advantage when your task queue grows fast.

### **Filtering tasks**

We have also made [the task list filterable](https://docs.meilisearch.com/reference/api/tasks.html#filtering-tasks). You can now get tasks by `status`, `type`, or `indexUid`.

For example, the following command returns all tasks belonging to the index movies that succeeded:

```bash
curl -X GET 'http://localhost:7700/tasks?indexUid=movies&status=succeeded'
```

‌These modifications have led to the deletion of the `GET /indexes/:indexUid/tasks`  and the `GET /indexes/:indexUid/tasks/:taskUid` endpoints.

**Breaking change: dumps**
--------------------------

Dump creation has always been an asynchronous operation but used a separate queue from the task queue. With v0.28, dumps have become tasks. This has resulted in a new task type called `dumpCreation`.

Despite being tasks and thus sharing the same queue, **dumps are given priority**. They’ll be processed as soon as the current task is done running. You can think of dumps as VIPs in a club; even though they arrived last - which is reflected in their `taskUid` - they get to skip the line.

[via GIPHY](https://giphy.com/gifs/fail-fun-entertainment-hKYigb3Djp8pa)

**Contributors’ experience**
----------------------------

We have worked hard on improving the contribution experience of our tokenizer: [charabia](https://github.com/meilisearch/charabia/). The tokenizer’s role is to split a sentence or phrase into smaller units of language, called tokens. It is a critical factor in the quality of search results. Now, it is much easier to add languages to Meilisearch. You just need to follow the instructions on [CONTRIBUTING.md](https://github.com/meilisearch/charabia/blob/main/CONTRIBUTING.md).

Meilisearch works perfectly with any space-separated language and has special support for Japanese and Chinese. **We now support Hebrew,** too, thanks to our awesome community! Other languages will still work, but the quality and relevancy of search results may vary significantly.

We would love to provide global language support. The more feedback we get from native speakers, the easier it is for us to understand how to improve performance for those languages. If you want to help us [support your language](https://docs.meilisearch.com/learn/what_is_meilisearch/language.html#improving-our-language-support), we are eager to hear from you and see how we could make progress together!

**Other changes**
-----------------

* We have added pagination to the response of the `GET /indexes` and the `GET /keys` endpoints, and we have improved pagination for the `GET /indexes/{uid}/documents`
* For performance reasons, we have decided to limit the number of facet values returned per faceted attribute. This limit is customizable and defaults to 100
* You can customize the number of documents Meilisearch returns on search. The default limit is 1000 and protects the database from malicious scraping. Beware that increasing this limit can affect performance

We apologize in advance for any inconvenience caused by all these changes. It’s for a good cause: we are making these changes now to move towards v1.0 and avoid breaking changes later. Don’t hesitate to reach out to us if you need support or have any doubts. We are always happy to help!

**Contributors**
----------------

We are really grateful for this amazing community. We want to thank [@0x0x1](https://github.com/0x0x1), [@choznerol](https://github.com/choznerol), [@pierre-l](https://github.com/pierre-l), [@ryanrussell](https://github.com/ryanrussell), [@Thearas](https://github.com/Thearas), and [@walterbm](https://github.com/walterbm) for their help with Meilisearch, and [@matthias-wright](https://github.com/matthias-wright) for his help with [milli](https://github.com/meilisearch/milli). We want to send a special shout-out to [@benny-n](https://github.com/benny-n) for adding Hebrew to our [tokenizer](https://github.com/meilisearch/charabia).

And that’s it for v0.28! Remember to check the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v0.28.0) for the full release notes, and see you next time!

‌‌

What's new in v0.28

This article looks at some of the main and breaking changes, but you can view the [full changelog](https://github.com/meilisearch/meilisearch/releases/tag/v0.27.0) here.

**New feature: nested field support**
-------------------------------------

Meilisearch now offers full support for nested document fields, allowing you to use dot notation to access these fields. 
Let's take an example. If you have a book store index, you might want to make search results filterable by an author's surname but not by their given name:

```json
curl \
 -X POST 'http://localhost:7700/indexes/books/settings/filterable-attributes' \
 -H 'Content-Type: application/json' \
 --data-binary '[
 	"author.surname"
 ]'

```

You may use dot notation to access object properties anywhere in Meilisearch where you would use an attribute. For instance, you can access index settings like sortable attributes and searchable attributes or search parameters like `attributesToHighlight`.

**New feature: customize typo tolerance**
-----------------------------------------

Meilisearch is typo tolerant! This means our engine can interpret your search even if it contains typos or misspellings. With v0.27, you can customize the typo tolerance settings for an index using the new [update typo tolerance](https://www.meilisearch.com/docs/reference/api/settings#update-typo-tolerance-settings) endpoint or the existing [update settings endpoint](https://docs.meilisearch.com/reference/api/settings.html#update-settings):

The new API endpoints for typo tolerance accept a `typoTolerance` object with the following properties:

* `enabled`: whether the typo tolerance feature is enabled. Defaults to true
* `disableOnAttributes`: disables typo tolerance on specific document attributes. Typo tolerance is enabled on all attributes by default.
* `disableOnWords`: disables typo tolerance on a set of query terms given during search. Typo tolerance does not ignore any words by default.
* `minWordSizeForTypos`: Meilisearch only accepts typos in words over a certain size
* `oneTypo`: Customize the minimum word size for accepting one typo. Defaults to 5 characters.
* `twoTypos`: Customize the minimum word size for accepting two typos. Defaults to 9 characters.

Suppose you have a `movie` index that contains many character names. You might want Meilisearch to never look for typos in certain words but be more permissive with everything else:

```json
curl --location --request POST 'http://localhost:7700/indexes/children-movies/settings/typo-tolerance' \
--header 'Authorization: Bearer <API_KEY>' \
--header 'Content-Type: application/json' \
--data-raw '{
 "enabled": true,
 "minWordLengthForTypo": {
 "oneTypo": 3,
 "twoTypos": 7
 },
 "disableOnWords": ["Shrek"],
 "disableOnAttributes": []
}'

```

You can read more on typo tolerance [here](https://docs.meilisearch.com/learn/configuration/typo_tolerance.html).

New feature: improved highlighting and cropping search parameters
-----------------------------------------------------------------

This release brings three new search parameters: `highlightPreTag`, `highlightPostTag`, and `cropMarker`. The first two allow you to customize the appearance of highlighted search terms further, while the last one should be used in conjunction with `attributesToCrop`. 
When using `attributesToHighlight`, Meilisearch encloses matching terms in `` and strings by default. You can now configure these values to include any string using the `highlightPreTag`, and `highlightPostTag`. As the name indicates, `highlightPreTag` specifies the tag before the highlighted query terms, and `highlightPostTag` after the highlighted query terms. 
Likewise, when using `attributesToCrop`, you can now change the default `"…"` `cropMarker` to any string for marking crop boundaries. 
When designing an application that allows users to search through a database of movies, you might want to change `cropMarker` and the highlight tags to match your design:

```json
curl --location --request POST '<http://localhost:7700/indexes/movies/search>' \\
--header 'Authorization: Bearer <API_KEY>' \\
--header 'Content-Type: application/json' \\
--data-raw '{
 "q":"shifu",
 "attributesToCrop":["overview"],
 "attributesToHighlight":["overview"],
 "cropLength":10,
 "highlightPreTag":"",
 "highlightPostTag":"",
 "cropMarker":"[…]"
}'

```

Using the above query, the search term `shifu` is enclosed within tags, and the cropped text is marked by `[…]`:

```json
{
 "id": "50393",
 "title": "Kung Fu Panda Holiday",
 "overview": "The Winter Feast is Po's favorite holiday. Every year he and his father hang decorations, cook together, and serve noodle soup to the villagers. But this year Shifu informs Po that as Dragon Warrior, it is his duty to host the formal Winter Feast at the Jade Palace. Po is caught between his obligations as the Dragon Warrior and his family traditions: between Shifu and Mr. Ping.",
 "_formatted": {
 "id": "50393",
 "title": "Kung Fu Panda Holiday",
 **"overview": "[…]the villagers. But this year Shifu informs Po that as[…]"**
 }
}

```

For more information, check out our guide on [search parameters](https://docs.meilisearch.com/reference/api/search.html#search-parameters).

**Breaking change: \`cropLength\` behavior**
--------------------------------------------

The `cropLength` search parameter is now specified in the number of words instead of characters. 
Let’s take an example query:

```json
curl --location --request POST '<http://localhost:7700/indexes/movies/search>' \\
	--header 'Authorization: Bearer <API_KEY>' \\
--header 'Content-Type: application/json' \\
--data-raw '{
 "q":"shifu",
 "attributesToCrop":["overview"],
 "cropLength":10
}'

```

Meilisearch v0.27 returns:

```json
{
 "id": "50393",
 "title": "Kung Fu Panda Holiday",
 "overview": "The Winter Feast is Po's favorite holiday. Every year he and his father hang decorations, cook together, and serve noodle soup to the villagers. But this year Shifu informs Po that as Dragon Warrior, it is his duty to host the formal Winter Feast at the Jade Palace. Po is caught between his obligations as the Dragon Warrior and his family traditions: between Shifu and Mr. Ping.",
 "_formatted": {
 "id": "50393",
 "title": "Kung Fu Panda Holiday",
 **"overview": "…the villagers. But this year Shifu informs Po that as…",**
 }
}

```

**Breaking change: improved Docker workflow**
---------------------------------------------

Docker now starts from a different working directory: `/meili_data`. This streamlines situations when you might want to use snapshots and dumps with Docker. 
Usage remains mostly the same, but don't forget to update the data path:

```json
docker run -it --rm \\
-p 7700:7700 \\
-v $(pwd)/meili_data:/meili_data \\
getmeili/meilisearch:latest

```

With this change, dumps and snapshot features should work with Docker right out of the box. Rather than mounting separate volumes, it mounts all Meilisearch data into the meili\_data directory. 
Another consequence of this change is that the `meilisearch` binary has been moved to a new location. When using instance options to configure Meilisearch, you must now use `meilisearch`, `/meilisearch`, or `/bin/meilisearch` instead of ./meilisearch. We recommend using `meilisearch`, though. 
The updated command is:

```json
docker run -it --rm -p 7700:7700 getmeili/meilisearch:latest **meilisearch** --master-key="foobar"

```

**Other Changes**
-----------------

1. A typo on the first letter of a word will count as two typos. This reduces the search space for typo tolerance resulting in a faster search response time
2. Meilisearch now has a non-customizable limit of returning 1000 documents per search to protect the database from malicious scraping.
3. We added two new instance options to allow you more control over the use of machine resources during indexation(--max-indexing-memory and —-max-indexing-threads).
4. We now support Japanese.
5. The speed of adding new documents to a non-empty index has improved.

**Contributors**
----------------

A big thank you to all our contributors! We wouldn't have made it here without your support. This month, we want to send a special shout-out to [@miiton](https://github.com/miiton), [@djKooks](https://github.com/djKooks), [@mosuka](https://github.com/mosuka) for adding Japanese support, [@2shiori17](https://github.com/2shiori17) for adding support for new instance options for RAM and CPU usage during indexation. 
 
And that's all, folks! Remember to check out the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v0.27.0) for the full release notes, and see you next month!

What's new in v0.27

In this article, let's take a look at some of the most significant changes in Meilisearch's latest update. You can check out the [full changelog here](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.26.0).

New feature: tenant tokens
--------------------------

In software development, multitenancy means that multiple users—also called tenants—share the same computing resources but have different levels of access to system-wide data. In Meilisearch, you might have one index containing data belonging to many distinct tenants.

**Tenant tokens** are small packages of encrypted data containing **security credentials** and a set of **search rules**. These search rules effectively limit what documents a search query can return. For example, you might use tokens to ensure users of a healthcare application can only search through their own medical records, even when all records are stored in the same index.

To start using tenant tokens, set up a token generation process in your application's back-end:

```js
const currentUserID = 'aUserID';

const searchRules = {
  'patient_medical_records': {
    filter: `user_id = ${currentUserID}`
  }
};

const apiKey = 'yourApiKey';

const expiresAt = new Date('2025-12-20'); // optional

const token = client.generateTenantToken(searchRules, {
  apiKey: apiKey,
  expiresAt: expiresAt,
});

```

After you have created your tenant token, you can send it to your end-user's front-end and use it to query an index:

```js
const frontEndClient = new MeiliSearch({ host: 'http://127.0.0.1:7700', apiKey: token });
frontEndClient.index('patient_medical_records').search('blood test');

```

You can read more about multitenancy and how to implement tenant tokens in your application in [our dedicated guide](https://docs.meilisearch.com/learn/security/tenant_tokens.html).

Experimental feature: auto-batching
-----------------------------------

When it comes to improving Meilisearch's indexation performance, we have often recommended users to split their documents into batches. However, finding the right batch size can present an additional challenge. On the one hand, large batches use a lot of resources; on the other, small batches are significantly slower. We know we can do better than this.

In this release, we are introducing an experimental auto-batching feature. Auto-batching will help you improve indexing speed by **automatically batching together consecutive document additions**.

You can activate auto-batching by passing the `--enable-auto-batching` flag when launching Meilisearch:

```sh
./meilisearch --enable-auto-batching

```

We are very hopeful that this feature will make Meilisearch indexation faster and easier to use. You can read more about auto-batching on [our documentation website](https://www.meilisearch.com/docs/learn/core_concepts/documents#auto-batching).

We decided to ship it in an experimental state since this feature may be heavily reworked. We need your [feedback](https://github.com/meilisearch/MeiliSearch/discussions/2070) to measure its success and make it evolve.

Breaking change: new dump behavior
----------------------------------

Importing a dump into an instance with an existing database will throw an error and Meilisearch will not launch. Prior to v0.26 the import only failed silently and gave no indication anything had gone wrong.

```sh
./meilisearch --import-dump path_to_dump_file

```

If necessary for your workflow, you can use two new command-line flags to suppress import errors: `--ignore-dump-if-db-exists` and `--ignore-missing-dump`.

Breaking change: command-line flag behavior
-------------------------------------------

Starting with v0.26, the following instance options no longer accept values when used in the command-line: `-no-analytics`, `--schedule-snapshot`, `--ignore-missing-snapshot`, `--ignore-snapshot-if-db-exists`, `--ssl-require-auth`, `--ssl-resumption`, and `--ssl-tickets`.

```js
# new behavior
meilisearch --no-analytics

# old behavior
meilisearch --no-analytics=true

```

A value is still required when using these options as environment variables.

Other changes
-------------

*   Empty CSV cells are now converted to `null` values
*   Valid but empty payloads no longer throw errors when adding and updating documents
*   armv8 binaries are deprecated in favour of aarch64 binaries
*   Improved highlighting for searches containing non-unicode characters
*   Improved `_geoPoint` sorting behavior

Contributors
------------

Thank you very much to the contributors that helped us out! We want to give a special shout-out to [@robjtede](https://github.com/robjtede), [@Thearas](https://github.com/Thearas), and [@Samyak2](https://github.com/Samyak2)—we're humbled by your effort and generosity.

* * *

And that's all folks! Remember to check out the [changelog on GitHub](https://github.com/meilisearch/meilisearch/releases/tag/v0.26.0) for the full release notes and see you soon for v0.27!

What's new in v0.26

In this article, we will take a look at some of the main and breaking changes introduced by v0.25. You can also read the [full changelog on GitHub](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.25.0).

Breaking changes
----------------

### API key management

Starting from this release, developers can now create API keys with specific permissions and expiration dates, allowing precise control over which users/applications can access which indexes and endpoints.

As part of this feature, the `private` and `public` keys are deprecated and replaced by two default API keys with similar permissions: `Default Admin API Key` and `Default Search API Key`.

The `X-MEILI-API-KEY: <apiKey>` header is replaced by the `Authorization: Bearer <apiKey>` header.

Let's say that you're creating an application in the medical industry. Here's how to create an API key that only has access to search endpoints on the `patient_medical_records` index:

```bash
curl \
 -X POST 'http://localhost:7700/keys/' \
 -H 'Content-Type: application/json' \
 -H 'Authorization: Bearer [your_master_key]' \
 --data-binary '{
 "description": "Search patient records key",
 "actions": [
 "search"
 ],
 "indexes": ["patient_medical_records"],
 "expiresAt": "2023-01-01T00:00:00Z"
 }'

```

**Response: `201 Created`**

```json
{
 "description": "Search patient records key",
 "key": "d0552b41536279a0ad88bd595327b96f01176a60c2243e906c52ac02375f9bc4",
 "actions": [
 "search"
 ],
 "indexes": [
 "patient_medical_records"
 ],
 "expiresAt": "2023-01-01T00:00:00Z",
 "createdAt": "2022-01-01T10:00:00Z",
 "updatedAt": "2022-01-01T10:00:00Z"
}

```

A user could then supply this key to make search requests on the `patient_medical_records` index:

```bash
curl \
 -X POST 'http://localhost:7700/indexes/patient_medical_records/search' \
 -H 'Content-Type: application/json' \
 -H 'Authorization: Bearer d0552b41536279a0ad88bd595327b96f01176a60c2243e906c52ac02375f9bc4' \
 --data-binary '{ "q": "trypophobia" }'

```

For more information, read our new guide to [securing a Meilisearch instance](https://www.meilisearch.com/docs/learn/security/master_api_keys).

### Asynchronous operations

With v0.25, we refashioned Meilisearch's asynchronous operations from the ground up, replacing the `updates` API with the new `tasks` API.

In a related change, creating, updating, and deleting indexes are now asynchronous operations. This fix eliminates the potential for race conditions.

Before v0.25, any request handled asynchronously (e.g. adding documents) returned an `updateId` for tracking the status of the operation:

```json
{ "updateId": 1 }

```

After v0.25, asynchronous operations will now return a summarized version of the _task_ object:

```json
{
 "uid": 1,
 "indexUid": "patient_medical_records",
 "status": "enqueued",
 "type": "documentAddition",
 "enqueuedAt": "2021-08-10T14:29:17.000000Z",
}

```

You can get the full _task_ object—and the current status of the operation—using the `/tasks` route.

```bash
curl \
 -X GET 'http://localhost:7700/tasks/1'
 
```

**Response: `200 Ok`**

```json
{
 "uid": 1,
 "indexUid": "patient_medical_records",
 "status": "succeeded",
 "type": "documentAddition",
 "details": { 
 "receivedDocuments": 1,
 "indexedDocuments": 1
 },
 "duration": "PT1S",
 "enqueuedAt": "2021-08-10T14:29:17.000000Z",
 "startedAt": "2021-08-10T14:29:18.000000Z",
 "finishedAt": "2021-08-10T14:29:19.000000Z"
}

```

Here the task has succeeded. Other possible statuses for a task are `enqueued`, `processing`, or `failed`.

When a task fails, Meilisearch returns an `error` object as part of the response:

```json
{
 "uid": 2,
 "indexUid": "patient_medical_records",
 "status": "failed",
 "type": "documentAddition",
 "details": { 
 "receivedDocuments": 1,
 "indexedDocuments": 0
 },
 "error": {
 "message": "Document does not have a `:primaryKey` attribute: `:documentRepresentation`.",
 "code": "internal",
 "type": "missing_document_id",
 "link": "https://docs.meilisearch.com/errors#missing-document-id",
 },
 "duration": "PT1S",
 "enqueuedAt": "2021-08-11T14:29:17.000000Z",
 "startedAt": "2021-08-11T14:29:18.000000Z",
 "finishedAt": "2021-08-11T14:29:19.000000Z"
}

```

For more information, check out the [Asynchronous operations](https://docs.meilisearch.com/learn/advanced/asynchronous_operations.html) article or the [tasks API reference](https://docs.meilisearch.com/reference/api/tasks.html).

Other changes
-------------

* **Cross-version compatibility:** Meilisearch v0.25 and subsequent releases are incompatible with dumps created prior to v0.22. 
 To migrate a dump from pre-v0.22 to v0.25 Meilisearch, first load your dump into v0.24.0, create a dump using v0.24.0, and then import this dump into v0.25.0.
* We have added several new error messages, most of which are related to managing API keys.
* `matches` now highlights string and numeric values.
* We added some new telemetry metrics.

Contributors
------------

A big thank you to all our contributors! This project couldn't get very far without your help.

Thanks to [@Mcdostone](https://github.com/Mcdostone) and [@Thearas](https://github.com/Thearas) for your help with Meilisearch, and to [@datamaker](https://github.com/datamaker) and [@Samyak2](https://github.com/Samyak2) for your help with our Tokenizer library.

* * *

If you want more details on updates we didn’t mention here, check out our [release changelog](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.25.0).

What's new in v0.25

In this new version of Meilisearch, we focused on improving our error handler and reviewing our data collection policy—but don't worry, we still managed to squeeze in quite a few quality-of-life updates. We'll go through the most important bits here, but you can check out the [full changelog](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.24.0) on our GitHub release page.

Breaking change: Improved error handling
----------------------------------------

Our revamp of the error handler started with minor changes to field and value names: `errorCode`, `errorType`, and `errorLink` have been shortened to `code`, `type`, and `link` respectively, and the three possible values for `type` have been shortened to `invalid_request`, `internal`, and `auth`.

```JSON
 {
 "message": "authorization header is missing",
 "code": "missing_authorization_header",
 "type": "auth",
 "link": "https://docs.meilisearch.com/errors#missing_authorization_header"
 }
```

In a related change, the update response in case of an error has been altered.

```JSON
{
 "status": "failed",
 "updateId": 1,
 // ...
 "error": {
	 "message": "Document `:documentId` not found.",
 	"code": "document_not_found",
 	"type": "invalid_request",
 	"link": "https://docs.meilisearch.com/errors#document_not_found"
 },
 }
```

**Instead of being located directly in the body, error fields are now located in a nested error object.**

Most importantly, we added new error codes and changed some old ones to make resolving errors faster and more intuitive. [Take a look at our release changelog for a more detailed list of updates](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.24.0).

Breaking change: `attributesToHighlight` now works on numeric values
--------------------------------------------------------------------

When you use the [`attributesToHighlight`](https://www.meilisearch.com/docs/reference/api/search#attributes-to-highlight) search parameter in your query, returned documents will include a `_formatted` object containing matching query terms wrapped in `` tags.

Starting with v0.24, `attributesToHighlight` works with both strings and numeric values. This means numbers inside `_formatted` will be automatically converted to strings:

```sh
curl \
 -X POST 'http://localhost:7700/indexes/games/search' \
 -H 'Content-Type: application/json' \
 --data-binary '{ "q": "101", "attributesToHighlight": ["id, title, description"] }'
```
```JSON
{
 "id": 101,
 "title": " The Wonderful 101: Remastered",
 "description": "Lead an army of 101 Wonderful Ones from around the world in this Unite Action game!",
 "_formatted": {
 "id": "101",
 "title": " The Wonderful 101: Remastered",
 "description": "Lead an army of 101 Wonderful Ones from around the world in this Unite Action game!"
 }
}
```

Telemetry page update
---------------------

By default, we collect usage data from all Meilisearch instances that do not explicitly opt out.  A few releases ago, we [added an article to our documentation website](https://www.meilisearch.com/docs/learn/what_is_meilisearch/telemetry) describing why and how we collect usage data from Meilisearch instances. With v0.24, we have expanded that article to include [an exhaustive list of all the collected data points](https://www.meilisearch.com/docs/learn/what_is_meilisearch/telemetry#exhaustive-list-of-all-collected-data).

As part of this renewed commitment to data transparency, we have also put in place a process for users to request the deletion of their collected data. Simply drop us a message at [privacy@meilisearch.com](/blog/p/49021b33-6bd0-45b6-b3d8-2b962acbaae9/privacy@meilisearch.com)ncluding your instance's UID and we'll remove everything from our records, no questions asked.

Privacy is a difficult subject in tech and there is no shortage of companies who repeatedly violate it out of carelessness or because they stand to profit from unethical data harvesting. We hope to speak more about this in an upcoming blog post, but the important thing to know is that we believe honest and constant communication about these issues is the only way to ensure your data is safe and properly handled. If at any time you feel something looks odd or suspicious, you can use the [privacy@meilisearch.com](/blog/p/49021b33-6bd0-45b6-b3d8-2b962acbaae9/privacy@meilisearch.com)ddress to contact us and hold us accountable.

Other changes
-------------

* Performance improvements related to the \`attribute\` ranking rule
* Maximum attribute size increased from 1000 to 65535 positions
* Fixed erroneous CSV Content-Type error message
* Improved error handling when importing an empty dump
* Fixed indexation performance when using mounted disks

Contributors
------------

Not going to lie, this is our favourite part of the release blog post: thanking everyone who has dedicated their time to improving Meilisearch. You're all wonderful 🤩

This month, we'd like to give a special shout out to [@Jhnbrn90](https://github.com/Jhnbrn90), [@nav1s](https://github.com/nav1s), [@nfsec](https://github.com/nfsec), [@patrickdung](https://github.com/patrickdung), [@sanders41](https://github.com/sanders41), [@CNLHC](https://github.com/CNLHC), [@fumblehool](https://github.com/fumblehool), and [@palfrey](https://github.com/palfrey).

* * *

That's it for this release. If you want more details on updates we didn't have the time to mention here, do check out our [release changelog](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.24.0).

See you next month for v0.25!

What's new in v0.24

This month’s release brings you geosearch and support for CSV and NDJSON payloads! You can read the full [changelog](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.23.1), but let's take a look at some of the main and breaking changes.

New feature: Geosearch
======================

Meilisearch v0.23 supports geosearch. This allows you to filter and sort results based on their location.

To start using this feature, the `_geo` field must be present in your indexed documents and added to either the `filterableAttributes` or `sortableAttributes` list, depending on the type of operation you’d like to perform.

To filter your results so that they are within a certain area, you can use the built-in `_geoRadius` filter rule. Let’s say we're in the center of Muscat and want to view restaurants that are within two kilometers:

```bash
curl  
    -X POST 'http://localhost:7700/indexes/restaurants/search' \
    --data-binary '{ 
		"filter": "_geoRadius(23.5880, 58.3829, 2000)" 
	}'
```

`_geoRadius` establishes a circular area based on a central point and a radius. It requires three parameters: `lat`, `lng`, and `distance_in_meters`—the latitude, longitude, and distance in meters respectively.

You can also sort search results in ascending or descending order based on their distance from a geographic location using `_geoPoint`_._ Let’s say you want to order documents based on how close they are to the Eiffel Tower:

```bash
curl \
    -X POST 'http://localhost:7700/indexes/restaurants/search' \
	-H 'Content-type:application/json'
	--data-binary '{
	  "sort": "_geoPoint(48.8583701,2.2922926):asc" 
	}`
```

The search response will return an extra field called `_geoDistance`. This is the distance in meters computed from the `_geoPoint` and the `_geo` coordinates of the search results.

New feature: support for NDJSON and CSV payloads
================================================

Meilisearch now accepts CSV and NDJSON along with JSON as data sources. You will now need to specify a Content-Type header for HTTP requests through tools like `curl`.

Both CSV and NDJSON offer better indexation performance, consume less RAM and are not as CPU-intensive as JSON. NDJSON is easier to validate and, unlike CSV, can handle nested structures.  

Breaking change: Content-Type headers are now mandatory
=======================================================

Since Meilisearch now supports JSON, NDJSON, and CSV, you need to specify content headers for HTTP requests through tools like `curl`.

*   `Content-Type: application/json` for JSON
*   `Content-Type: application/x-ndjson` for NDJSON
*   `Content-Type: text/csv` for CSV

An example of the JSON header when creating an index:

```bash
curl \
    -X POST 'http://localhost:7700/indexes' \
    -H 'Content-Type: application/json' \
    --data-binary '{
      "uid": "movies",
      "primaryKey": "movie_id"
    }'
```

Contributors
============

A big thank you to all our contributors! We wouldn't have made it here without your support. This month, we want to send a special shout out to [@anirudhRowjee](https://github.com/anirudhRowjee), [@felixonmars](https://github.com/felixonmars), [@happysalada](https://github.com/happysalada), [@kappa-wingman](https://github.com/kappa-wingman), [@k-nasa](https://github.com/k-nasa), and [@shekhirin](https://github.com/shekhirin).

* * *

And that's all folks! Remember to check out the [changelog](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.23.1) for the full release notes and see you next month!

What's new in v0.23

After the full rewrite of the core engine required by v0.21, we are finally back on track with our monthly release schedule. You can [read the full changelog](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.22.0), but let's take a look at some of the new version's main changes.

New feature: sorting at search time
-----------------------------------

Though Meilisearch already allowed developers to influence the sorting process by creating custom ranking rules, this was something done during index configuration and applied to all searches in that index. Allowing users to change sorting order and parameters, also known as sorting at search time, was a cumbersome task that required plenty of workarounds like creating several indexes with slightly different ranking rules.

With the introduction of the `sortableAttributes` index setting, [the `sort` search parameter](https://www.meilisearch.com/docs/reference/api/search#sort) and [the `"sort"` ranking rule](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#5-sort), users can now sort results by document field values in ascending or descending order at search time.

To sort at search time, you must first update the `[sortableAttributes](https://www.meilisearch.com/docs/reference/api/settings#sortable-attributes)` index setting. We currently support attributes containing strings and numbers:

```javascript
$ curl \
	-X POST 'http://localhost:7700/indexes/books/settings/sortable-attributes' \
	--data '[
			"price",
			"author"
	]'

```

Once `sortableAttributes` has been configured, you can use the `sort` search parameter to sort results at search time. `[sort](https://www.meilisearch.com/docs/reference/api/search#sort)` requires a list of attributes given in the `attribute:sorting_order` format:

```javascript
$ curl \
  -X POST 'http://localhost:7700/indexes/books/search' \
  --data '{
    "q": "science fiction",
    "sort": [
      "price:asc"
    ]
  }'

```

[For more information on sorting, take a look at our dedicated guide.](https://www.meilisearch.com/docs/learn/advanced/sorting)

Breaking change: custom ranking rules
-------------------------------------

We have aligned the syntax for custom ranking rules and `sort`:

```javascript
# current syntax
"price:asc"

# old syntax
"asc(price)"

```

Custom ranking rules are still the best way of promoting specific types of documents on every search, like when you want to make sure discounted products always feature in a webshop's top search results.

New indexer
-----------

Our second major change on v0.22 should actually be quite invisible. We have refactored our indexer, solving many long-standing issues regarding crashes due to memory overconsumption.

**The transition to the new indexer should be completely seamless and require no change on your side.** Barring any hidden bugs, the only thing you should notice is that indexation is faster and that crashes have become much, much more rare. If you want to know more about indexation and performance, we have added [a new article to our documentation](https://www.meilisearch.com/docs/learn/advanced/indexing#improving-indexing-performance).

In case you still experience issues during indexation, let us know (and [check the FAQ for some of our recommended fixes](https://www.meilisearch.com/docs/faq))!

Other changes
-------------

*   We no longer use Sentry to collect usage data from instances
*   Custom ranking rules now work on strings as well as numbers
*   The default installation script available at [https://install.meilisearch.com](https://install.meilisearch.com) now works with Windows machines using [Cygwin](https://cygwin.com/) or an equivalent environment

Contributors
------------

We wouldn't have gotten here without the support from our community. You're really, truly great and we can't thank you enough for all the effort, suggestions, comments and time you dedicate to Meilisearch. This month, we want to send a special thank you to [@sanders41](https://github.com/sanders41), [@bb,](https://github.com/bb) and [@shekirin](https://github.com/shekhirin).

* * *

And that's all for v0.22! Remember to check out the [changelog for the full release](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.22.0) notes and see you again next month!

What's new in v0.22

Today marks the launch of Meilisearch v0.21, our first new release since March 2021. As you might expect, this one packs a whole lot of changes to our search engine, both big and small. You can check our [exhaustive changelog outlining all breaking changes and download the binaries here](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.21.0), but in this post we wanted to take a closer look at some of the most important updates.

Search performance
------------------

First, one of the best news we could give you: v0.21 is the fastest Meilisearch release (so far). How fast? Our tests using a 1GB dataset with 115 million documents indicate that complex queries return results up to ten times faster than v0.20; filtering those results, previously a slow operation, takes a maximum of 50ms.

Refactor
--------

Clémentine already wrote about [why we chose to refactor](/blog/refactoring-meilisearch/)undamental parts of our core engine, but in case you missed it: it became increasingly hard for us to keep our code easy to read for external contributors and to add new features without negatively impacting performance.

We are very proud of what we accomplished in these months, not only because we managed to greatly improve Meilisearch's performance, but also because we know our search engine has a new, rock-solid foundation.

Facets and filters
------------------

Some of the biggest visible (and breaking) changes in v0.21 concern facets and filters.

First of all, [the search parameters `filters` and `facetFilters` have been replaced by `filter`](https://docs.meilisearch.com/reference/features/search_parameters.html#filter). This new parameter allows you to refine search results based on document attributes whose values are either numbers or strings.

```javascript
$ curl 'http://localhost:7700/indexes/movies/search' \
--data '{ "q": "thriller", "filter": ["genres = Horror", "genres = Mystery"]}'
```

In this example, we create a filter that only returns movies classified as both horror and mystery.

To use an attribute with the `filter` search parameter, you must first add it to [the new `filterableAttributes` index setting](https://docs.meilisearch.com/reference/api/filterable_attributes.html). `filterableAttributes` works exactly like and replaces the `attributesForFaceting` index setting.

```javascript
$ curl \
 -X POST 'http://localhost:7700/indexes/movies/settings' \
 --data '{
 "filterableAttributes": [
 "director",
 "genres"
 ]
 }'

```

Meilisearch no longer distinguishes between facets and filters: you can use the new `filter` search parameter and the `filterableAttributes` index setting to create [faceted search interfaces](https://docs.meilisearch.com/reference/features/filtering_and_faceted_search.html).

Phrase search
-------------

Meilisearch v0.21 supports [phrase searches](https://docs.meilisearch.com/reference/features/configuration.html#disable-analytics). If you enclose one or more search terms in double quotes (for example, `“african-american poet”` or `“John Waters”`), Meilisearch will only return documents that contain those terms exactly as they were written and in the order they were given. Phrase searches are a great way to make a query more precise and come in handy when you know your search should only get one result, such as when looking for a book using its ISBN number.

```javascript
$ curl 'http://localhost:7700/indexes/movies/search' \
 --data '{ "q": "\"John\" Waters" }'
```

Web interface
-------------

In case you didn't know, [Meilisearch automatically creates a web interface](https://docs.meilisearch.com/reference/features/web_interface.html) when you launch an instance in development mode so you can test our search engine right away. We have updated this interface so it's even easier to use (and nicer to look at)!

![](https://unable-actionable-car.media.strapiapp.com/trumen_fast_eba051545b.gif)

New telemetry page and simpler analytics settings
-------------------------------------------------

By default, Meilisearch collects anonymized data from all instances using our search engine. We have made it easier to [disable this behavior](https://docs.meilisearch.com/reference/features/configuration.html#disable-analytics) and [added a new page to our docs explaining exactly what data we collect and how we use it](https://docs.meilisearch.com/learn/what_is_meilisearch/telemetry.html). Privacy is an important issue for us, so please feel free to share any comments, concerns or suggestions you may have.

```javascript
$ ./meilisearch --no-analytics=true
```

Highlight matches in array and object fields
--------------------------------------------

Meilisearch v0.20 allowed you to [highlight matched search terms in string fields](https://docs.meilisearch.com/reference/features/search_parameters.html#attributes-to-highlight) so users could see where exactly a term was found within a document. In v0.21, we have extended this functionality to work with arrays of strings and nested objects. Like before, you can find the highlighted matches inside the `_formatted` object of each returned document.

```javascript
$ curl 'http://localhost:7700/indexes/movies/search' \
 --data '{ 
		"q": "adventure", 
 	"attributesToHighlight": ["title", "genres"] 
	}'
```

```javascript
{
 "id": "50393",
 "title": "The Adventures of Huck Finn",
 "overview": "Climb aboard for an extraordinary version of Mark Twain's sweeping adventure when Walt Disney presents The Adventures of Huck Finn, starring Elijah Wood.",
 "release_date": 733712400,
 "genres": ["Adventure","Family"]
 "_formatted": {
 "id": "50393",
 "title": "The Adventures of Huck Finn",
 "overview": "Climb aboard for an extraordinary version of Mark Twain's sweeping adventure when Walt Disney presents The Adventures of Huck Finn, starring Elijah Wood.",
 "release_date": 733712400,
 "genres": ["Adventure","Family"]
 }
}
```

Title and genre highlighted matches are available inside `_formatted`.

Other changes
-------------

* You might notice minor performance improvements in the indexer: this is a work-in-progress and we are eager to have your feedback on it
* The `wordsPosition` [ranking rule](https://docs.meilisearch.com/learn/core_concepts/relevancy.html#ranking-rules) has been merged into `attribute`
* `max-mdb-size` has been renamed to `[max-index-size](https://docs.meilisearch.com/reference/features/configuration.html#max-index-size)`
* It is now possible to specify `[max-index-size](https://docs.meilisearch.com/reference/features/configuration.html#max-index-size)` and `[http-payload-limit-size](https://docs.meilisearch.com/reference/features/configuration.html#payload-limit-size)` using human-readable units such as `Kb` or `Gb`
* We have removed the limit of 200 indexes per instance

Bug fixes
---------

* Windows instances no longer immediately occupy the maximum index size on launch
* The [ranking rule](https://docs.meilisearch.com/learn/core_concepts/relevancy.html#ranking-rules) `Words` now works as expected
* [Dumps](https://docs.meilisearch.com/reference/features/dumps.html) now export all document fields correctly, regardless of whether they are present in `displayedAttributes`

Contributors
------------

A lot of this was made possible thanks to our community: we are honestly humbled by your generosity. Meilisearch is a better search engine because of you.

We want to send a special shout out to [@sanders41](https://github.com/sanders41) and [@bb](https://github.com/bb): your bug reports and feedback were super helpful!

Finally, a big thank you to [@shekhirin](https://github.com/shekhirin) for all the help with the new search engine implementation.

* * *

That's about it for this release! To have a full overview of all changes, breaking and otherwise, you can [check out our changelog](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.21.0).

What’s new in v0.21

Today we are releasing the first release candidate (RC) of Meilisearch v0.21. This is the initial step towards the next version of Meilisearch, and a pretty important one given that it will be our first major release since March.

A big part of v0.21 will be relatively invisible to most end-users: our core team has been hard at work refactoring Meilisearch’s engine, something that is as time-consuming as it is crucial for the long-term health of our codebase. In this post, I want to go over why we decided to refactor Meilisearch, how that impacted our product, and where we’ll go from here.

We won’t be discussing everything that’s changed between v0.20 and v0.21—that’s for another blog post—but we will touch on several of the new features we are most excited about.

Why was refactoring needed?
===========================

We first realized that something was wrong when it became increasingly difficult for our core team to add new features without impacting performance. We could see our code was becoming more difficult to maintain, which is something particularly concerning for an open-source project: our community is our greatest ally and keeping code clean, readable, and pleasant-to-use is crucial if we want to keep our allies happy. We were clearly reaching the limits of what the previous codebase could offer us in terms of performance and ease-of-use, so we decided it was time for a change.

How did we do?
==============

We have a lot of good news—but, as with any major software release, there were some goals we weren’t able to attain.

### Search relevancy

The v0.21.0 will provide new ways to make your search more relevant:

*   You can now use numeric attributes as facets—think things like the price of a product or the release date of a book
*   This was actually requested by our users: the equivalent of Algolia’s `words` criterion [is now available](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#1-words) in Meilisearch!
*   Phrase search: users can now force the engine to only return exact matches by enclosing query terms in double quotes (`"`)
*   Highlighting is now available for array and object values

### Search performance

Searching through huge datasets with Meilisearch has never been so fast! With a 1GB dataset containing around 115 million documents:

*   Queries that return many results are more than ten times faster: searching for a single letter takes less than 50ms in v0.21, compared with 2 seconds in v0.20
*   Similar speed gains when filtering on numbers: most searches take less than 50ms in v0.21

### Indexation

TLDR; indexation is now faster in most cases, but there are exceptions. Lots of room for improvement!

First, the good bits: indexation is globally faster with the refactored search engine.

Unfortunately, we noticed a loss of performance when parsing datasets with lengthy description values (think a list of blog posts or scientific articles with excerpts and abstracts) in v0.21. This is not a complete surprise: the new search engine requires more calculations, which means indexation might take more time. This is not ideal, but we would rather release something that’s not completely optimized than delay v0.21 and all the new features it will introduce. Rest assured, we are doing our best to improve indexation performance.

Another bit of bad news: while we have made significant steps in the right direction and reduced memory consumption across the board, the issue of Meilisearch sometimes crashing due to excessive memory usage has still not been fixed. This is another priority for us going forward.

### **Are there any breaking changes**?

There are indeed! We did our best to avoid breaking changes, but that’s not always possible when we want to keep improving things and making our API easier to use. The two biggest changes related to v.021’s refactoring are:

*   The parameters `filters` and `facetFilter` have been replaced by `filter`. Having two parameters doing almost the same thing added unnecessary complexity when filtering query results
*   Several naming changes that we believe make using Meilisearch more intuitive. None of these have any impact on the end-user experience. A full list will be available when we officially release v0.21

What’s next?
============

Once v0.21 is officially released some time within the next three months, we will return to our monthly release cycle. We are particularly looking forward to working on `sortBy` and geographical search—two features that are highly requested by our users. We hear you loud and clear!

If you are curious about Meilisearch’s upcoming version, feel free to test it and give us as much feedback as possible—we depend on you to catch as many remaining bugs as possible. You can [download the release candidate on GitHub](https://github.com/meilisearch/MeiliSearch/releases/tag/v0.21.0rc0).

Also, a word of advice: we do not recommend using this RC in production. We are confident v0.21 RC1 is looking pretty good, but there are still issues that we need to work out before it is ready for the spotlight.

Thank you again, and see you soon for the official v0.21.0 release!

Meilisearch latest news and company updates