What's new in v1.4?

Let's take a look at some of the most significant changes in Meilisearch's latest update. We’ll go over the main changes in this article, but you can also view the full changelog on GitHub.

New feature: custom text separators

---

To make string data searchable, Meilisearch relies on separators, as they serve to divide a string into tokens or words. Examples of separators include whitespaces, full-stops, or number signs (#). They play a crucial role in helping Meilisearch segment text effectively and enhance search relevancy.

Meilisearch comes with a predefined list of separators. However, these separators do not suit all use cases. For example, in a hashtag search, the number sign should not be considered a separator, but part of the word.

Starting with v1.4, Meilisearch allows you to customize the separators list to suit your specific needs.

Using an SDK or the API

Adding characters to the separators list

To include a character in the separators list, update your index’s Separators Tokens settings:

client.index('myIndex').updateSeparatorTokens(['§'])

This request adds the encoded version of the section sign character (`§`) to the separator tokens list.

Removing characters from the separators list

To remove one or more characters from the separators list, update your index’s Non Separator Tokens settings:

client.index('myIndex').updateNonSeparatorTokens(['@', '#', '&'])

Using Meilisearch Cloud

On Meilisearch Cloud, you can configure Separator Tokens from your index’s settings. Both fields accept JSON format. To include separators, add them to the 'separators' field. To exclude certain characters as separators, list them under 'non-separators'.

New feature: custom dictionary

Now, you can add a custom word dictionary to improve Meilisearch's segmentation of specific words. This is particularly useful when working with domain-specific terminology like “Node.js” or proper nouns like “E. E. Cummings”.

To add words to the dictionary, update your index’s Dictionary settings:

client.index('myIndex').updateDictionary(['e. e.', 'e.e.', 'e e'])

On Meilisearch Cloud, you can configure a Custom Dictionary by providing a JSON array in your index’s settings. You can also upload a JSON file as dictionary.

Using custom dictionary with stop words and synonyms

The introduction of a custom dictionary acts as a powerful adjunct to existing features like stopWords and synonyms. Together, they synergize to improve the relevancy of the search results.

Let’s consider a literary database where an author's name may appear in various forms or abbreviations. This leads to fragmented search results, making it challenging for users to find works by that specific author. For example, take the different ways one might search for works by E. E. Cummings. Using the custom dictionary feature along synonyms can standardize these name variations, thereby enhancing the relevancy of search results.

Using synonyms and custom dictionary together, here’s an example of index settings that address this scenario:

{
"dictionary": ["E. E.", "E.E.", "E E"],
"synonyms": {
    "E. E.": [ "E.E.", "E E", "Edward Estlin"],
    "E.E.": ["E. E.", "E E", "Edward Estlin"],
    "E E": ["E. E.", "E.E.", "Edward Estlin"],
    "Edward Estlin": ["E. E.", "E.E.", "E E"]
}

Breaking bug fix: improved filtering with backslashes

In v1.4, we've addressed a longstanding bug that users faced when using backslashes (\) at the end of the `filter` search parameter expression.

Let’s consider the following documents:

[
  {
    "id": 1,
    "path": "my\\test\\path"
  },
  {
    "id": 2,
    "path": "my\\test\\path\\"
  }
]

Note: The double backslashes in the example are for JSON escaping.

Before v1.4.0, trying to filter on the second document using either of the filters path = "my\\test\\path\\" or path = "my\\test\\path\\\" would lead to errors.

Now, you can use any filter expression with backslashes. Just make sure to escape each \ character in your filter.

Using our example, to filter on the second document successfully, the filter should be written as: path = "my\\\\test\\\\path\\\\".

⚠️ Warning: If you're upgrading from v1.3.X or earlier and have previously used backslashes in filters, be aware that in v1.4.0, the correct filter for the first document should be path = "my\\\\test\\\\path".

Two layers of escaping are applied: : first, escape for JSON, then for the Meilisearch filter. Meilisearch turns \\\\ back to \\, and JSON parsing will result in a single \.

Contributors

We are really grateful for all the community members who participated in this release. We would like to thank: @dogukanakkaya, @JannisK89, and @vivek-26 for their help with Meilisearch. We want to send a special shout out to mmachatschek for his help and involvement with the backlash bug.

Conclusion

And that’s it for v1.4! Remember to check the changelog for the full release notes, and see you next time!

You can stay in the loop by subscribing to our newsletter. To learn more about Meilisearch's future and help shape it, take a look at our roadmap and come participate in our Product Discussions.

For anything else, join our developers community on Discord.