[Nikita](https://nikita.js.org/current/guide/tutorial/) is a Node.js toolkit for automating deployment workflows. Even though we think it is great, we noticed Nikita's documentation website doesn’t really give users a way to quickly search and find the information they’re looking for. Nikita's content is open source and the documentation website is built with Gatsby, though, so we decided to take the opportunity to show you how to add a nice search interface to a Gatsby-based project.

In this tutorial we will install Meilisearch together with a few other dependencies. Then, we will clone Nikita’s Gatsby repository and index Nikita’s docs with Meilisearch. Finally, we’ll add a search UI to our local version of Nikita’s docs.

You can check the finalized code, including all content ready to be run in a server, [at this github repo](https://github.com/meilisearch/demos/tree/main/src/gatsby-plugin-guide).

Setup
=====

Before we begin, let’s take a quick look at everything you need to follow this tutorial.

Introducing our Tools
=====================

Gatsby
------

[Gatsby](https://www.gatsbyjs.com/) is a React-based open-source framework used for Static Site Generation (SSG). It combines the functionality of React, GraphQL and Webpack and provides great performance, scalability, and security right out of the box.

Requirements
============

To be able to follow this tutorial, you'll need the following:

*   An open [terminal or command prompt](https://www.ionos.com/help/email/troubleshooting-mail-basicmail-business/access-the-command-prompt-or-terminal)
*   [Node.js](https://nodejs.org/en/download/) {">= 14.15.X and <= 16.X"}: this makes it possible to run JS outside a browser
    *   You can check your active version with the command `node --version`
    *   If your version of Node is outside this range, we recommend that you [install nvm](https://github.com/nvm-sh/nvm/blob/master/README.md#about) and use it to access Node 14
*   Yarn >= 1.22 : Yarn is a package manager that doubles down as project manager
*   [Docker](https://www.docker.com/): Docker will help us create a working development environment with relatively little effort

Steps
=====

1.  Start Meilisearch
2.  Setup the Gatsby project
3.  Add the Meilisearch plugin
4.  Add the content of the Gatsby app to Meilisearch
5.  Build the frontend

Start Meilisearch
=================

There are multiple ways to [download and run a Meilisearch instance](https://www.meilisearch.com/docs/learn/getting_started/installation#download-and-launch). For this guide, we'll be using Docker. Want to avoid a local installation? Try our [Meilisearch cloud](https://cloud.meilisearch.com/) with a free 14-day trial!

After installing Docker in your machine, fetch and run the latest Meilisearch container image. To do that, open your terminal and use the following commands:

```bash
docker pull getmeili/meilisearch:latest 

docker run -it --rm -p 7700:7700 getmeili/meilisearch:latest ./meilisearch --master-key=masterKey

```

Meilisearch will start running on [http://localhost:7700](http://localhost:7700/). If everything went well, you should see something like this in your terminal window:

![Screenshot-2023-04-13-at-14.48.31](https://unable-actionable-car.media.strapiapp.com/Screenshot_2023_04_13_at_14_48_31_d715bb8f41.png)

Setup the Gatsby project
========================

With Meilisearch ready, it’s time to download Nikita’s documentation.

In another terminal window, navigate to your working directory, then clone the Nikita project that we will be using for this guide:

```bash
git clone https://github.com/adaltas/node-nikita.git

```

You can find the content of the Gatsby app in the markdown files inside the `docs/content` folder. This is the data Gatsby uses to generate Nikita’s documentation.

| _Directory structure for the node-nikita project_ |

Add the Meilisearch plugin and launch Gatsby
============================================

Alright: both Meilisearch and Gatsby are installed. We will start by installing all the project dependencies. Run the following command in your terminal:

```bash
cd node-nikita/docs/website
yarn

```

Once that is done, we will install the official Meilisearch plugin for Gatsby with Yarn:

```bash
yarn add gatsby-plugin-meilisearch

```

Great! We’re ready to launch Gatsby. To do that, just run the following command on your console:

```bash
yarn develop

```

This is how the UI of the Nikita docs website looks like:  
![nikita1](https://user-images.githubusercontent.com/19529592/161043477-afd719ab-fe45-4f1a-ae0b-5f27e4983df6.jpg)

Add the content of the Gatsby app to Meilisearch
================================================

Now that we have installed the plugin and have both the Gatsby app and the Meilisearch instance running, things start to get interesting! Let's configure `gatsby-meilisearch-plugin` to make the content searchable.

The main code for the docs website lies in the directory `docs/website`.

**Configure the plugin**
------------------------

The first thing we need to do is add your Meilisearch [credentials](https://www.meilisearch.com/docs/learn/getting_started/quick_start#securing-meilisearch) to the Meilisearch Gatsby plugin. Open `docs/website/gatsby-config.js` and add this bit of code at the bottom of the `plugins` array:

```jsx
{
    resolve: 'gatsby-plugin-meilisearch',
    options: {
      host: 'http://localhost:7700', // your host URL goes here
      apiKey: 'masterKey',   // your API key goes here       
      indexes: [],
    },
  },

```

So far, our configuration tells the plugin where to find our Meilisearch instance and gives it the credentials to read the content inside Gatsby. Now we need to define what data we want to add to Meilisearch and how. This happens in the [indexes field](https://github.com/meilisearch/gatsby-plugin-meilisearch#-indexes). This field can be considered the heart of the Gatsby Meilisearch plugin.

The `indexes` field is an array composed of multiple index objects. Each index object must contain the following fields: `indexUid`, `query`, and `transformer`. Let's examine these fields.

### `indexUid`

This field contains the name of a Meilisearch index. For this guide, we will use `nikita-api-docs`:

```jsx
indexUid: 'nikita-api-docs'

```

### `query`

This field contains the GraphQL query the plugin will use to fetch content from Nikita’s documentation. Gatsby comes with inbuilt support for GraphQL—it even includes a bundled GraphQL tool, accessible on ([http://localhost:8000/\_\_\_graphql](http://localhost:8000/___graphql)). You can read more about GraphQL on the project’s official [documentation.](https://graphql.org/learn/)

Our query looks like this.

```jsx
query: `query MyQuery {
    allMdx {
      edges {
        node {
          frontmatter {
            title
            navtitle
            description
          }
          slug
          rawBody
          id
        }
      }
    }
  }`,

```

In this query, we first say we are only interested in the markdown files by wrapping our query with the `allMdx` statement. We can do that thanks to two very useful tools: `gatsby-plugin-mdx` and `gatsby-source-filesystem`.

Then, we specify which document fields we want to include in our index: `title`, `navtitle`, `description`, `slug`, `id`, and, finally, `rawBody`—that is, the document’s raw markdown content.

### `transformer`

This is the last configuration field. It gets the data we fetched from Gatsby with our `query` and converts it into a format that Meilisearch will understand.

In our case, the data will look a bit like this:

```jsx
data = {
    allMdx: {
        edges: [
          {
            node: {
              frontmatter: {
                title: "Introduction",
                navtitle: "intro",
              },
              body: "Introduction to the Nikita.js",
              slug: "/introduction",
              id: "1",
            },
          },
          {
            node: {
              frontmatter: {
                title: "Architecture",
                navtitle: "architecture",
              },
              body: "Architechture of Nikita.js",
              slug: "/architecture",
              id: "2",
            },
          },
        ],
    },
};

```

This data looks great, but is not in a format that Meilisearch can easily understand. We can change that by adding a parser function to `transformer`:

```jsx
transformer: (data) => {
    data.allMdx.edges.map(({ node }) => {
      // Node property has been destructured here
      return {
        id: node.id,
        lvl0: node.frontmatter.title,
        lvl1: node.frontmatter.navtitle,
        content: node.body,
        anchor: node.slug,
      };
    });
}


```

With that, `gatsby-plugin-meilisearch` will get the raw data we extract with our query and turn the whole object into an array.

```jsx
 // It will return a list of transformed structured object
  
[
    {
      id: "1",
      lvl0: "Introduction",
      lvl1: "introduction",
      content: "Introduction to the Nikita.js",
      anchor: "/introduction"
    },
    {
      id: "2",
      lvl0: "Architecture",
      lvl1: "architecture",
      content: "Architechture of Nikita.js",
      anchor: "/architecture"
    }
]

```

### Putting it all together and finalizing plugin configuration

If you’ve been following along, the `gatsby-plugin-meilisearch`configuration at the end of your `docs/website/gatsby-config.js` should look something like this now:

```jsx
{
  resolve: 'gatsby-plugin-meilisearch',
  options: {
    host: 'http://localhost:7700',
    apiKey: 'masterKey',
    batchSize: 1,
    indexes: [
      {
        indexUid: 'nikita-api-docs',
        settings: {
          searchableAttributes: ['lvl0', 'lvl1', 'lvl2', 'content'],
        },
        transformer: (data) =>
          data.allMdx.edges
            .filter(
              ({ node }) =>
                node.slug.search('guide') !== -1 ||
                node.slug.search('project') !== -1 ||
                node.slug.search('api') !== -1
            )
            .map(({ node }) => {
              // Have to update for versioning
              const currentVersion =
                node.slug.substring(0, 8).search('project') === -1
                  ? '/current'
                  : ''

              return {
                id: node.id,
                lvl0:
                  node.frontmatter.navtitle || node.frontmatter.title || '',
                lvl1:
                  node.frontmatter.title || node.frontmatter.navtitle || '',
                lvl2: node.frontmatter.description || '',
                content: node.rawBody,
                url: `${currentVersion}/${node.slug}`,
              }
            }),
        query: `query MyQuery {
            allMdx {
              edges {
                node {
                  frontmatter {
                    title
                    navtitle
                    description
                  }
                  slug
                  rawBody
                  id
                }
              }
            }
          }`,
      },
    ],
  },
},

```

We began `gatsby-plugin-meilisearch` configuration by adding basic data and credentials to `docs/website/gatsby-config.js.`

We continued configuration by specifying the content we want to search should be added to the `nikita-api-docs` index, used a GraphQL query to select content for indexation, and finally used a `transformer` function to format the data for indexation.

**Build the project**
---------------------

The `gatsby-plugin-meilisearch` will fetch and send data for indexation to Meilisearch during the build process. To get that started, run the following command:

```bash
yarn build

```

You should see a message in the terminal once your content is being indexed:

```bash
success gatsby-plugin-meilisearch - 0.920s - Documents are send to Meilisearch, track the indexing progress using the tasks uids.

```

You can verify this by going to [http://localhost:7700](http://localhost:7700), entering your API key, and checking that Nikita’s documentation has been added to Meilisearch.

Build the frontend
==================

Now that the data is indexed, let’s build the user interface and create a great search experience for our end-users.

For this example, we will use [docs-searchbar.js](https://github.com/meilisearch/docs-searchbar.js). It is a front-end SDK for Meilisearch that provides an easy way of integrating a search bar into our documentation site.

Add the search bar component
----------------------------

Let’s start by installing docs-searchbar.js in the frontend directory of the project.

```markdown
# With Yarn
yarn add docs-searchbar.js

```

With that done, we can import the docs-searchbar module into `website/src/components/shared/AppBar.js` by adding this to the top of the file:

```jsx
import 'docs-searchbar.js/dist/cdn/docs-searchbar.css'

```

Next, we need to add a `useEffect` hook to add the docsSearchBar function to `AppBar.js`. Add this below the other `useEffect` hook:

```jsx
useEffect(() => {
    if(window !== undefined){
      const docsSearchBar = require('docs-searchbar.js').default

      docsSearchBar({
        hostUrl: 'http://localhost:7700',
        apiKey: 'masterKey',
        indexUid: 'nikita-api-docs',
        inputSelector: '#search-bar-input',
        meilisearchOptions: {
          limit: 5,
        },
        enhancedSearchInput: true,
      })
    }
  }, [])

```

The `docsSearchBar` function comes in with a number of different parameters:

*   `hostUrl` and `apiKey` allows the search bar to access your Meilisearch instance.
*   `indexUid` tells the search bar which index it should search through.
*   `inputSelector`is a selector matching the HTML element where our users will type their queries. In our case, that’s `#search-bar-input`. Don’t worry, we’ll add this element to Gatsby in a moment.
*   Finally, `enhancedSearchInput` tells `docs-searchbar` to apply a theme to the search box, improving its appearance and making it a bit more user-friendly.

All that’s left is adding our search element in `website/src/components/shared/AppBar.js`. Remember to use the same `id` we configured as the `inputSelector`. Add the element after `</Link>` tag:

```jsx
<input type="search" id="search-bar-input" />

```

And that’s it! We’re done!

**Test the implementation**
---------------------------

Ready to see all this hard work in action? On your terminal, re-run the following command:

```markdown
yarn develop

```

And then use your browser to access [`http://localhost:8000`](http://localhost:8000/). You should see your local copy of NikitaJS’s docs with a brand new search bar:

![nikitalfinall](https://user-images.githubusercontent.com/19529592/161043686-a34453b0-e155-4f03-8ba2-2a9225190757.jpg)

Conclusion
==========

We hope this article was a pleasant introduction to the new [Meilisearch plugin for Gatsby](https://github.com/meilisearch/gatsby-plugin-meilisearch)!

If you have any questions, please join us [on Discord](https://discord.gg/meilisearch). For more information on Meilisearch, check out our [Github repository](https://github.com/meilisearch/meilisearch/) and our official [documentation](https://docs.meilisearch.com/).

Add a search bar to your Gatsby project using Meilisearch

If you haven't heard it yet, [Meilisearch v0.22](__GHOST_URL__/whats-new-in-v0-22/) is out, and it comes with a highly requested new feature: sorting at search time.

Search-time sorting is when results are sorted according to parameters decided at query time. By default, Meilisearch orders results according to their [relevancy](https://www.meilisearch.com/docs/learn/core_concepts/relevancy), but you can configure it to let users decide at search time what results they want to see first.

Let me show you how powerful this feature is. If I wanted to search for a MacBook and sort the results by ascending price, I would send the following query:

```json
curl \
  -X POST 'http://localhost:7700/indexes/computers/search' \
  --data '{
    "q": "MacBook",
    "sort": [
        "price:asc"
    ]
}'

```

I could do the opposite and search by descending price. I would just need to replace `asc` by `desc` in my request.

The old way
-----------

Before Meilisearch v0.22, you could only sort search results using the [custom ranking rules](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#custom-rules).

Thanks to the custom ranking rules, we could have our search results sorted by the numeric attribute of our choice. However, the sorting was not at search time, the sorting order was decided in our index settings.

To change the sort order of the search results in the user interface, we had to duplicate the index and set different custom ranking rules for each index.

That's what I did for our [MoMA demo](https://moma.meilisearch.com/) a few months ago. For those who missed it, the MoMA demo is a Meilisearch demo created with the artworks dataset of the [Museum of Modern Art](https://www.moma.org/) available in their [GitHub repository](https://github.com/MuseumofModernArt/collection). You can learn more about the demo in [this blog post](__GHOST_URL__/demo-moma/).

I wanted the users to be able to sort the artworks by date, so I had to create three indexes:

\-  `artWorks` with the default built-in ranking rules  

\- `artWorksAsc` with a custom ranking rule for ascending sort

\- `artWorksDesc` with a custom ranking rule for descending sort

```javascript
// Get or create indexes
const artWorksIndex = await client.getOrCreateIndex('artWorks', { primaryKey: 'ObjectID' })
const artWorksAscIndex = await client.getOrCreateIndex('artWorksAsc', { primaryKey: 'ObjectID' })
const artWorksDescIndex = await client.getOrCreateIndex('artWorksDesc', { primaryKey: 'ObjectID' })

const defaultRankingRules = [
  'typo',
  'words',
  'proximity',
  'attribute',
  'exactness'
] 

const rankingRulesAsc = [
  'typo',
  'words',
  'proximity',
  'attribute',
  'exactness',
  'asc(DateToSortBy)' // ascending sort
]
const rankingRulesDesc = [
  'typo',
  'words',
  'proximity',
  'attribute',
  'exactness',
  'desc(DateToSortBy)' // descending sort
]

```

The duplication, or triplication, gave the impression of sorting at search time as Meilisearch is blazingly fast.

At that time, sorting was on our [roadmap](https://roadmap.meilisearch.com/), in the Under Consideration tab, and it was one of the most voted features (86 votes).

> We have a [public roadmap](https://roadmap.meilisearch.com/) where you can submit feature or integration ideas and vote for the existing ones.

With the new v0.22 sort feature
-------------------------------

Today, it's a reality. And I've updated the demo to integrate this feature. Let's go through what has changed.

### Behind the scenes

First of all, I have added the following line to the settings:

```javascript
sortableAttributes: ['DateToSortBy']

```

This addition is necessary because we need to inform Meilisearch of the attributes we want to use for sorting.

We only need one index (not three) and the default ranking rules. This has considerably reduced the back-end code. You can change the position of the `sort` rule to tune the relevancy of the search results according to your needs. By default, the `sort` rule in 5th position of the ranking rules to promote results relevant to the user's search.

You can learn more about the ranking rules and how they affect relevancy [here](https://www.meilisearch.com/docs/learn/core_concepts/relevancy).

And if you are curious about why the `sort` rule is in 5th position by default, you can check [this GitHub issue](https://github.com/meilisearch/MeiliSearch/issues/1680#issuecomment-915789791) where our Product Manager explains this choice.

With these modifications, the big block of code above has been reduced to the following line:

```javascript
// Get or create the index
const index = await client.getOrCreateIndex('artWorks', { primaryKey: 'ObjectID' })

```

Easy, right? But, what about the front end?

### In the spotlight

For this demo I used [Vue InstantSearch](https://github.com/algolia/vue-instantsearch/), combined with [Instant Meilisearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch). This connects the Meilisearch instance with the open-source [InstantSearch](https://github.com/algolia/instantsearch.js) front-end tools allowing us to customize the search environment effortlessly.

The previous code looked like this:

```javascript
<ais-sort-by
:items="[
  { value: 'artWorks', label: 'Featured' },
  { value: 'artWorksAsc', label: 'Date asc.' },
  { value: 'artWorksDesc', label: 'Date desc.' }                    
]"
:class-names="{
    'ais-SortBy': 'MyCustomSortBy'
}"
/>

```

I just needed to transform it into this:

```javascript
<ais-sort-by
:items="[
  { value: 'artWorks', label: 'Featured' },
  { value: 'artWorks:DateToSortBy:asc', label: 'Date asc.' },
  { value: 'artWorks:DateToSortBy:desc', label: 'Date desc.' }                    
]"
:class-names="{
  'ais-SortBy': 'MyCustomSortBy'
}"
/>

```

As you can see, instead of using 3 different indexes (`artWorks`, `artWorksAsc`, and `artworksDesc`), we only need to append the attribute's name to the `artWorks` index followed by the desired sort order, `asc` or `desc`.

```javascript
'artWorks:DateToSortBy:asc'
'artWorks:DateToSortBy:desc'

```

And this is it. Smooth and simple. You can test it [here](https://moma.meilisearch.com/).

![](https://unable-actionable-car.media.strapiapp.com/sortby_a3723cbe47.gif)

Web interface with the query "Magritte" sorting by ascending and then descending date gif

The end-user may not notice the difference, but resource-wise it's much more efficient.

Another great advantage of v0.22 is the new indexer. It's way faster than before, so if you're still using an older version of Meilisearch, I strongly encourage you to [upgrade it](https://www.meilisearch.com/docs/learn/update_and_migration/updating).

The demo source code is available on [GitHub](https://github.com/meilisearch/demos/tree/main/src/MoMA). I invite you to play with it, you can add other attributes to sort by, as long as they are strings or numeric values. For a more in-depth explanation of sorting, take a look at the [dedicated section](https://www.meilisearch.com/docs/learn/getting_started/filtering_and_sorting#sorting) of the documentation.

* * *

_Picture by [Héctor J. Rivas](https://unsplash.com/@hjrc33) on [Unsplash](https://unsplash.com/)_

The art of sorting

<Callout>
  This guide is for Strapi v3. If you are using Strapi v4, check [this post](__GHOST_URL__/strapi-v4-plugin-meilisearch/).
</Callout>

As the web evolves, more and more tools are available to take care of complex jobs for us. Why create a CRUD API from scratch when you can generate one in a few clicks? Why struggle with difficult search operations and queries, when simple and efficient solutions exist?

Maybe we're intimidated by new technologies, or concerned about investing time in tools that don't pan out. Or, more likely, _we may not even be aware that a simple solution exists._

![](https://unable-actionable-car.media.strapiapp.com/truemen_d2b99b9b0a.gif)

Meilisearch in action

This tutorial will show you, in the fewest steps possible, how to use Strapi and Meilisearch to create a search-based web-app. First, let's get acquainted with our tools!

Introducing our Tools
---------------------

### Strapi

[Strapi](https://strapi.io/) is a headless CMS. It provides a quick way to create a back-end where we can add and store data. The data is then available through a REST API without having to code anything or configure our database.

### Meilisearch

[Meilisearch](https://www.meilisearch.com/docs/learn/what_is_meilisearch/overview) is a fast, open-source search engine that's easy to use and deploy. The aim of Meilisearch is to create a relevant search experience out of the box in very few steps and with no configuration needed.

Requirements
------------

To be able to follow this tutorial, you'll need the following:

*   An open [terminal or command prompt](https://www.ionos.com/help/email/troubleshooting-mail-basicmail-business/access-the-command-prompt-or-terminal)
*   [Node.js](https://nodejs.org/en/download/) {">= 10.16 and <= 14"}: this makes it possible to run JS outside a browser
    *   You can check the active version with the command `node --version`
    *   If your version of Node is outside this range, we recommend that you [install nvm](https://github.com/nvm-sh/nvm/blob/master/README.md#about) and use it to access Node 14
*   NPM and NPX >= 6.x (installed with Node.js): package managers that help you access and use JS libraries

Steps:
------

1.  [Create a working directory](#create-a-directory-for-our-app)
2.  [Create Strapi Back End](#create-a-back-end-using-strapi)
3.  [Start Meilisearch](#launch-and-start-meilisearch)
4.  [Connect Strapi and Meilisearch](#connect-strapi-and-meilisearch)
5.  [Conclusion](#conclusion)

Create a directory for our app
------------------------------

We don't want our app to be all over the place! Let's create a directory where we will add every part of our application.

```bash
mkdir my-app
cd my-app
```

If you are not using a [UNIX-based operating system](https://www.eukhost.com/blog/webhosting/linux-flavors/) please follow this guide to [create a directory](https://www.computerhope.com/issues/ch000742.htm).

Create a back end using Strapi
------------------------------

In this section we are going to create and run a back-end using Strapi.

Our first step is to generate a Strapi Application. Go to your open terminal window and run the following command:

```bash
npx create-strapi-app@3.5.4 back --quickstart

```
<Callout>
  Using [npx](https://www.npmjs.com/package/npx) we run a specific version of the package without installing it globally. Note that this guide may not work with a different version.
</Callout>

This command creates a Strapi app in a new directory called `back` and opens the admin dashboard. Create an account or log in so that you have access.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_14_28_54_1_453a624ffa.png)

If you lose this page at any time, just go to `http://localhost:1337/admin` in your browser

Once you have created your account, you should be redirected to Strapi's admin dashboard. This is where we will configure our back-end API.

Our next step is to create a new collection type—in this case, it'll be a collection of restaurants. Complete [step 3 of Strapi's quick start guide](https://strapi.io/documentation/developer-docs/latest/getting-started/quick-start.html#_3-create-a-restaurant-content-type) to continue.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_15_31_41_2effed8eb3.png)

Admin dashboard landing page

After finishing [step 3 of Strapi's quick start guide](https://strapi.io/documentation/developer-docs/latest/getting-started/quick-start.html#_3-create-a-restaurant-content-type), a new collection named `Restaurants` should have appeared under `Collection Types`. If we click on it, we can see that there are still no restaurants. Let's add some!

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_15_07_36_63c48765e5.png)

`Restaurants` collection landing page

Click on `+ Add New Restaurants` in the upper-right corner of the dashboard. Now let's add three restaurants, one by one:

*   Name: `The Butter Biscotte`. Description:  `All about butter, nothing about health.`
*   Name: `The Slimy Snail` Description: `Gastronomy is made of garlic and butter.`
*   Name: `The Smell of Blue` Description: `Blue Cheese is not expired, it is how you eat it. With a bit of butter and a lot of happiness.`

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_15_15_38_2c431c5150.png)

Create an entry in the Restaurant Collection.

For each restaurant, you need to press `save` and then `publish`. Once the three restaurants have been added, you should end up with the following page:

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_20_at_15_22_55_e0137d6c9d.png)

Main page of `Restaurants` showcasing the restaurants we have added

Our Strapi back end is now up and running! Strapi automatically creates a REST API for our `Restaurants` collection. Check out Strapi's documentation for all available [API endpoints](https://strapi.io/documentation/developer-docs/latest/developer-resources/content-api/content-api.html#api-endpoints).

Next, it's time to bring Meilisearch into the mix.

Launch and Start Meilisearch
----------------------------

**tl;dr:** [download, install and start Meilisearch](https://docs.meilisearch.com/learn/getting_started/installation.html#download-and-launch).

Want to avoid a local installation? Use our [Meilisearch cloud](https://cloud.meilisearch.com/) free 14-day trial or our [one-click install on DigitalOcean](https://marketplace.digitalocean.com/apps/meilisearch) and [continue to the next section](#connect-strapi-and-meilisearch).

If you made it this far, you should already have an open terminal window running Strapi. If not or if you closed it by accident, please [follow this guide](https://www.ionos.com/help/email/troubleshooting-mail-basicmail-business/access-the-command-prompt-or-terminal), navigate to the directory `.../my-app/back` that we created, and run the command `npm run develop` to restart Strapi.

Let's take a moment to **open a second terminal window** and move to the `my-app` directory. This window will be used to run Meilisearch.

Inside `my-app`, create a new directory called `meilisearch-binary` .

```bash
mkdir meilisearch-binary
cd meilisearch-binary
```

Create and move to directory where we will run Meilisearch

Execute the following command to download Meilisearch in the current directory. If cURL doesn't work for you, consider trying one of the [many other methods to download and install](https://docs.meilisearch.com/learn/getting_started/installation.html#download-and-launch) Meilisearch.

```bash
# Install Meilisearch
curl -L https://install.meilisearch.com | sh
```

Download and install Meilisearch

We now have the `meilisearch` binary in our directory. Let's run it!

```bash
# Launch Meilisearch
./meilisearch
```

Launch Meilisearch

You should see a big chunk of text (including some ASCII art) confirming that Meilisearch is up and running.

![](https://unable-actionable-car.media.strapiapp.com/MS_adress_75a76c4b2b.png)

Running Meilisearch

Find the line titled `Server listening on`. This lets us know where Meilisearch is being served. Once you've found it, open this adress in your browser.

You should arrive on a web page without much to look at.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_21_at_13_21_44_d9162084ee.png)

The Meilisearch web interface w/ empty index

This is the Meilisearch [web interface](https://www.meilisearch.com/docs/learn/what_is_meilisearch/search_preview). It will remain empty until we add some data to our Meilisearch instance.

Data, you say? It just so happens we already have some in our Strapi app! Time to connect the two and start searching through our data with Meilisearch.

Connect Strapi and Meilisearch
------------------------------

In order to add the Meilisearch plugin to Strapi, we need to quit our Strapi app. Go to the terminal window running Strapi (_make sure it's not the one running Meilisearch!_) and push `control + c` to kill the process.

Once you've done that, move inside the `back` directory and install the plugin.

```json
cd back
npm install strapi-plugin-meilisearch
```

After the installation, we have to rebuild our Strapi app before starting it again in development mode, since it makes configuration easier.

```bash
npm run build
npm run develop
```

At this point, our Strapi app should be running once again on the default address: [`http://localhost:1337/admin/`](http://localhost:1337/admin/) . Visiting it in your browser, you should see an admin sign-in page; enter the same credentials as before.

Once connected, you should see a new `Plugin` on the left side of the screen: Meilisearch! Let's click on it.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_21_at_14_32_44_c00e3d34f8.png)

Arriving on the Meilisearch plugin page, we can see that the information is split into two sections:

*   Meilisearch credentials
*   Strapi collections

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_21_at_14_44_12_6b821d2d2e.png)

Meilisearch plugin page of Strapi admin dashboard

First, we need to add the credentials of our Meilisearch instance so that Strapi knows where to send our data.

You may remember from the previous step that Meilisearch runs on `http://localhost:7700` by default. Add this in the `Meilisearch Host` field, then click `Add` to connect your Meilisearch instance to Strapi.  

![](https://unable-actionable-car.media.strapiapp.com/credentials_6fe7804ffa.gif)

Add credentials to Meilisearch.

> Note that we leave `Meilisearch Api Key` blank because we started Meilisearch without a [master key](https://www.meilisearch.com/docs/learn/configuration/instance_options#master-key).

Great! Now it's time to add your Strapi collection to Meilisearch.

Try clicking on the check box next to `restaurant` and watch as Meilisearch instantly begins processing your data. Because we only have three documents, the indexing time is practically non-existent.

![](https://unable-actionable-car.media.strapiapp.com/Large_GIF_956x212_5f52b4fd0f.gif)

Add restaurants to Meilisearch

We can now go back to our Meilisearch host address: `http://localhost:7700`. After refreshing, you should observe that the page is not as empty as before.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2021_04_22_at_10_07_08_6dbd424e4d.png)

Demo page of Meilisearch

We did it! Try inputting "butter" and check out the results.

#### Front-end Interface

Using [instant-meilisearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch), you can implement a front-end search interface in [React](https://github.com/meilisearch/meilisearch-react), [Vue](https://github.com/meilisearch/meilisearch-vue), or [Angular](https://github.com/meilisearch/meilisearch-angular) in no time at all.  
  
Here's an interactive [code sandbox](https://codesandbox.io/s/react-movies-demo-e1dqh) (in React) where you can get a feel for the different components. You can't break anything, so feel free to play around!

### Conclusion

I hope this article was a pleasant introduction to the new [Meilisearch plugin for Strapi](https://github.com/meilisearch/strapi-plugin-meilisearch). However, this is just the beginning!

If you have any questions, please join us [on Discord](https://discord.gg/meilisearch). If you like Meilisearch, [a star on GitHub means a lot](https://github.com/meilisearch/meilisearch). For more information, see our:

*   [Documentation](https://www.meilisearch.com/docs/)
*   [GitHub](https://github.com/meilisearch/meilisearch)
*   [Strapi Plugin](https://github.com/meilisearch/strapi-plugin-meilisearch)
*   [Discord](https://discord.gg/meilisearch)

How to add relevant search to your Strapi app using Meilisearch

_[Originally published](https://docs.airbyte.io/tutorials/slack-history) on Airbyte site_

**Overview**
------------

The [Slack free tier](https://slack.com/pricing/paid-vs-free) saves only the last 10K messages. For social Slack instances, it may be impractical to upgrade to a paid plan to retain these messages. Similarly, for an open-source project like [Airbyte](https://airbyte.io) where we interact with our community through a public Slack instance, the cost of paying for a seat for every Slack member is prohibitive.

However, searching through old messages can be really helpful. Losing that history feels like some advanced form of memory loss. What was that joke about Java 8 Streams? This contributor question sounds familiar—haven't we seen it before? But you just can't remember!

This tutorial will show you how you can, for _free_, use Airbyte to save these messages (even after Slack removes access to them). It will also provide you a convenient way to search through them.

Specifically, we will export messages from your Slack instance into an open-source search engine called [Meilisearch](https://github.com/meilisearch/meilisearch). We will be focusing on getting this setup running from your local workstation. We will mention at the end how you can set up a more productionized version of this pipeline.

We want to make this process easy, so while we will link to some external documentation for further exploration, we will provide all the instructions you need here to get this up and running.

**Set up Meilisearch**
----------------------

First, let's get Meilisearch running on our workstation. Meilisearch has extensive docs for [getting started](https://docs.meilisearch.com/reference/features/installation.html#download-and-launch). For this tutorial, however, we will give you all the instructions you need to set up Meilisearch using Docker.

```Shell
docker run -it --rm \
  -p 7700:7700 \
  -v $(pwd)/data.ms:/data.ms \
  getmeili/meilisearch
```

That's it!

> Meilisearch stores data in `$(pwd)/data.ms`, so if you prefer to store it somewhere else, just adjust this path.

**How to replicate your Slack messages to Meilisearch**
-------------------------------------------------------

### Set up Airbyte

Make sure you have Docker and Docker Compose installed. If you haven’t set Docker up, follow the [instructions here](https://docs.docker.com/desktop/) to set it up on your machine. Then, run the following commands:

```Shell
git clone https://github.com/airbytehq/airbyte.git
cd airbyte
docker-compose up
```

If you run into any problems, feel free to check out our more extensive [getting started](https://docs.airbyte.io/getting-started) for more help.

Once you see an Airbyte banner, the UI is ready to go at `http://localhost:8000/`. Once you have set your user preferences,  you will be brought to a page that asks you to set up a source. In the next step, we'll go over how to do that.

### Set up Airbyte’s Slack source connector

In the Airbyte UI, select Slack from the dropdown. We provide step-by-step instructions for setting up the Slack source in Airbyte [here](https://docs.airbyte.io/integrations/sources/slack#setup-guide). These will walk you through how to complete the form on this page.

![](https://lh3.googleusercontent.com/ZCxW4_nPPsLB4_fiszkcCIWFIDS8nSifxcwLweC2tH__sUdhEF-34YXQmQXJCSZGg4upzpv1NbICIKNKS0ayap4X41Unxhf7qLmfmDCzEbkl8M2R9ggUnMi-czMU-v_QSAD2mL07)

By the end of these instructions, you should have created a Slack source in the Airbyte UI. For now, just add your Slack app to a single public channel (you can add it to more channels later). Only messages from that channel will be replicated.

The Airbyte app will now prompt you to set up a destination. Next, we will walk through how to set up Meilisearch.

### Set up Airbyte’s Meilisearch destination connector

Head back to the Airbyte UI. It should still be prompting you to set up a destination. Select "Meilisearch" from the dropdown. For the host field, set: `http://localhost:7700`. The api\_key can be left blank.

### Set up the replication

On the next page, you will be asked to select which streams of data you'd like to replicate. We recommend unchecking "files" and "remote files" since you won't really be able to search them easily in this search engine.

![](https://lh6.googleusercontent.com/O-bUkd7k_8Eo-K9PNZA1kCd-bbg0DekSwyNfaXdDHIb3Pcyd9gi8nJXljU0NJL3ELaY6sDq8HoUuaMklFi5fQBpTplfTDklOuYdoQF_lpe_vFAZ9wSOLtPp76iWM7VFnwuC9rkbh)

For frequency, we recommend every 24 hours.

**Search Meilisearch**
----------------------

After the connection has been saved, Airbyte should start replicating the data immediately. When it completes you should see the following:

![](https://lh5.googleusercontent.com/ZErxFD14kuzXkBx2Fv6OtjPM80AogKllbLPlMQJZ8rfb1OTVhKRdt0rr1rS6e7bcnngfs0qJ7pLzNL4CsXYAByzJSspjSXPgbqkyemyiaRP7aPeMT8Wqs7gqtkuTUmKSmZSwdN_3)

When the sync is done, you can sanity check that this is all working by making a search request to Meilisearch. Replication can take several minutes depending on the size of your Slack instance.

```Shell
curl 'http://localhost:7700/indexes/messages/search' --data '{ "q": "<search-term>" }'
```

For example, I have the following message in one of the messages that I replicated: welcome to airbyte..

```Shell
curl 'http://localhost:7700/indexes/messages/search' --data '{ "q": "welcome to" }'
# => {"hits":[{"_ab_pk":"7ff9a858_6959_45e7_ad6b_16f9e0e91098","channel_id":"C01M2UUP87P","client_msg_id":"77022f01-3846-4b9d-a6d3-120a26b2c2ac","type":"message","text":"welcome to airbyte.","user":"U01AS8LGX41","ts":"2021-02-05T17:26:01.000000Z","team":"T01AB4DDR2N","blocks":[{"type":"rich_text"}],"file_ids":[],"thread_ts":"1612545961.000800"}],"offset":0,"limit":20,"nbHits":2,"exhaustiveNbHits":false,"processingTimeMs":21,"query":"test-72"}
```

**Search via a UI**
-------------------

Making curl requests to search your Slack History is a little clunky, so we have modified the example UI that Meilisearch provides in [their docs](https://docs.meilisearch.com/learn/tutorials/getting_started.html#integrate-with-your-project) to search through the Slack results.

Download (or copy and paste) this [html file](https://github.com/airbytehq/airbyte/blob/master/docs/tutorials/slack-history/index.html) to your workstation. Then, open it using a browser. You should now be able to write search terms in the search bar and get results instantly!

![](https://lh6.googleusercontent.com/JfzrnvxFz-08gIL6hMyWeW5TB8_XGgAUibx26o0gXhxJVc97IgW_YnqvWEV2ta8OvJbTQwQUUpYpsHggvbvRSJp8YVTw9w6Y3Ocl8E33Ab-L0E5visu-Beh_kqJwolt-9m_ZEk8N)

**"Productionizing" saving Slack history**
------------------------------------------

You can find instructions for how to host Airbyte on various cloud platforms [here](https://docs.airbyte.io/deploying-airbyte).

Documentation on how to host Meilisearch on cloud platforms can be found [here](https://www.meilisearch.com/docs/learn/cookbooks/running_production).

If you want to use the UI mentioned in the section above, we recommend statically hosting it on S3, GCS, or equivalent.

Save and search through your Slack history on a free Slack plan

Hey everybody! As some of you may know, I [recently joined Meilisearch](__GHOST_URL__/caro/) as a Developer Advocate. Part of my job is to help the community by answering questions, so my priority over the last few months has been to get to know Meilisearch. I believe the best way to learn new software is to use it just as any developer would: to build something. So, I set out to make a demo.

<Callout>
  This demo was created when the latest version of Meilisearch was v0.19, some of the workarounds described in this blogpost are no longer necessary due to the addition of new features in more recent versions of Meilisearch.
</Callout>

Meilisearch x MoMA
------------------

The first step was to find a nice dataset that I enjoyed working with. After several unsuccessful attempts to tame some unstructured datasets, I stumbled upon the [repository of the Museum of Modern Art (MoMA)](https://github.com/MuseumofModernArt/collection). What a lucky find! They have two datasets: artists and artworks. I decided to go for the artworks dataset since it has all the essential information about each art piece: title, artist, dimensions, medium, even the artist’s biography. But the best part is that it’s available in JSON format! No conversion needed, no MongoDB dump to restore, no scraping: Meilisearch ready 😀

I used [Meilisearch JavaScript](https://github.com/meilisearch/meilisearch-js) to create the indexes, add the documents, and customize the settings. For the front-end, I decided to go for [Instant Meilisearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch) combined with [Vue InstantSearch](https://github.com/algolia/vue-instantsearch/) as described in the [meilisearch-vue repository](https://github.com/meilisearch/meilisearch-vue/). I’m a little lazy, so if I find a tool that provides nice results effortlessly, I will use it! Fortunately, InstantSearch provides all the front-end tools to customize the search environment at will, so you can experience the power of Meilisearch and its search-as-you-type feature with just a few lines of code. Finally, I tried to add some decent styling, but it’s not my forte 😅 I apologize in advance for the visual design!

**You can see the result** [**here**](https://moma.meilisearch.com/)**.**

![Web interface with searchbar and results for the query "Picasso"](https://unable-actionable-car.media.strapiapp.com/Capture_d_e_cran_2021_02_18_a_16_40_57_41c7a62d0f.png)

Solving Problems
----------------

So, I had found a dataset, indexed it in a Meilisearch instance, and set up a front-end for the search. All done, right? Not exactly; now came the time to iterate and make the search results feel good.

I was lucky to find a structured JSON dataset that worked effortlessly with Meilisearch. Nevertheless, I had to make some small changes to better display the results. Since Meilisearch’s [automatic highlighting](https://www.meilisearch.com/docs/learn/what_is_meilisearch/features#highlighting) doesn’t work on query results that are stored in an array, I had to **convert array values into strings** before adding them.

I also wanted to allow users to **sort the results by date**. Since the date field didn’t have a standard format, I had to retrieve the year and store it in a new \`sortByDate\` field. Then there was another problem: Meilisearch doesn’t support sorting at query time (**at least, not yet**)! To circumvent that and implement this feature, I created two more indexes with custom ranking rules desc(DateToSortBy) and asc(DateToSortBy). Every time you change the sorting option in the drop-down menu, Meilisearch is actually searching in a completely different index, but it’s so fast that the change is not visible to the human eye 🤯

Here you can see the sorting in action:

![Web interface with the query "Picasso" sorting by ascending and then descending date gif](https://unable-actionable-car.media.strapiapp.com/sorting2_f59a0e2432.gif)

🧐 You might have noticed that I said Meilisearch doesn’t support query-time sorting **yet**—that’s because it’s one of the most requested features on [our public roadmap](https://roadmap.meilisearch.com/tabs/1-under-consideration), and it’s now under consideration 🥳 So if you haven’t done it yet, take a look at Meilisearch’s roadmap and vote for your favorite feature or submit your own ideas! **👉 Update note:** query-time sorting was released with Meilisearch v0.22.

The most challenging part of creating this demo was working with an unknown dataset of 138,151 documents, because the large number of documents makes it difficult to know the possible values and length of the different fields. I first wanted the user to be able to filter results by \`medium\` (e.g.  “photograph” or “paint”). That is, until I realized some of the artworks had \`medium\` fields that were many lines long! Take this one for example:

> "H (vol. IX): Cover with lithographed manuscript text by Kliun on front; 1 lithographed illustration by Terent'ev; lithographed manuscript text. L (vol. XIII): Cover with lithographed illustration and manuscript design by Kirill Zdanevich on front; lithographed manuscript and typed text. M (vol. XIV): Cover with lithographed manuscript design and illustration on front by Kirill Zdanevich; lithographed manuscript and typed text. P (vol. XVII): Cover with lithographed manuscript design on front by Goncharova; and lithographed manuscript text by Mikhail Pustynin and Olga Olesha-Suok"

Can you imagine having this as a [facet filter](https://www.meilisearch.com/docs/learn/advanced/faceted_search)? Not very nice for the UI…

Close to Perfect
----------------

The easiest part of making this demo was setting up Meilisearch. Once I decided which attributes I wanted to be [displayed](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#displayed-fields) and [searchable](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#searchable-fields), the rest was a piece of cake. Aside from the highlighting and sorting issues mentioned above, the last thing I changed was [the attribute ranking order](https://www.meilisearch.com/docs/learn/core_concepts/relevancy#attribute-ranking-order). By default, the attribute ranking order is generated automatically based on the attributes' order of appearance in the first document indexed. In my case, the first attribute was \`title\`, which meant that artwork with matching query terms in the \`title\` field topped the results list. This delivered decent results, but I was pretty sure I could do better.

I not only had to consider which attributes are most important for this particular dataset; I also had to think about what types of queries are more likely to be made. If a user queries for a specific art piece using its title, he will get relevant results because the words used are unlikely to be found in lots of other fields. However, if a user queries the name of a famous artist, it is very possible to find that name in an artwork title, for example as a tribute. So I decided to place \`artist\` at the top of [the searchable attributes list](https://www.meilisearch.com/docs/learn/configuration/displayed_searchable_attributes#the-searchableattributes-list). Before I made the change, this is what you would find among the first results when searching for “Van Gogh” :

![Information and image of a litograph called "Do You Leave without seeing the 150 Van Gogh's" by Willem Jacob Henri Berend Sandberg](https://unable-actionable-car.media.strapiapp.com/Capture_d_e_cran_2021_02_22_a_11_34_08_073b869794.png)

As you can see, it’s an art piece created by an artist named Willem Jacob Henri Berend Sandberg where the name “Van Gogh” appears in the title, but it’s not an actual Van Gogh.

Now, after changing the order of the searchable attributes in [the settings](https://www.meilisearch.com/docs/reference/api/settings#searchable-attributes), the first result is well and truly a piece of art created by Vincent Van Gogh:

![Information and image of the litograph "The Potato Eaters" by Vincent van Gogh](https://unable-actionable-car.media.strapiapp.com/Capture_d_e_cran_2021_02_22_a_11_41_41_7bdabf659b.png)

  
Pretty amazing how a simple change like this can impact the search results. Meilisearch is ultra relevant out-of-the-box, but it lets you tune the relevancy of the results according to your needs, and that is essential to provide an awesome experience for the end user.

An Open-Source Museum Experience
--------------------------------

The demo source code is available on [GitHub](https://github.com/meilisearch/demo-MoMA/). Feel free to play with it and show me what you can do! 😁

How cool is it that we’re able to access this extraordinary catalog? Thanks to MoMA’s generosity, you can explore the museum from anywhere in the world. And now, thanks to Meilisearch, it’s only 50 milliseconds away 🚀

* * *

_Photo by [Robert Bye](https://unsplash.com/@robertbye?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText) on [Unsplash](https://unsplash.com/?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)_

Meilisearch: a blank canvas

As a Rails and Ruby lover, I often search for the gem that will perfectly cover my use case. When I need to solve a problem I'm having, I want to choose the _right one_, the most suitable solution.

Ruby gems are extensive libraries created inside the Ruby community. The best place to find a ready solution for any task is the website [rubygems.org](https://rubygems.org/), a public repository of gems that can be searched using the search box on the home page. RubyGems’ website is an efficient tool that facilitates the sharing and the installation of packages. But although its search bar is very helpful, I decided to create an [alternative search bar](https://rubygems.meilisearch.com/) more suited to our needs.

![](https://unable-actionable-car.media.strapiapp.com/rubygemsv3_510e7b8445.gif)

The search-as-you-type experience
---------------------------------

First, I wanted to implement a search-as-you-type experience. Which means:

*   a response time lower than 50 milliseconds
*   presenting all matching results immediately under the search box while the user is typing and without them pressing enter

That is not yet the case in the RubyGems' website since a new page is loaded each time a request is made.

The relevancy
-------------

You can get relevant and accurate results by using the RubyGems' search bar, but most of the time only by performing an Advanced Search, which is not always convenient. You have to decide what to fill in the different sections. Are you going to search for a specific package by entering its name (e.g. "devise") or find a package whose summary would match a keyword (e.g. "deployment")?

However, despite this functionality, you may not find gems that meet your needs. For example, if you enter "pagination" you would expect to see the gem "kaminari", which is the most popular gem for pagination among the RoR community, showing up in the results. Here is [the return we get from RubyGem's search bar](https://rubygems.org/search?utf8=%E2%9C%93&query=pagination) when submitting the keyword "pagination". As you can see, "kaminari" doesn’t appear before the 9th result.

Even when [refining the search](https://rubygems.org/search?utf8=%E2%9C%93&query=summary%3A+pagination+description%3A+pagination), the first result to show is "kanimari-core" which is not exactly the more appropriate and famous "kaminari" package we would like to find, but it’s still better than nothing.

Then if we do a search [containing a typo in the request](https://rubygems.org/search?utf8=%E2%9C%93&query=pagintion), like "pagintion", the page is displayed without any result and suggests you a similar word for your next search.

After this experience as a user, I aimed to create **a single search bar able to understand what you want and to instantly find it**!

![](https://unable-actionable-car.media.strapiapp.com/speed_b868b434f8.jpg)

Meilisearch checks all these points, and more!
----------------------------------------------

I've never implemented a search engine; I've never even used one, except a basic Elasticsearch instance with no configuration for a proof of concept. For that reason, all I needed was an easy-to-setup tool, able to handle both speediness and relevancy at the same time. That's why Meilisearch was a perfect fit for this project.

**[Meilisearch](https://github.com/meilisearch/MeiliSearch) is an ultra-relevant and fast search engine**. In other words, it can return the most relevant results of your dataset under 50 ms, and therefore it gives a strong sense of immediacy.

Plus, without the need to configure anything, it handles search miss-spelling: i.e. Typos. Try to submit "devose" instead of "devise" and Meilisearch will return "devise" as the first result.

Finally, **Meilisearch is open-source and integrates a simple [RESTful API](https://www.meilisearch.com/docs/reference/api/overview)**. You can seamlessly communicate with the API using cURL or [one of Meilisearch's wrappers](https://www.meilisearch.com/docs/learn/what_is_meilisearch/sdks).

Creating the alternative search bar
-----------------------------------

All the [gems data](https://rubygems.org/pages/data) are available on the RubyGems' website as a PostgreSQL dump file and are updated daily. Thus, I wrote a Ruby script to download the latest dataset, parse the PostgreSQL dump file, and push all the data into my Meilisearch instance. Of course, it uses the [meilisearch-ruby wrapper](https://github.com/meilisearch/meilisearch-ruby) to communicate with the API. This script is hosted in Heroku and runs every day thanks to the Heroku Scheduler.

About the Meilisearch instance, at Meili we manage an internal Kubernetes cluster, which is a handy tool to host demos like this one. For curious readers who would like to find out more, Meilisearch is quite [easy to download and run](https://www.meilisearch.com/docs/learn/getting_started/installation) by yourself (Homebrew, APT, Docker...).

Regarding HTML and CSS, I kept much of the original structure of the RubyGems' website. My intention was to develop a "search-as-you-type experience" in the same spirit as the original website. The front is deployed using GitHub Pages.

![](https://unable-actionable-car.media.strapiapp.com/chang_duong_F1_UI_4q_Dv1r_Q_unsplash_e212aa22e9.jpg)

Easily improving the relevancy
------------------------------

Without having to set anything up, Meilisearch returns pretty relevant results. Our search engine can quickly find the most suitable packages when inputting gem names like "devise" or "faraday". Unfortunately, for now it’s not always the case with keywords.

Let’s get back to my "pagination" example. If I run the search again without configuring anything, the first result to be displayed by Meilisearch will be the [Pagination](https://rubygems.org/gems/pagination) gem. I don't see [Kaminari](https://rubygems.org/gems/kaminari) at all in the results. That’s because by default, finding a document with a requested word in the title takes precedence over a document with a requested word in the description. Since there are many gems containing "pagination" in their title in the dataset, it explains why Kaminari does not appear at all.

I needed Meilisearch to include the libraries popularity as well. In my dataset, the popularity of the Ruby gems is indicated by the number of downloads. I classified my gems in eight groups of fame (downloaded more than 50M times, more than 30M, and so forth) from `0` to `7`. The latter being considered as the most famous group.

I added this information to every document (i.e. gems) as a field named `fame`. Then, I integrated this rule in Meilisearch settings as a custom ranking rule.

![](https://unable-actionable-car.media.strapiapp.com/5_519da9ed48.png)

Take a look at this snippet above. To put it simply, Meilisearch will execute one by one all of these rules (`_sum_of_typos`, `_number_of_words` ...) and sort your documents following this sequence. When I add my custom rule, i.e. `fame` in `rankingOrder` and `fame: 'dsc'` in `rankingRules`, I'm in fact asking Meilisearch to sort by fame in descendant order.

You might have noticed I have a second custom rule in the example: `total_downloads`, so that my results will be sorted by the number of downloads. But since I chose to place this rule at the end of the list, meaning it was considered less important than the other ones, it will be the last one to be applied. The order of sequence definitely matters.

I'm not going to give further details about the [Meilisearch default ranking rules](https://docs.meilisearch.com/learn/core_concepts/relevancy.html#ranking-rules), even if it’s a particularly interesting topic. Describing how our search engine works deserves indeed a separate article of its own! 😉 _Spoiler alert: Meilisearch uses a bucket sort!_

Now, if you type a [global keyword like "pagination"](https://rubygems.meilisearch.com/?q=pagination) you will find Kaminari in the first place; and if you try again with a [less famous gem name like "pagy" for example](https://rubygems.meilisearch.com/?q=pagy) you will still get the gem you expected! 🎉

Meilisearch + you = 💛
----------------------

These minor settings were really easy to integrate, and your projects might require the same kind of behavior.

Here are some useful links if you want to get ready for your own Meilisearch experience:

*   the [documentation](https://docs.meilisearch.com/)
*   the [Meilisearch's repository](https://github.com/meilisearch/MeiliSearch) on GitHub: support Meili by starring it! ⭐️
*   the [repository of this project](https://github.com/meilisearch/finding-rubygems/)

If you are interested in our project, how it works, or if you have any feedback, do not hesitate to [contact the team](mailto:bonjour@meilisearch.com)! 😁

Meilisearch finds Rubygems

Today, I am about to guide you in the depths of crates.io and how [I made an alternative search bar](https://crates.meilisearch.com) using our instant search engine: [Meilisearch](https://github.com/meilisearch/MeiliDB).

![](https://unable-actionable-car.media.strapiapp.com/crates_io_demo_798a562ddd.gif)

The Meili instant, relevant, and typo-tolerant search engine

[Crates.io](https://crates.io) is the official website that stores Rust community crates (packages), and it's the place where the cargo package manager uploads, updates, and downloads those.

[Sean Griffin](https://twitter.com/sgrif) is part of the crates.io team and maintains the current search engine of it along with the whole website. [Kornel Lesinski](https://kornel.ski/) built [lib.rs](https://lib.rs/) an alternative to [crates.io](http://crates.io/) and uses [Tantivy](https://github.com/tantivy-search/tantivy/) to power its search bar. To be honest, I prefer its color design, that's why I took it for our search demo.  
  
I decided to run our instant search engine and test its relevancy overtime against these existing solutions. Our search engine uses completely different algorithms; it is based on prefix search and is typo tolerant.

![](https://unable-actionable-car.media.strapiapp.com/current_results_1_73457862c1.png)

Meili is typo-tolerant and supports a whole lot of other features

So, I asked myself: Why not use our new instant search engine and make it useful to my beloved community? It would give us lots of feedback and probably some pull requests during the process.

* * *

At Meili we manage an internal Kubernetes cluster, this is useful to host demos for clients. The Meilisearch server for this demo currently run on a pod in this cluster.

To make Meilisearch exhibit those crates, we needed to find all the currently available packages on crates.io. Fortunately, this index is available on GitHub in the form of several subfolders with the names and versions of the packages, containing something like _32 000_ files. A commit is done each time a crate is updated to a new version, or a version is yanked.

So, I used the [crates.io-index repository](https://github.com/rust-lang/crates.io-index) to **initialize** our newly created Meili search engine but needed more data first, like, the description, keywords, and categories of each of those crates. Again, the Rust crates.io team was here for us, I talked to [Pietro Albini](https://twitter.com/pietroalbini), and he pointed me to the not rate-limited servers that deliver packages content.

Now that we can retrieve useful data, I create an [async crawler](https://github.com/Kerollmops/meili-crates/blob/67770fe179436eee6478cdebff7d8613f9263049/src/lib.rs) that downloads, extracts, retrieves the Cargo.toml, and upload the essential data to Meilisearch.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2019_11_08_Meili_Dashboard_c9d2ae42f1.png)

The Meili dashboard interface showing raw documents

Meilisearch now understands that data and gives us instant, relevant and typo tolerant responses. But, what about new crates? We want to be notified about new crates and be able to send those to Meilisearch.

[Docs.rs](https://docs.rs) is the official website to compute and store the documentation of all the Rust packages hosted by crates.io. It diffs the crates.io index each minute to know about new crates updates. Luckily, it provides an Atom feed of those updates.

It is where [Heroku](https://heroku.com) enters the game. Heroku offers something like _1000_ free hours per month of computing power on their servers and provides us schedulers. We can consume those credits and freely [ask docs.rs about newly updated crates](https://github.com/Kerollmops/meili-crates/blob/67770fe179436eee6478cdebff7d8613f9263049/src/bin/live.rs), every 10 minutes, by fetching the Atom feed, download the updated packages like previously, and finally update our search engine, **live**!

![](https://unable-actionable-car.media.strapiapp.com/heroku_live_timings_1_fbb7661c94.png)

Fetching new crates takes us 4 seconds

Search results were satisfying but not as expected, there was something wrong. When we wrote "serde", for example, the first result was relevant, but the next ones were not. It was related to the fact that Meilisearch did not have enough data to rate crates except the query matching words.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_2019_11_10_Meili_crates_browser_2000_1_5cfc7e603c.png)

Meilisearch doesn't know how to settle equal crates with only the matching words

**Downloads counts** amazingly improved the search results. [This data is available through crates.io](https://crates.io/data-access). A full database export is done each day, and it contains the number of downloads of each crate. I decided to use those as the last ranking criterion to help Meilisearch settle about crates considered equal.

![](https://unable-actionable-car.media.strapiapp.com/Screenshot_Meili_crates_browser_2000_good_1_21f3eb0098.png)

Meilisearch shows better results thanks to the number of downloads

I deployed a Heroku scheduler to run each day to update all the crates downloads; it takes something like 30 seconds to download the tarball, extract it, read the CSV and upload the _32 000_ crates downloads counts to Meilisearch. So we are far from _1000_ free hours per month.

* * *

I think this search demo is pretty good, but I also thought about adding synonyms and stop-words as Meilisearch supports those. For example, it would be pleasant to write "db" and see results associated with "database". Stop-words would help ignore useless words like "the" or "that" which could pollute search results sometimes, but we need to be careful as crates names could be composed of stop-words.

Meilisearch also supports basic filtering, and it would be great in the future to be able to search for crates in a category or with a specific keyword.

All these improvements can be made by you; [the demo is open source](https://github.com/Kerollmops/meili-crates). The [core engine source code](https://github.com/meilisearch/MeiliDB) is also available on GitHub, that's the whole point of this article! Take a look and talk about it, more people involved, more features!