How to implement document boosting with a second index

In this guide, we'll walk you through implementing promoted search results with Meilisearch. Our goal is to search results prioritizing specific documents when certain keywords match a user's query. These boosted documents should be returned at the top of the search results.

Overview

Here's a simplified breakdown of how to implement document boosting using a second index for “pinned documents” and the multi-search feature:

1. Create Indexes: Set up two indexes: one for regular search and one for boosted results. The boosted index will have a special attribute, keywords, to trigger the boosting.
2. Populate the 'games' Index: Populate a games index with your dataset using the provided JSON file. This index will serve as the source for our boosted documents.
3. Configure the 'pinned_games' Index: Configure the pinned_games index to display attributes without revealing keywords. Adjust the searchable and displayed attributes accordingly.
4. Boost Documents: Identify documents you want to boost and assign relevant keywords to them. For instance, you can assign the keywords fps and shooter to the game Counter-Strike.
5. Implement the Multi-Search: Utilize Meilisearch's multi-search feature to perform a search query across both the regular and boosted indexes. This way, boosted documents matching keywords will appear first.
6. Display Results: Present the search results in a user-friendly format, highlighting boosted documents with a visual indicator.

Implementation

Installation

Before diving in, make sure you have Meilisearch up and running. If you haven't installed it yet, follow these steps:

1. Launch a Meilisearch instance — you can run Meilisearch in local or via Meilisearch Cloud.
2. Ensure you have your favorite language SDK (or framework integration) installed.

Initializing indexes

For our example, we'll work with a dataset of Steam games. You can adapt this process to your own data:

1. Download the steam-games.json and settings.json files for our Steam games dataset
2. Load the dataset in your Meilisearch instance by adding documents from the steam-games.json file.

games index

import meilisearch
import json
from typing import Callable

client = meilisearch.Client(url="http://localhost:7700")
games = client.index("games")

# helper to wait for Meilisearch tasks
def wait_with_progress(client: meilisearch.Client, task_uid: int):
    while True:
        try:
            client.wait_for_task(task_uid, timeout_in_ms=1000)
            break
        except meilisearch.errors.MeilisearchTimeoutError:
            print(".", end="")
    task = client.get_task(task_uid)
    print(f" {task.status}")
    if task.error is not None:
        print(f"{task.error}")
            
print("Adding settings...", end="")
with open("settings.json") as settings_file:
    settings = json.load(settings_file)
    task = games.update_settings(settings)
    wait_with_progress(client, task.task_uid)


with open("steam-games.json") as documents_file:
    documents = json.load(documents_file)
    task = games.add_documents_json(documents)
    print("Adding documents...", end="")
    wait_with_progress(client, task.task_uid)

pinned_games index

This index will contain the promoted documents. The settings of the pinned_games index are the same as the games index, with the following differences:

• the only searchableAttributes is the keywords attribute containing the words that trigger pinning that document.
• the displayedAttributes are all the attributes of the documents, except for keywords (we don't want to show the keywords to end-users)

pinned = client.index("pinned_games")

print("Adding settings...", end="")
with open("settings.json") as settings_file:
    settings = json.load(settings_file)
    settings["searchableAttributes"] = ["keywords"]
    # all but "keywords"
    settings["displayedAttributes"] = ["name", "description", "id", "price", "image", "releaseDate", "recommendationCount", "platforms", "players", "genres", "misc"]
    task = pinned.update_settings(settings)
    # see `wait_with_progress` implementation in previous code sample
    wait_with_progress(client, task.task_uid) 

Updating the promoted documents index

We'll now populate the index with documents from the games index that we want to promote.

As an example, let's say we want to pin the game "Counter-Strike" to the "fps" and "first", "person", "shooter" keywords.

counter_strike = games.get_document(document_id=10)
counter_strike.keywords = ["fps", "first", "person", "shooter"]

print("Adding pinned document...", end="")
task = pinned.add_documents(dict(counter_strike))
wait_with_progress(client, task.task_uid)

Customizing search results

Now, let’s create a function to return the search results with the pinned documents.

from copy import deepcopy
from typing import Any, Dict, List
from dataclasses import dataclass

@dataclass
class SearchResults:
    pinned: List[Dict[str, Any]]
    regular: List[Dict[str, Any]]

def search_with_pinned(client: meilisearch.Client, query: Dict[str, Any]) -> SearchResults:
    pinned_query = deepcopy(query)
    pinned_query["indexUid"] = "pinned_games"
    regular_query = deepcopy(query)
    regular_query["indexUid"] = "games"
    results = client.multi_search([pinned_query, regular_query])
    # fetch the limit that was passed to each query so that we can respect that value when getting the results from each source
    limit = results["results"][0]["limit"]
    # fetch as many results from the pinned source as possible
    pinned_results = results["results"][0]["hits"]
    # only fetch results from the regular source up to limit
    regular_results = results["results"][1]["hits"][:(limit-len(pinned_results))]
    return SearchResults(pinned=pinned_results, regular=regular_results)

We can use this function to retrieve our search results with promoted documents:

results = search_with_pinned(client, {"q": "first person shoot", "attributesToRetrieve": ["name"]})

The results object should look like:

SearchResults(pinned=[{'name': 'Counter-Strike'}], regular=[{'name': 'Rogue Shooter: The FPS Roguelike'}, {'name': 'Rocket Shooter'}, {'name': 'Masked Shooters 2'}, {'name': 'Alpha Decay'}, {'name': 'Red Trigger'}, {'name': 'RAGE'}, {'name': 'BRINK'}, {'name': 'Voice of Pripyat'}, {'name': 'HAWKEN'}, {'name': 'Ziggurat'}, {'name': 'Dirty Bomb'}, {'name': 'Gunscape'}, {'name': 'Descent: Underground'}, {'name': 'Putrefaction'}, {'name': 'Killing Room'}, {'name': 'Hard Reset Redux'}, {'name': 'Bunny Hop League'}, {'name': 'Kimulator : Fight for your destiny'}, {'name': 'Intrude'}])

Tada 🎉 You now have a search results object that contains two arrays: the promoted results and the “regular” ones.

Going further

This tutorial explored one approach for implementing promoted results. An alternative technique would be implementing documents pinning in the frontend; take a look at our React implementation guide. This different approach has the benefits of being compatible with InstantSearch.

Both techniques can achieve similar results. We also plan to integrate promoted documents in the Meilisearch engine. Give your feedback on the previous link to help us prioritize it.

For more things Meilisearch, you can subscribe to our newsletter. You can learn more about our product by checking out the roadmap and participating in our product discussions.

For anything else, join our developers community on Discord.

Cheers!