Getting started with AI-powered search experimental

AI-powered search, sometimes also called vector search or hybrid search, is an experimental technology that uses large language models (LLMs) to retrieve search results based on the meaning and context of a query.

This tutorial will walk you through configuring AI-powered search in your Meilisearch project. You will see how to activate this feature, generate document embeddings with OpenAI, and perform your first search.

• A running Meilisearch project
• An OpenAI API key
• A command-line console

First, create a new Meilisearch project. If this is your first time using Meilisearch, follow the quick start then come back to this tutorial.

Next, create a kitchenware index and add this kitchenware products dataset to it. It will take Meilisearch a few moments to process your request, but you can continue to the next step while your data is indexing.

AI-powered search is an experimental feature and is disabled by default. You must manually activate it either via the Meilisearch Cloud UI, or with the experimental features endpoint.

Navigate to your project overview and find "Experimental features". Then click on the "AI-powered search" box.

Use the /experimental-features route to activate vector search during runtime:

curl \
  -X PATCH 'MEILISEARCH_URL/experimental-features/' \
  -H 'Content-Type: application/json'  \
  --data-binary '{
    "vectorStore": true
  }'

Replace MEILISEARCH_URL with your project's URL. In most cases, this should look like https://ms-000xx00x000-xx.xxx.meilisearch.io if you're using Meilisearch Cloud, or http://localhost:7700 if you are running Meilisearch in your local machine.

In this step, you will configure an OpenAI embedder. Meilisearch uses embedders to translate documents into embeddings, which are mathematical representations of a document's meaning and context.

Open a blank file in your text editor. You will only use this file to build your embedder one step at a time, so there's no need to save it if you plan to finish the tutorial in one sitting.

In your blank file, create your embedder object:

{
  "products-openai": {}
}

products-openai is the name of your embedder for this tutorial. You can name embedders any way you want, but try to keep it simple, short, and easy to remember.

Meilisearch relies on third-party services to generate embeddings. These services are often referred to as the embedder source.

Add a new source field to your embedder object:

{
  "products-openai": {
    "source": "openai"
  }
}

Meilisearch supports several embedder sources. This tutorial uses OpenAI because it is a good option that fits most use cases.

Models supply the information required for embedders to process your documents.

Add a new model field to your embedder object:

{
  "products-openai": {
    "source": "openai",
    "model": "text-embedding-3-small"
  }
}

Each embedder service supports different models targeting specific use cases. text-embedding-3-small is a cost-effective model for general usage.

Log into OpenAI, or create an account if this is your first time using it. Generate a new API key using OpenAI's web interface.

Add the apiKey field to your embedder:

{
  "products-openai": {
    "source": "openai",
    "model": "text-embedding-3-small",
    "apiKey": "OPEN_AI_API_KEY",
  }
}

Replace OPEN_AI_API_KEY with your own API key.

Meilisearch embedders only accept textual input, but documents can be complex objects containing different types of data. This means you must convert your documents into a single text field. Meilisearch uses Liquid, an open-source templating language to help you do that.

A good template should be short and only include the most important information about a document. Add the following documentTemplate to your embedder:

{
  "products-openai": {
    "source": "openai",
    "model": "text-embedding-3-small",
    "apiKey": "OPEN_AI_API_KEY",
    "documentTemplate": "An object used in a kitchen named '{{doc.name}}'"
  }
}

This template starts by giving the general context of the document: An object used in a kitchen. Then it adds the information that is specific to each document: doc represents your document, and you can access any of its attributes using dot notation. name is an attribute with values such as wooden spoon or rolling pin. Since it is present in all documents in this dataset and describes the product in few words, it is a good choice to include in the template.

Your embedder object is ready. Send it to Meilisearch by updating your index settings:

curl \
  -X PATCH 'MEILISEARCH_URL/indexes/kitchenware/settings/embedders' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "products-openai": {
      "source": "openAi",
      "apiKey": "OPEN_AI_API_KEY",
      "model": "text-embedding-3-small",
      "documentTemplate": "An object used in a kitchen named '{{doc.name}}'"
    }
  }'

Replace MEILISEARCH_URL with the address of your Meilisearch project, and OPEN_AI_API_KEY with your OpenAI API key.

Meilisearch and OpenAI will start processing your documents and updating your index. This may take a few moments, but once it's done you are ready to perform an AI-powered search.

AI-powered searches are very similar to basic text searches. You must query the /search endpoint with a request containing both the q and the hybrid parameters:

curl \
  -X POST 'MEILISEARCH_URL/indexes/kitchenware/search' \
  -H 'content-type: application/json' \
  --data-binary '{
    "q": "kitchen utensils made of wood",
    "hybrid": {
      "embedder": "products-openai"
    }
  }'

For this tutorial, hybrid is an object with a single embedder field.

Meilisearch will then return an equal mix of semantic and full-text matches.

Congratulations! You have created an index, added a small dataset to it, and activated AI-powered search. You then used OpenAI to generate embeddings out of your documents, and performed your first AI-powered search.

Now you have a basic overview of the basic steps required for setting up and performing AI-powered searches, you might want to try and implement this feature in your own application.

For practical information on implementing AI-powered search with other services, consult our guides section. There you will find specific instructions for embedders such as LangChain and Cloudflare.

For more in-depth information, consult the API reference for embedder settings and the hybrid search parameter.