How to build a rag pipeline: А step-by-step guide

[In Meilisearch v1.6](https://www.meilisearch.com/blog/meilisearch-1-6), we revolutionized search by combining the precision of full-text search with the intelligence of AI. This hybrid approach isn't just another feature—it's a fundamental reimagining of how search should work. Today, we'll pull back the curtain on our implementation and show why Meilisearch's unique approach to hybrid search delivers superior results.

Join us as we explore the mechanics of hybrid search, examine conventional implementation methods, and reveal how Meilisearch's innovative solution raises the bar for search relevance.

## What is hybrid search?

Search is always about finding the documents that are the most relevant to a user’s query.

Hybrid search is about combining the results of two complementary approaches to search: full-text search and semantic search.

[Full-text search](https://www.meilisearch.com/blog/how-full-text-search-engines-work) analyzes the words that appear in a query in a lexical manner, excelling at correcting typos and completing incomplete queries. It is also fast, and its results are easy to understand in retrospect.

[Semantic search](https://www.meilisearch.com/blog/semantic-search) leverages Large Language Model (LLM) technology to understand the meaning of words in the query. It excels at finding documents that contain synonyms to the user’s query and other similar concepts.

Both search approaches build a list of relevant documents according to their respective methods. Hybrid search combines these lists to produce a final list of results.

The golden question is, how to combine them?

![Diagram containing two lists of documents as shapes (squares for full-text, diamonds or triangles for semantic), and an arrow with an interrogation mark that produces a new list of documents containing both kinds of shapes](https://unable-actionable-car.media.strapiapp.com/how_to_merge_55aea77e87.png)

## The traditional way of mixing results is flawed

The traditional method of mixing the results from both lists is called [_Reciprocal Rank Fusion_ (RRF)](https://medium.com/@devalshah1619/mathematical-intuition-behind-reciprocal-rank-fusion-rrf-explained-in-2-mins-002df0cc5e2a), because it considers the **rank** of the documents in each list of results, and uses that as a basis to compute the final list of results.

In this context, the rank of the document refers to its position in the list of results, indicating how close it appears to the top.

For example, in a list of 20 documents, the result ranked first can be attributed 20 points. RRF can then attribute points to documents in each of the lists, sum the points for each document, and reorder the documents accordingly. Here’s a [detailed explanation](https://learn.microsoft.com/en-us/azure/search/hybrid-search-ranking) about an implementation of RRF on Azure AI Search.

However, this method of mixing results is flawed, because to function properly, it assumes that documents from all the lists of results are of comparable relevance: that is, all the documents in all lists are “good documents” that we want to return.

![Keyword search. Pikachu uses "imprecise query". It's super effective!](https://unable-actionable-car.media.strapiapp.com/pikachu_hybrid_search_f4f9e6c29f.jpg)

But, like Pokémons, each search method is vulnerable to different types of search queries. Covering for the weakness of the other method is the whole point of mixing them!

So let’s say you’re using a query that is fine for semantic search, but gives underwhelming results for keyword search. Using RRF, you might get some variant of the following:

![Illustrated example of the downfall of rank fusion](https://unable-actionable-car.media.strapiapp.com/RFF_merge_789ff4e216.png)

Recursive rank fusion is like wasting fine chocolate by mixing it with spoiled milk!

To address this issue, we require additional information about the documents listed in our results.

## Ranking is not enough: Scoring documents

The main idea is that you can score _how_ relevant a document is from 0 (not relevant at all) to 1 ("perfect” match), independently of its rank in the list.

Naturally, documents closer to the top of the list are more relevant, but the first document might be anywhere between 0 and 1. Then, you compare the scores in both lists and rank the documents across both lists that way.

Fortunately, Meilisearch already scores documents for full-text search (for more details about this, refer to [our previous article about scoring](https://www.meilisearch.com/blog/from-ranking-to-scoring)), and semantic search natively ranks documents depending on how similar they are to the query (although the details are complicated).

So we have a score for both full-text search and semantic search, but can we compare them?

## Making the scores comparable

When you have two scores ranging from 0.0 to 1.0, it can be tempting to compare them, but for this to work, similarly relevant documents must be scored similarly with both methods.

One difficulty that arises when performing this comparison is that the models used in semantic search tend to use only a portion of the full vector space they could use, resulting in vectors that have a similarity of at least 0.5 even for texts that appear entirely dissimilar. Furthermore, even very similar texts might be capped in similarity around 0.7, which will cause them to rank lower than good full-text results.

Fortunately, Meilisearch provides the tools to solve that particular problem. Meilisearch allows you to perform an affine transformation to the scores of the semantic search, so that the scores can be offset by the observed mean similarity between vectors, and the results that are packed around that mean similarity can be spread more in the 0.0, 1.0 space. For more details, refer to [the documentation on the](https://www.meilisearch.com/docs/reference/api/settings#distribution) `distribution` [field](https://www.meilisearch.com/docs/reference/api/settings#distribution).

For models known to Meilisearch (such as OpenAI models), a suitable default distribution is already applied to configured embedders, so users don’t have to configure it themselves!

With this operation, our scores are comparable, and depending on the query, the method that produces the most relevant documents will be favored by the hybrid search

![Failing example, running with Meilisearch, yields better results](https://unable-actionable-car.media.strapiapp.com/meili_merge_9bb48a7b7d.png)

## What else goes into making hybrid search right?

Scoring documents is a decisive relevancy advantage of Meilisearch over the traditional approach based on Reciprocal Rank Fusion, but there are a lot more aspects involved in making hybrid search right:

- We developed our bespoke vector store, [arroy](https://github.com/meilisearch/arroy), to leverage our database backend (LMDB) and meet our performance target. Arroy supports Approximate Nearest Neighbor search, multiple distances, native document filters, and binary quantization. Arroy doesn’t block a reader while writing or a writer while reading, and allows for incremental updates.
- We kept flexibility in mind with the mantra of making easy things simple, and hard things possible: users that are new to hybrid search can quickly setup an embedder with our default provider and configuration, but users can also carefully optimize their embedding process with multiple supported provider (or even the ability to provide your own with [REST embedders](https://www.meilisearch.com/docs/learn/ai_powered_search/configure_rest_embedder) and [user-provided embeddings](https://www.meilisearch.com/docs/learn/ai_powered_search/search_with_user_provided_embeddings)) and [document templates](%5B%3Chttps://www.meilisearch.com/docs/learn/ai_powered_search/document_template_best_practices%3E%5D(%3Chttps://www.meilisearch.com/docs/learn/ai_powered_search/document_template_best_practices#sample-document%3E)) to customize how documents are turned into text to embed.
- Best-in-class full-text search engine, seamlessly integrated with semantic search, with support for efficient and expressive [sorting and filtering](https://www.meilisearch.com/docs/learn/filtering_and_sorting/working_with_dates) of documents.

Search is foundational; it needs to be rock solid to reliably serve the needs of your application. Meilisearch’s hybrid search implementation scores documents to merge results while preserving their relevancy. Scores also power the [federated search](https://www.meilisearch.com/docs/learn/multi_search/multi_search_vs_federated_search#what-is-federated-search%3F), a generalized version of semantic search that unifies results from multiple lists of intermediate hits.

Our users use federated search to perform hybrid searches across multiple vector stores, frequently combining image and text search requests.

## Why Meilisearch's hybrid search stands apart

Meilisearch's innovative approach to hybrid search goes beyond simply combining full-text and semantic search results.

By implementing a sophisticated scoring system instead of traditional rank fusion, carefully calibrating score comparability, and building a robust technical foundation with arroy, we've created a hybrid search solution that consistently delivers more relevant results.

This thoughtful implementation ensures that the strengths of each search method are preserved while compensating for their weaknesses, setting a new standard for search relevance in modern applications.

Why traditional hybrid search falls short (and how we fixed it)

Imagine you're building a blog in Rails, and your users keep misspelling post titles in the search bar, only to find no results. Frustrating, right? 

What if your Rails app or website had a search feature that’s lightning-fast *and *smart enough to handle typos?

Meilisearch is your answer. It's a lightweight, typo-tolerant, and lightning-fast search engine that offers a seamless developer experience and out-of-the-box relevancy tuning.

You can make a search engine with Ruby on Rails in Meilisearch by:

1. Setting up Meilisearch and installing dependencies.
2. Configuring Meilisearch in the Rails app.
3. Defining a model and enabling Meilisearch indexing.
4. Populating and indexing data.
5. Implementing a backend and frontend.
6. Customizing settings.

A Ruby on Rails–powered search engine can be used for building e-commerce platforms, blogs, or searchable knowledge bases. Regardless of your use case, a search engine built with Ruby on Rails in Meilisearch comes with a slick search interface that’s both powerful and user-friendly.

## 1. Set up Meilisearch and install dependencies

First, we need to get Meilisearch up and running and integrate it with our Rails app. This will lay the foundation for your search engine.

Meilisearch is open source and free if you choose to [host it locally](https://www.meilisearch.com/docs/learn/self_hosted/getting_started_with_self_hosted_meilisearch). Or, you can choose Meilisearch Cloud for a fully managed hosting experience, with no need to handle infrastructure or updates yourself. 

For this tutorial, we’ll use a local instance, which is perfect for testing. You’ll need Ruby 3.0+ and Rails 6.1+ to ensure compatibility. Older versions may still allow it to function, but they are not officially supported.

The `meilisearch-rails` gem simplifies the integration, handling indexing and syncing with your database. Let’s install Meilisearch and add the gem to your project.

To install Meilisearch locally, follow the instructions in the official [Meilisearch documentation on GitHub](https://github.com/meilisearch/meilisearch-rails). You can download the binary and run it with a simple command. Once running, Meilisearch is accessible at [http://127.0.0.1:3000](http://127.0.0.1:3000).

Open your `Gemfile `and, with `gem` in the command line, add:


```ruby
gem install meilisearch-rails

source 'https://rubygems.org'

gem 'meilisearch-rails'
```


Run `bundle install` to install the gem. If you hit any issues, double-check your Ruby and Rails versions, as the gem requires recent versions for full compatibility.

With Meilisearch and the gem installed, we’re ready to configure our Rails app to connect to it.


## 2. Configure Meilisearch in your Rails app

Now, let’s configure our Rails app to connect to Meilisearch. Think of this step as setting the coordinates that guide your app to the search engine.

Create a new configuration file to specify the `MEILISEARCH_HOST `and `MEILISEARCH_API_KEY`. This ensures smooth and secure communication between your app and Meilisearch. 

In development, you can skip the API key, but in production, it’s essential for maximum security. Proper configuration, such as what we’re doing now, prevents connection errors and ensures smooth indexing and searching.

Create a new file at `config/initializers/meilisearch.rb`:


```ruby
Meilisearch::Rails.configuration = {
  meilisearch_url: ENV.fetch('MEILISEARCH_HOST', 'http://localhost:7700'),
  meilisearch_api_key: ENV.fetch('MEILISEARCH_API_KEY', 'YourMeilisearchAPIKey')
}
```


If you’re using Meilisearch Cloud, replace the URL with your cloud instance’s address, which is available after signing up for Meilisearch Cloud. For local development, the default URL works fine. If you encounter connection issues, ensure Meilisearch is running and the URL is correct.

Now that our app knows where to find Meilisearch, let’s set up a model to index data.


## 3. Define a model and enable Meilisearch indexing

This step is where the magic starts: we’ll make our data searchable by indexing it with Meilisearch. Indexing means cataloging your information and data so Meilisearch can quickly find matches during user searches.

For this example, we’ll use the hypothetical Rails model `Article` and include the `MeiliSearch::Rails` module to enable automatic indexing. You’ll decide which attributes to index, like `title `and `body`, and configure how they’re used in searches. Meilisearch syncs with your database via `ActiveRecord `callbacks, so you don’t have to worry about updating it manually.

First, generate a model if you don’t have one:


```ruby
rails generate model Article title:string body:text category:string
rails db:migrate
```


Then, open `app/models/article.rb` and add:


```ruby
class Article < ApplicationRecord
  include MeiliSearch::Rails

  meilisearch do
    attribute :title, :body, :category
    searchable_attributes [:title, :body]
    filterable_attributes [:category]
    sortable_attributes [:created_at]
  end
end
```


Here’s what’s happening:

* `attribute` lists the fields requiring indexing.
* `searchable_attributes` are fields users can search.
* `filterable_attributes` allow filtering of the results, e.g., by category.
* `sortable_attributes `enable sorting, e.g., by creation date.

Here, choosing the right attributes is crucial. For example, indexing `title` and `body` makes sense for a blog, but you might want to add `tags` or `author` depending on your use case. Choose only the most relevant fields to index for better performance and faster search results.

With our model ready, let’s add some data to the index.


## 4. Populate and index sample data

To test our search engine, we need data to search through. For this purpose, we will populate our database with sample articles.

Using a seed file, we’ll create sample records that Meilisearch will automatically index. This will let you see how Meilisearch handles real data and prepare you for testing search queries. Meilisearch’s efficiency really shines here, as it can index thousands of records at lightning speeds.

To start, create or update `db/seeds.rb` with the articles you want searched:


```ruby
# Clear existing articles — use only in development or testing!
Article.destroy_all
articles = [
  { title: "Ruby on Rails Basics", body: "Learn the fundamentals of Rails.", category: "Beginner" },
  { title: "Advanced Rails Techniques", body: "Dive into complex Rails features.", category: "Advanced" },
  { title: "Rails Performance Tips", body: "Optimize your Rails app.", category: "Performance" },
  { title: "Building APIs with Rails", body: "Create robust APIs in Rails.", category: "API" },
  { title: "Rails Security Best Practices", body: "Secure your Rails application.", category: "Security" }
]
articles.each { |article| Article.create!(article) }
puts "Articles seeded!"
```


Then, run:


```ruby
rails db:seed
```


Meilisearch will index these records automatically using the aforementioned `ActiveRecord `callbacks. To verify, you can use Meilisearch’s API or dashboard to check the `articles `index. If (correct) data isn’t appearing, ensure your Meilisearch instance is running and the configuration is correct.

Now, things get even more interesting — it’s time to implement the search functionality.


<Callout intent="tip">
Ready to try it out? See Meilisearch’s search capability in full force in a [free trial](https://www.meilisearch.com/cloud)!
</Callout>


## 5. Implement backend search functionality

First, we’ll create a controller to handle search queries and display accurate results.

This step sets up a `SearchController `to process user input and fetch results from Meilisearch. The `meilisearch-rails` gem makes this easy by providing a `search `method that returns `ActiveRecord `objects, so you can treat results like regular model instances. Additionally, Meilisearch’s typo tolerance ensures users get relevant results even with misspellings.

Create `app/controllers/search_controller.rb`:


```ruby
class SearchController < ApplicationController
  def index
    @query = params[:q]
    @results = Article.search(@query) if @query.present?
  end
end
```


Update your routes in `config/routes.rb`:


```ruby
Rails.application.routes.draw do
  get 'search', to: 'search#index'
end
```


Create a view at `app/views/search/index.html.erb`:


```erb
<h1>Search Articles</h1>
<%= form_with url: search_path, method: :get, local: true do %>
  <%= text_field_tag :q, params[:q], placeholder: "Search articles..." %>
  <%= submit_tag "Search" %>
<% end %>

<% if @results %>
  <h2>Results for "<%= @query %>"</h2>
  <ul>
    <% @results.each do |result| %>
      <li><strong><%= result.title %></strong>: <%= result.body %></li>
    <% end %>
  </ul>
<% end %>
```


Visit [http://localhost:3000/search](http://localhost:3000/search) and try queries like ‘Ralis’ (a typo for ‘Rails’). Meilisearch’s typo tolerance will still return relevant results, as expected. If results aren’t appearing at all, or they seem to be incorrect, check your model’s indexing settings or Meilisearch logs.

Now, let’s take it up a notch with a frontend search interface through our [JavaScript API Client](https://github.com/meilisearch/meilisearch-js/) or [Instant Meilisearch plugin](https://github.com/meilisearch/instant-meilisearch).


## 6. Add a frontend search with instant Meilisearch

For a search-as-you-type experience, we’ll integrate Meilisearch’s simple frontend interface using `instant-meilisearch`.

This step enables dynamic, instant search functionality in your app, giving users an interactive experience as they type. Meilisearch’s impressive response speed — with response times ≤50ms — ensures results appear almost instantly. We’ll use a simple JavaScript setup, but you can extend it with [frameworks like React](https://www.meilisearch.com/with/react) for a polished UI.

Install the library via `npm `(or `yarn`). Note: this assumes your Rails app is already set up with a JavaScript bundler such as Webpacker or Importmaps.


```
npm install @meilisearch/instant-meilisearch
```


Create `app/javascript/search.js`:


```js
import { instantMeiliSearch } from '@meilisearch/instant-meilisearch';

const searchClient = instantMeiliSearch(
  '',// URL of Meilisearch instance
  '' // Meilisearch API key
);

const search = async (query) => {
  const response = await searchClient.search([{
    indexName: 'articles',
    query
  }]);
  const resultsDiv = document.getElementById('results');
  resultsDiv.innerHTML = response.hits.map(hit => `<div>${hit.title}: ${hit.body}</div>`).join('');
};

document.getElementById('search-input').addEventListener('input', (e) => {
  search(e.target.value);
});
```


Update `app/views/search/index.html.erb`:


```erb
<h1>Search Articles</h1>
<input type="text" id="search-input" placeholder="Search articles...">
<div id="results"></div>
```


As before, test the search at [http://localhost:3000/search](http://localhost:3000/search). Typing ‘Rail’ should show results instantly.

Next, let’s explore our customization options.


## 7. Customize Meilisearch settings

To make your search engine even better, you can tweak Meilisearch’s settings to fit your product’s needs.

Meilisearch allows you to customize ranking rules, synonyms, and stop words to improve search relevance. For example, you might want ‘Rails’ and ‘Ruby on Rails’ to be treated as synonyms, or prioritize results based on recency.

For instance, here is how to adjust ranking rules to prioritize certain fields:


```ruby
Article.meilisearch_index.update_settings({
  rankingRules: [
    "words",
    "typo",
    "proximity",
    "attribute",
    "sort",
    "exactness",
    "created_at:desc"
  ]
})
```


These settings ensure newer articles rank higher. Feel free to play around and test different configurations to see what works best for your users.

Finally, let’s ensure your search index stays in sync with your database for the most accurate results.


## 8. Handle updates and deletions

The `meilisearch-rails` gem automatically handles indexing when you create or update records. However, it’s still worth understanding how to manage updates and deletions yourself, especially if you’re building more complex web applications. This will ensure your search results always reflect the latest data and avoid stale or missing entries.

To manually delete a record from your index:


```ruby
article = Article.find(1)
article.remove_from_index!
Article.destroy! # Removes from Meilisearch index
```


If you need to reindex all records, for example, after changing any indexed attributes, run:


```
Article.reindex!
```


To clear your index entirely, use:


```
Article.clear_index!
```


Be cautious with reindexing large datasets, however, as it can be resource-intensive. In production environments, you can schedule such tasks during low-traffic periods.


## Can I make a search engine with Ruby on Rails for free?

Yes, you can build a search engine with Ruby on Rails and Meilisearch at no cost for development and small-scale production.

Meilisearch’s open-source version is free to self-host on your own servers or local machine, and the `meilisearch-rails` gem has no licensing fees. For development, this is ideal, as you only need a server to run Meilisearch. 

For production, however, hosting costs depend on your infrastructure. Small apps can run on affordable servers, while the costs may skyrocket for larger projects.

Meilisearch Cloud offers a 14-day free trial without requiring a credit card, making it perfect for testing. Post-trial, pricing scales with usage (documents and search requests), starting with the Build plan for small businesses at $30/month.

Let’s compare Meilisearch to other open-source search engines.

## What are the best open-source search engines for Ruby on Rails?

Open-source search engines provide powerful, license-free solutions for adding search to Rails apps. Here are the top options:

* **Meilisearch**: Known for its speed (under 50ms) and simplicity, Meilisearch integrates seamlessly with Rails via the `meilisearch-rails` gem. It’s ideal for small to medium apps, offering [typo-tolerance](https://www.meilisearch.com/blog/typo-tolerance) and search-as-you-type features. Its lightweight nature makes it easy to set up and maintain yourself, while also providing a hosted option for a maintenance-free experience.
* **Elasticsearch**: A robust engine, Elasticsearch excels in handling large datasets and complex queries. Gems like Searchkick simplify integration. It’s perfect for enterprise apps but requires more resources and setup than Meilisearch.
* **Solr**: Solr offers advanced functionalities like full-text search and faceting. The Sunspot gem provides Rails integration. It’s highly configurable, which can get complex for smaller projects.
* **Typesense**: A lightweight alternative for creating search engines for Ruby on Rails, Typesense offers fast, typo-tolerant search. While it lacks a dedicated Rails gem, its API is easy to use for simple setups.

Now, let’s look at other languages for building search engines.


## What programming languages besides Ruby are used to build search engines?

Aside from Ruby, several programming languages are well-suited for building search engines, each with its unique strengths. Some common languages you can use to build search engines are:


* **JavaScript**: Using Node.js, you can build server-side search with libraries like Lunr or integrations with Meilisearch. JavaScript’s asynchronous nature is great for real-time search. Learn how to [build a search engine in JavaScript](https://www.meilisearch.com/blog/javascript-search-engine).
* **PHP**:Frameworks like Laravel use Scout to integrate with Meilisearch, making PHP a solid choice for web apps that include search engines. Learn how to [build a search engine in PHP](https://www.meilisearch.com/blog/php-search-engine).
* **Golang**: Golang’s performance and concurrency make it ideal for high-speed search engines. Libraries like Bleve support full-text search, further increasing its applicability. Learn how to [build a search engine with Golang](https://www.meilisearch.com/blog/golang-search-engine).

## Creating a search engine in Ruby on Rails made simple

Building a search engine with Ruby on Rails and Meilisearch is easier than you’d think. Set up Meilisearch, index data, add a backend and frontend, and customize the settings to your heart’s content. Meilisearch’s speed, typo-tolerance, and seamless Rails integration make it a top choice for developers looking for fast search engine configuration.

Full tutorial on how to design a search engine with Ruby on Rails

If you want to build a fast and reliable search engine, the simplest answer is Golang and Meilisearch Go SDK. Combine these, and not only can you index your content as it should be, but you can also search queries within milliseconds and keep everything lightweight as ever.

In this guide, you’ll learn how to:

* Set up Go and connect it to Meilisearch
* Index structured data using JSON
* Build a clean API for real-time search

We’re going to share a method that is simple, fast, and works at scale flawlessly. This guide will help you avoid all the heavy frameworks and large teams that only slow you down. So, if you’re ready to build a lightweight search in Golang, let’s begin.

## 1. Set up your Golang development environment

First, you must prepare your Go environment and install the Meilisearch Go SDK. This is the primary prerequisite for building your search engine. It’s crucial because it enables your system to actually run Go code and communicate with your Meilisearch instance. 

Start by initializing your Go module:


```
go mod init search-engine
```


Then, install the Meilisearch SDK:


```
go get github.com/meilisearch/meilisearch-go
```


Again, we must confirm that Go is installed and Meilisearch runs locally or in the cloud. Then can use this setup to [index data](https://www.meilisearch.com/blog/introducing-indexer-2024) and handle queries for API creation. 

Once your environment is ready, you can connect to Meilisearch and start building.


## 2. Connect to a search engine database

Before all the indexing and searching, your Golang project must connect to a Meilisearch instance.

Meilisearch is an open-source, lightning-fast search engine that supports real-time search. It is also capable of [typo tolerance](https://www.meilisearch.com/blog/typo-tolerance) and full-text indexing, making your search even more potent. It’s time to connect the Meilisearch Go SDK to your database and perform search operations. 

Here’s how to initialize the client in Go:


```go
package main

import (
  "github.com/meilisearch/meilisearch-go"
)

func main() {
  client := meilisearch.NewClient(meilisearch.ClientConfig{
    Host: "http://127.0.0.1:7700",
    APIKey: "masterKey",
  })

  // Use `client` to index documents and run search queries
}
```


Remember, if you’re using the hosted version, you must replace the host URL with the respective Meilisearch Cloud address. 

Once connected, your Go application can add documents, create indexes, and run lightning-fast search queries with Meilisearch’s JSON-based API.


## 3. Create your document indexing logic

Is your Go app connected to Meilisearch already? Next up is indexing your data. Since Meilisearch works using JSON docs, you must define your content in structs before pushing it to the search engine. This enables [full-text search](https://www.meilisearch.com/blog/how-full-text-search-engines-work), filters, and fast retrieval.

Here’s how to index documents in Go:


```go
type Movie struct {
  ID     int    `json:"id"`
  Title  string `json:"title"`
  Genre  string `json:"genre"`
}

movies := []Movie{
  {ID: 1, Title: "Inception", Genre: "Sci-Fi"},
  {ID: 2, Title: "The Matrix", Genre: "Action"},
}

res, err := client.Index("movies").AddDocuments(movies)
if err != nil {
  log.Fatal(err)
}
```


Each document becomes searchable immediately after it is indexed. You can create multiple indexes like “products,” “posts,” or “users,” depending on your project. Document indexing is the backbone of your [Golang search engine](https://www.meilisearch.com/blog/golang-search-engine#{step-name}). This is the reason why Meilisearch can parse and retrieve results in milliseconds.


## 4. Build the search API and frontend

The connection is set, and the documents have been indexed. It is time to build the search functionality into your Golang application. The aim is to create an API endpoint that takes the search query to Meilisearch and returns the desired results. This is where features such as full-text search, typo tolerance, and filtering truly matter.

Here’s a simple search handler in Go:

```go
http.HandleFunc("/search", func(w http.ResponseWriter, r *http.Request) {
  query := r.URL.Query().Get("q")
  searchRes, err := client.Index("movies").Search(query, &meilisearch.SearchRequest{})
  if err != nil {
    http.Error(w, err.Error(), http.StatusInternalServerError)
    return
  }
  json.NewEncoder(w).Encode(searchRes.Hits)
})
```


Another good news is that this endpoint can be integrated with various frontend frameworks, such as React, Vue, plain HTML, and JavaScript. The accuracy and speed of all search results depend on the Meilisearch Go SDK.


## 5. Test and refine your Golang search engine

With your Golang search engine running, it's time to test real queries and improve your results, using all the capabilities we have covered so far. 

The next step is to search queries using partial matches, typos, and filters. Track how Meilisearch ranks and returns results. Measure speed. Review how your backend handles load. If using the Meilisearch Go SDK, confirm that your API responds in milliseconds and parses JSON documents accurately.

Want deeper insights? If you're using [Meilisearch Cloud](https://www.meilisearch.com/cloud), check the built-in analytics:

* Track top search terms
* Monitor failed searches
* Analyze performance by endpoint

Refine your transformer logic, tweak `rankingRules`, adjust typo tolerance, or set filters like `filterableAttributes`.


## What are some examples of Golang search engine uses?

If your goal is to build a fast search engine that’s equally responsive and lightweight, Golang is ideal. It provides a bonus set of features, including typo tolerance, ranking rules, and filterable results, when combined with the Meilisearch Go SDK.

Let’s look at some of the most frequent and efficient [use cases for search engines](https://www.meilisearch.com/blog/golang-search-engine#examples) in Go.

* **App search:** This is one of the most prominent uses of a Go search engine. Startups using lightweight applications with custom algorithms and text analysis often opt for this. Plus, they get a responsive UI and can use tokenization to handle dynamic query input in literal milliseconds.
* **Site search for blogs and documentation:** You can use Golang with Meilisearch to build fast, Boolean-enabled search features that support conjunction and disjunction queries. It can crawl and index Markdown or JSON content from static site generators like Gatsby or Hugo.
* **eCommerce product search:** When customers visit your e-commerce store, they don’t have the time for time-consuming or inaccurate search results. You will lose that customer instantly. \
To avoid such situations, developers use Golang with Meilisearch to create smart search functionality that retains customers through filters such as price, tags, category, and in-stock items.
* **Internal tool search:** It is also used in internal systems for teams. For instance, a team handling tools without a dedicated search system will frustrate members because they cannot find the tool they seek. \
On the other hand, if you use Go and Meilisearch, you can build an instant search system with quick and accurate response times. It utilizes the principles of tokenization and exact matching to return search results from your dashboards, CRMs, and inventory tools.
* **Knowledge base and help desk search:** When someone needs help, they don’t want to dig through articles. Meilisearch resolves this by giving them instant search results through Boolean logic and tokenization. Add Golang on top of this, and you can tie all of this together rather nicely. 
* **Regulated industry search (legal, healthcare, finance):** Private records are private for a reason. They are only available to a certain number of people. And these people expect to find these private records instantly. For instance, large datasets like legal contracts or medical files must be indexed properly so the search engine can bring them up quickly. 

Search engines built with Golang and Meilisearch are better than those without them, especially regarding performance, relevance, and customization. 


## Which languages besides Go are used to build a search engine?

We’ve learned how great Go is, and how it gets even better when paired with Meilisearch. Well, it’s not your only option. Various other programming languages can also help you develop real-time search features that might get the job done. Let’s look at some below:


* You can [build a search engine in JavaScript](https://www.meilisearch.com/blog/javascript-search-engine) if you're working with interactive frontend frameworks like React or Vue. JavaScript is perfect for browser-based apps that need fast, client-side search.
* Developers who want to work with data-heavy applications often [build search engines using Python](https://www.meilisearch.com/blog/python-search-egine). It’s a flexible backend language that works well for indexing large datasets and performing complex queries.
* For server-rendered applications and CMS platforms, you can [build a search engine with PHP](https://www.meilisearch.com/blog/php-search-engine). It’s ideal for adding search to WordPress, Laravel, or custom e-commerce backends.

Whichever you choose, Meilisearch offers SDKs and documentation to help you integrate search fast.


## Can I build a search engine in Golang for free?

You can build a full-featured search engine in Golang for free using open-source tools like [Meilisearch](https://github.com/meilisearch/meilisearch-go). Golang and Meilisearch are free to use and offer powerful capabilities out of the box—no licensing fees or paid tiers to get started.

You can run Meilisearch locally or host it on your own infrastructure without paying anything. Even if you scale later, it remains cost-effective with minimal setup and maintenance. The Meilisearch Go SDK handles everything from indexing to real-time queries with clean code and full documentation.

If you want a fast, flexible, and affordable search engine, Golang and Meilisearch are a great pair.


## Building a Search Engine in Golang Made Simple

Building a search engine in Golang doesn’t need to be complicated. With the Meilisearch Go SDK, you can create a fast, lightweight, and scalable search solution using just a few lines of code.

Golang’s performance makes it ideal for backend services, while Meilisearch supports full-text search, typo tolerance, and efficient indexing with clean query string handling. You can define various query types depending on your content and use case.

Whether you're indexing JSON documents or managing custom data structures, the setup is simple and fast. This makes it great for app search, e-commerce, internal tools, or site search. If you’ve worked with BleveSearch or understand Rust syntax, you'll find the transition smooth and the developer experience clean.

Building a search engine in Golang: A comprehensive guide

If you’re looking for a fail-proof, fast and scalable search functionality to directly add to your static site, the answer is Gatsby search. With just a few clicks, Gatsby search enables any user to look for relevant content across a Gatsby site, so you don’t have to worry about traditional navigation. 

Gatsby users can implement search by using plugins such as `gatsby-plugin-meilisearch`, `gatsby-plugin-local-search`, or by connecting external search APIs. These tools use GraphQL queries to index data from Markdown, JSON, or HTML files and support client-side or pre-rendered search components.

This article is your guide on how to add a real-time, full-text search engine to your Gatsby site. We’ll also cover details about the plugins, configurations, and writing GraphQL queries. And on top of that, you’ll learn how to transform data and deploy a working search index. 

Let’s start by going through the steps to add search to any Gatsby project using Meilisearch. 

## 1. Install the Meilisearch plugin for your Gatsby site

The first step is to install an open-source plugin: <code>[gatsby-plugin-meilisearch](https://www.gatsbyjs.com/plugins/gatsby-plugin-meilisearch/)</code>. Why? 

This `gatsby-plugin` is the connection between your Gatsby site and Meilisearch instance using defined API credentials. The main benefit is that it allows your project to generate a live search index during build time. Additionally, it supports [full-text search](https://www.meilisearch.com/blog/how-full-text-search-engines-work), filters, and smooth client-side search operations. So, let’s begin.

Use npm:


```
npm install gatsby-plugin-meilisearch
```


After installation, go to your `gatsby-config.js file`. This is where you define site settings and Gatsby plugin options. Add the plugin to the plugins array with the correct host and apiKey for your Meilisearch instance.


```js
// gatsby-config.js

module.exports = {
  plugins: [
    {
      resolve: 'gatsby-plugin-meilisearch',
      options: {
        host: 'http://localhost:7700',
        apiKey: 'masterKey',
      },
    },
  ],
}
```


You can learn more about this plugin on [GitHub](https://github.com/meilisearch/gatsby-plugin-meilisearch).

This sets up the foundation for your search engine. Next, you’ll define the search index and configure the data you want to fetch.


## 2. Configure the search index and GraphQL query

Once you have installed the `gatsby-plugin-meilisearch`, you need to define the content you want to send to Meilisearch. This will happen inside `gatsby-config.js`, where you set up the indexes array. 

Each entry represents a search index. This is also where you specify the GraphQL query and index name aligned with how you transform the data. 

Here’s an example configuration:


```js
// gatsby-config.js

module.exports = {
  plugins: [
    {
      resolve: 'gatsby-plugin-meilisearch',
      options: {
        host: 'http://localhost:7700',
        apiKey: 'masterKey',
        indexes: [
          {
            indexUid: 'pages_index',
            query: `
              {
                allSitePage {
                  nodes {
                    path
                  }
                }
              }
            `,
            transformer: data =>
              data.allSitePage.nodes.map((node, index) => ({
                id: index,
                path: node.path,
              })),
          },
        ],
      },
    },
  ],
}
```


This `graphql` query fetches data from your Gatsby site pages. The transformer function prepares it for indexing by assigning each entry a unique ID. This guarantees Meilisearch can now store and return the relevant search results properly.


## 3. Build and index your Gatsby content

Now, it’s time to build and index the content. We’ve already configured the [search index](https://www.meilisearch.com/blog/introducing-indexer-2024) and GraphQL. When you run `gatsby build`, `gatsby-plugin-meilisearch` takes the data from your Gatsby site and pushes it to your Meilisearch instance. This step connects your local content with a real-time search index.

To run the build:


```
gatsby build
```


Once complete, you’ll see a message like:


```
success gatsby-plugin-meilisearch - Documents added to Meilisearch
```


This is confirmation that your data has been indexed. The Meilisearch dashboard or local instance will help you inspect the indexed content. 

If you forget this step, your search query will not return any results since there will be no content to search through.

You can also refer to the official [Gatsby plugin documentation](https://www.gatsbyjs.com/plugins/gatsby-plugin-meilisearch/) for additional options, such as `batchSize`, `skipIndexing`, or custom settings like `searchableAttributes`.

Now that your content is indexed, it's time to connect the search UI.


## 4. Add a search component to your Gatsby frontend

Building a user interface comes after your Gatsby site content is indexed. To simplify this, you can use a search component like <code>[docs-searchbar.js](https://www.meilisearch.com/blog/add-a-search-bar-to-your-gatsby-project-using-meilisearch)</code>. It works because it integrates with Meilisearch and supports the search functionality we’re building.

Start by installing the package:


```
yarn add docs-searchbar.js
```


Then, in your `src/components` folder, locate the appropriate `JS file` (like <code>[AppBar.js](http://AppBar.js)</code>). Now, import the search module and CSS:


```js
import 'docs-searchbar.js/dist/cdn/docs-searchbar.css'
```


In your React component, use a `useEffect `hook to initialize the search:


```js
useEffect(() => {
  const docsSearchBar = require('docs-searchbar.js').default
  docsSearchBar({
    hostUrl: 'http://localhost:7700',
    apiKey: 'masterKey',
    indexUid: 'your-index',
    inputSelector: '#search-bar-input',
    meilisearchOptions: { limit: 5 },
    enhancedSearchInput: true,
  })
}, [])
```


Finally, add the search input field in your JSX:


```js
<input type="search" id="search-bar-input" placeholder="Search..." className="searchbox" />
```


This setup connects your static frontend to a live search index. Users get results as they type, and search query parsing, autocomplete, and typo tolerance are fully supported.

Next up is customization. You will learn how to modify a search experience to better match your site’s design and structure.


## 5. Customize your search experience

To unlock customization for your Gatsby site, you need a working search component. Once it’s working, you can modify the search experience according to your site’s structure and design. 

To begin, you will modify the transformer function in `gatsby-config.js`. 

This function determines how your data appears in Meilisearch. You can enrich each search index item with metadata like `frontmatter.title`, `description`, `slug`, or any custom fields.


```js
transformer: (data) =>
  data.allMdx.edges.map(({ node }) => ({
    id: node.id,
    title: node.frontmatter.title,
    description: node.frontmatter.description,
    url: node.slug,
  }))
```


Prioritization is a significant part of customization. To prioritize which fields Meilisearch should match first, you will update `searchableAttributes `in the plugin settings:


```js
settings: {
  searchableAttributes: ['title', 'description', 'content']
}
```


Meilisearch brings several other perks to your customization process. For instance, you can easily adjust query behavior, such as typo tolerance and filtering. Plus, you can always tweak the CSS styles of your search input and add a `classname `to match your design system.

Building components manually through React components? You might refer to the [React Search Engine guide](https://www.meilisearch.com/blog/react-search-engine) for increased frontend control. Customization is all about enhancing the user experience, and this is no exception. 

## 6. Build and deploy your Gatsby site with search

With the search functionality in place and customization all done, it’s time for the final step: to build and deploy your Gatsby site. Let’s run the protection build command:


```
gatsby build
```


During build, `gatsby-plugin-meilisearch` runs your GraphQL query, applies the transformer, and pushes data to the search index. This process runs on every deployment if your Gatsby site is hosted on platforms like Netlify.

Once live, test your searchbox, confirm search input behavior, and validate query results. On Meilisearch Cloud, you also get access to built-in analytics.

Use the official [Gatsby plugin documentation](https://www.gatsbyjs.com/plugins/gatsby-plugin-meilisearch/) or this[ Gatsby search tutorial for full setup steps](https://www.meilisearch.com/blog/add-a-search-bar-to-your-gatsby-project-using-meilisearch).


## How is search used in Gatsby?

Search adds speed and usability to static content—Gatsby sites often power blogs, documentation, ecommerce, and marketing pages.

With `gatsby-plugin`, content in Markdown, JSON, or HTML becomes searchable. This improves navigation and makes your site search fast and efficient.

Let’s have a look at some of the key use cases:

* **Site search:** A quick way to help users locate what they are looking for on your Gatsby site.
* **Docs search:** Makes it easier to look up commands in developer documentation built with Markdown and frontmatter.
* **Blog search:** Filtering by tag, category, or title makes it way easier to find the content you care about.
* **e-Commerce search:** Helps shoppers look up products and see suggestions as they type.
* **Static knowledge bases:** It is responsible for fast client-side search across indexed content.

In short, search is responsible for the user experience and engagement across all these formats.


### Does Gatsby support server-side search?

No, it does not. Gatsby is a static site generator, so it builds all your content ahead of time and shows pre-made HTML pages. An outside tool like Meilisearch could be the answer if you want to add live search to your site. It works by handling queries and returning the results through the API. For example, one way to add the desired search on your site is by using the [Meilisearch React integration](https://www.meilisearch.com/blog/react-search-engine).


### Does Gatsby support local search?

Yes. Gatsby supports local search using plugins like `gatsby-plugin-local-search` and `lunr`. These tools index content at build time and provide lightweight, browser-based search without relying on external APIs. While this suits smaller projects, larger sites often use Meilisearch for faster indexing, typo tolerance, and advanced search functionality. For complex content-driven sites, Meilisearch is a more flexible option.


## Building powerful search in Gatsby sites

Search is a must-have on any content-rich Gatsby site. Fast search is invaluable for blogs, docs, and especially online stores. Tools like `gatsby-plugin-meilisearch` bring real-time, typo-tolerant search functionality to your React components.

Meilisearch connects through your `gatsby-config.js `using a GraphQL query, a transformer function, and a defined search index. It supports Markdown, JSON, and nested content structures. With minimal setup, your static Gatsby site can offer dynamic search query handling and a modern user experience.


How to add search to your Gatsby site: Step-by-step guide

We're happy to announce the release of Meilisearch v1.15. In this article, we'll highlight the key changes and improvements in this release.

For a complete list of all updates and fixes, please visit the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.15.0) on GitHub.


<Callout intent="tip">

These powerful new features are available in [Meilisearch Cloud](https://www.meilisearch.com/cloud). Sign up now to be among the first to experience the latest improvements!

</Callout>

## New: Disable typo tolerance for numbers

With Meilisearch v1.15, you can turn off [typo tolerance](https://www.meilisearch.com/docs/learn/relevancy/typo_tolerance_settings) for numeric values—just like you already do for words and attributes.

A search for `2024` will match only `2024`, not `2025` or `2004`. This is especially helpful when searching for postal addresses or phone numbers.

Skipping number fuzziness can also speed up your indexing, especially for datasets that contain many unique numbers.

## New: Lexicographic string filters

You can now compare any string with `<`, `<=`, `>`, `>=` or an open range `TO`—just like numbers—using [lexicographic order](https://en.wikipedia.org/wiki/Lexicographic_order).

This is ideal for scenarios where you need to filter by human-readable values—such as dates (e.g., “2025-05-28”) instead of numeric timestamps.

The following filter finds every record released before July 17, 2023:

```
release_date < "2023-07-17" 
```

## Improvement: better support for Chinese

We’ve enhanced the Charabia tokenizer to provide more precise segmentation of Chinese text. **If your dataset contains Chinese-language content, you must reindex your database**—otherwise, Chinese queries and documents will not be processed correctly and may be ignored during search.

To reindex, you can use a [dump](https://www.meilisearch.com/docs/reference/api/dump) or—if you are using the [dumpless upgrade](https://www.meilisearch.com/docs/learn/update_and_migration/updating#dumpless-upgrade)—trigger a reindexing with one of the following methods:

1. **Add Chinese to your localized attributes**
Include the Chinese locale (e.g., cmn) in your [localizedAttributes](https://www.meilisearch.com/docs/reference/api/settings#localized-attributes).

2. **Update your stop-word list or dictionary**
Make any change to your [stop words](https://www.meilisearch.com/docs/reference/api/settings#update-stop-words) or custom [dictionary](https://www.meilisearch.com/docs/reference/api/settings#update-dictionary) entries. 

3. **Refresh the searchable attribute**
Temporarily remove an attribute from [searchable attributes](https://www.meilisearch.com/docs/reference/api/settings#update-searchable-attributes) list, then re-add it once the removal task has completed.

## Contributors shout-out

We want to give a massive thank you to the contributors who made this release possible. Special thanks to [CodeMan62](https://github.com/CodeMan62), [shaokeyibb](https://github.com/shaokeyibb), [HDT3213](https://github.com/HDT3213), [shu-kitamura](https://github.com/shu-kitamura), [DanasFi](https://github.com/DanasFi), [vuthanhtung2412](https://github.com/vuthanhtung2412), and [ZeroZ-lab](https://github.com/ZeroZ-lab) for their contributions to [Meilisearch](https://github.com/meilisearch/meilisearch); to [luflow](https://github.com/luflow), [mosuka](https://github.com/mosuka), and [HDT3213](https://github.com/HDT3213) for their work on [Charabia](https://github.com/meilisearch/charabia); and to [lucascool12](https://github.com/lucascool12) and [Dr-Emann](https://github.com/Dr-Emann) for their efforts on [RoaringBitmap](https://github.com/RoaringBitmap/roaring-rs).

---

And that’s a wrap for v1.15! These release notes only highlight the most significant updates. For an exhaustive listing, read the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.15.0) on GitHub.

---

For more information, subscribe to our [monthly newsletter](https://dub.sh/meili-newsletter?utm_campaign=release-v1-15&utm_source=blog), or join our [Product Discussions](https://github.com/orgs/meilisearch/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/?utm_campaign=release-v1-15&utm_source=blog).

Meilisearch 1.15

PHP combined with Meilisearch helps you create a search engine that’s accurate and straightforward to implement. Its capabilities include support for full-text search and typo tolerance, so you get lightning-quick responses every time. 

This guide is your go-to resource for setting up the engine, sending queries, and returning precise results using the official Meilisearch PHP SDK. Additionally, you will learn to avoid the complex logic of heavy configurations. 

Here’s what we will cover:

* How to set up Meilisearch in your PHP project
* How to index data and implement full-text search
* How to customize for UX through filters, sorting, and analytics 

This [step-by-step guide](https://www.meilisearch.com/blog/php-search-engine#set-up-meilisearch-in-your-php-project) is your answer whether you’re looking for a simple PHP script, web app, or even integration with Laravel. It outlines everything required to create a top-tier search experience that fits into your back end like a glove.


## 1. Set up Meilisearch in your PHP Project

The first step in setting up Meilisearch in your PHP project is to install the `Meilisearch PHP SDK`. This is also required to build a high-performance PHP search engine. Once you install the SDK, your PHP will get backend access to Meilisearch’s API.

Now, you can manage documents and run full-text searches easily. Additionally, it works smoothly with Laravel, SQL databases like MySQL, or a custom PHP script on any web app. Compatibility will never be the issue. 

Let’s get into the technicalities of this step.

Install the required packages using `Composer`. Here’s how to do it with Guzzle:


```
composer require meilisearch/meilisearch-php guzzlehttp/guzzle http-interop/http-factory-guzzle:^1.0
```


Then, create a connection using your Meilisearch host and API key:


```
require_once __DIR__ . '/vendor/autoload.php';

use Meilisearch\Client;

$client = new Client('http://127.0.0.1:7700', 'masterKey');
```


This connection we have formed is the core of your PHP search engine’s functionality. The features we highlighted, such as autocomplete, filters, [typo tolerance](https://www.meilisearch.com/blog/typo-tolerance), and fast full-text search, result from this connection. Now, the client will be initialized. Once it does, you can easily begin indexing content or organizing your data. 

Compatibility with Laravel, MySQL, and SQLite is what makes Meilisearch a go-to for content handling in modern PHP workflows. 

If you want to see what kinds of PHP projects can benefit from Meilisearch, check out [these real-world examples](https://www.meilisearch.com/blog/php-search-engine#examples).


## 2. Add Documents to Your PHP Search Engine

To add documents to your PHP search engine, verify that your PHP client is connected, as shown in the last step. These documents are often stored in JSON and serve as your primary data source for the search engine.

Each can include searchable fields like titles, categories, or metadata. Meilisearch’s magic handles this messy input, even with typos, and gives you the best results across web pages and backend APIs. 

Here’s how to add documents using a PHP script:


```php
$index = $client->index('movies');

$documents = [
  ['id' => 1, 'title' => 'Inception', 'genres' => ['Action', 'Sci-Fi']],
  ['id' => 2, 'title' => 'Interstellar', 'genres' => ['Adventure', 'Drama']]
];

$index->addDocuments($documents);
```


This step ensures that key features like text search, filtering, typo tolerance, and autocomplete are now functional. It works with any option, including Laravel, MySQL database, or a standalone PHP script. Remember, you won’t get accurate results from your PHP search engine if you don’t add the documents. 

You can visit our [multi-language guide](https://www.meilisearch.com/blog/php-search-engine#other-languages) to see how it compares with other languages.


## 3. Build the Search Function and Return Results

Now that your search engine has the documents that it needs to return accurate results, we will shift to writing the PHP function that performs the actual search.

How does it work? The function will take any user’s input and send a request to Meilisearch. From here, Meilisearch will return matching results that you are looking for. This feature is the heart of any PHP engine. 

Here’s an example of a basic search function using the Meilisearch PHP library:


```php
$results = $index->search('Interstellar')->getHits();

foreach ($results as $hit) {
  echo '<h2>' . $hit['title'] . '</h2>';
}
```


This is a prime example of how PHP and HTML work together to return the relevant results from the Meilisearch API. To format the output, we will use CSS. We will rely on caching and integrate it into a Laravel or MySQL-based web application for optimization. 

You can build a [PHP search engine for free](https://www.meilisearch.com/blog/php-search-engine#free), but some features, like advanced analytics, may require Meilisearch Cloud.


## 4. Add Filters and Facets for Smarter Search

Keyword matching is only the first element of building a real search engine in PHP. You need much more, including filters. Using Boolean conditions, Meilisearch lets your users narrow down their search based on filters, such as category, tag, or even custom fields. 

Before this filtering, you need to define which fields are filterable:


```php
$index->updateFilterableAttributes([
  'genres',
  'id'
]);
```


Now, you can run filtered searches using SQL-style logic:


```php
$index->search('drama', [
  'filter' => ['genres = Drama AND id > 1']
]);
```


This search functionality is important for any PHP script powering a blog, forum, or ecommerce site. It saves time and helps users find what they are looking for. To make your results even more dynamic, you can always count on formatting in HTML or improving markup with CSS. Don’t forget to connect the output to a MySQL database as well.

These filters are a straight-up bonus for your the SEO and usefulness of your PHP search engine.


## 5. Track User Behavior with Search Analytics

Analytics rules the world of data; our case is no different. Once the PHP engine is live, you must constantly measure results to improve it. 

Meilisearch Cloud has built-in analytics to track various metrics, including top queries, zero-search results, user search patterns, etc. But if you’re self-hosting, you must track these metrics using PHP, JSON logging, or browser-side JavaScript.  

These insights will help you improve the overall search experience through advanced data-driven changes and customization. For example, if users keep typing terms that don’t match your content, you might need to improve your SQL or update your search algorithm. You get instant improvements in SEO and user experience. 

Tracking doesn’t need to be basic. You can improve it by combining it with the data from your MySQL database. The plugins in Laravel apps gladly sync with your analytics. So, whether working with [full-text search](https://www.meilisearch.com/blog/how-full-text-search-engines-work) or [faceted filters](https://www.meilisearch.com/blog/nested-hierarchical-facets-guide), analytics helps you deliver more relevant results to users.


## What are some examples of PHP search engine uses?

A PHP search engine can be a basic text box, or more – the choice is yours. For instance, it can be built with tools like Meilisearch PHP, and it becomes an unstoppable machine that finds content across all types of applications. These could include small personal sites or large eCommerce brand platforms. 

Let’s have a look at where it’s most commonly used:


1. **App Search:** Developers often use PHP to supercharge the search functionality inside web applications. With Meilisearch, you can improve your search navigation in internal dashboards, customer portals, or even complex tools to generate more relevant results. 
2. **Site Search:** Do you have static content that needs to be scanned? A PHP script can power a custom site search that will do just that. From blogs to a complex documentation hub with SQL-backed content or even Markdown rendered with PHP, this custom site search will do it all.
3. **eCommerce Search:** PHP enables top-tier customization that helps users find items faster and more accurately through filters for categories, products, brands, pricing, etc.
4. **Forum Search:** Communities built in PHP (such as phpBB or custom Laravel forums) often struggle with slow or limited search functionality. A dedicated engine improves how users find threads or replies.
5. **Media Search:** You can build a PHP searchable video or image archive, tagging assets with keywords stored in a MySQL database and filtering them through a search API.
6. **Document Search:** Many backend systems use PHP for uploading and managing files. With the right indexing logic, users can search by file names, metadata, or even parsed content.

If you wish to see more examples, you can view them in our [PHP search engine examples](https://www.meilisearch.com/blog/php-search-engine#examples) section.


## Which languages besides PHP are used to build a search engine?

While building a PHP search engine is a popular approach, developers often choose other languages based on their project’s architecture, database integration, and overall search engine functionality.



* **JavaScript** is often used to build client-side search experiences in SPAs or modern web apps. With frameworks like React or Vue, you can integrate real-time search UIs powered by APIs or hosted engines.[ ](https://www.meilisearch.com/blog/javascript-search-engine)Learn how to build a [search engine in JavaScript](https://www.meilisearch.com/blog/javascript-search-engine).
* **Python** is a flexible scripting language ideal for crawling, indexing, and building logic-heavy search pipelines. It’s often used in AI-based semantic search or research-focused projects. Learn how to build a [search engine using Python](https://www.meilisearch.com/blog/python-search-egine).
* **Golang** offers concurrency and performance benefits that suit high-traffic search systems. It’s often used for scalable backend search services that demand speed.[ ](https://www.meilisearch.com/blog/golang-search-egine)Learn how to build a [search engine with Golang](https://www.meilisearch.com/blog/golang-search-egine).

Each of these programming languages is fully compatible with open-source tools like Meilisearch. They allow you to build scalable, high-performance search engines suited to different project needs.


## Can I build a search engine in PHP for free?

Yes, you can build a functional [PHP search engine without licensing costs](https://www.meilisearch.com/blog/php-search-engine#free). The Meilisearch PHP SDK is open-source and can be hosted on a Linux server or a local machine. It supports JSON, HTML, and XML, and works with Laravel, MySQL, or standalone PHP scripts.

However, production-grade systems may incur additional costs. These include third-party plugin dependencies, hosting expenses, API rate limits, and advanced search features such as analytics or autocomplete, which may require Meilisearch Cloud. This setup is effective for beginners and adaptable for long-term scalability.


## Built a high-functioning PHP search engine with Meilisearch

Meilisearch delivers essential search functionality for PHP applications. It represents features like full-text search, filters, typo tolerance, and autocomplete. It integrates easily with Laravel, MySQL, and JSON workflows. Naturally, this makes it ideal for e-commerce search boxes, forums, and custom crawlers.

How to build a search engine in PHP: Step-by-step guide

Here's a recap of everything that happened this month!

## 🔔 Reminder to upgrade

[![meilisearch-1.14.png](https://unable-actionable-car.media.strapiapp.com/meilisearch_1_14_166a7ff64e.png)](https://www.meilisearch.com/blog/meilisearch-1-14)

Don't forget to upgrade to Meilisearch 1.14 to benefit from improved AI search with composite embedders and embedding cache. v1.15 is coming soon!

[Read the release notes](https://www.meilisearch.com/blog/meilisearch-1-14)

## 📰 In the news

### 💬 Talk to Meilisearch with Claude desktop

Connect your LLM to Meilisearch using our MCP server. 

[Read the guide](https://www.meilisearch.com/docs/guides/ai/mcp)

### 🚀 How we're making AI work at Meilisearch

How we turned scattered AI usage into systematic success for our teams.

[Read the blog post](https://www.meilisearch.com/blog/making-ai-work)

### 🌱 Six years working with the open-source community

Our Head of Engineering Clémentine reflects on our experience working in OSS.

[Read the blog post](https://blog.curqui.com/six-years-working-with-the-open-source-community)

### 📣 How are CTOs spending their AI budgets?

You can help us understand you better by taking this survey!

[Take the survey](https://s.surveyplanet.com/befuwdt7)

### 📣 How long does it take to integrate search?

You can help us understand you better by taking this survey!

[Take the survey](https://s.surveyplanet.com/sjyv6prl)

## 🤗 Made with Meilisearch

HuggingFace's [Daily Papers](https://huggingface.co/papers) uses Meilisearch hybrid search to search across hand-picked machine learning papers published over the last two years.

[![huggingface-papers.gif](https://unable-actionable-car.media.strapiapp.com/huggingface_papers_fceeaf233b.gif)](https://huggingface.co/papers)

_We’re always looking to promote new projects from the community. Post yours in the [#built-with-meilisearch](https://dub.sh/meili-discord?utm_campaign=newsletter&utm_source=blog&utm_medium=may-25-updates) Discord channel!_

---

Receive this updates in your inbox by subscribing to our [monthly newsletter](https://dub.sh/meili-newsletter?utm_campaign=newsletter&utm_source=blog&utm_medium=may-25-updates). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/?utm_campaign=newsletter&utm_source=blog&utm_medium=may-25-updates) and participate in our [Product Discussions](https://github.com/orgs/meilisearch/discussions).

For anything else, join our developers community on [Discord](https://dub.sh/meili-discord?utm_campaign=newsletter&utm_source=blog&utm_medium=may-25-updates).

Meilisearch May updates

To build a fast and reliable JavaScript search engine for your website from scratch, you’ll need to

* Set up your environment
* Prepare your data
* Create an index
* Implement the search functionality

Less complicated ways to handle the task are also available. For example, Meilisearch, a lightweight, open-source, and full-text search engine solution, can provide users with immediate search results, typo tolerance, and easy installation.

Whether you build a custom solution or use solutions like Meilisearch will depend on your specific use case and unique needs.  Either way, this guide will walk you through the steps to create a functional JavaScript search engine exactly how you want it.

## 1. Set up the development environment

The search engine development process starts with creating a development environment where your JS code will run. To do this, you must establish a Node.js environment and install the necessary dependencies for the search engine logic.

The JS runtime is essential for executing code outside the web browser, such as server-side operations like indexing data or querying a search engine.

Furthermore, it is important to have a package manager as a medium for library installations; a typical example is npm. For a basic setup, Node.js should be present to start a new project. Then it's just typing a command to the terminal and done! This creates a package.json file to manage dependencies.

```
mkdir search-engine
cd search-engine
npm init -y
```



This sets the stage for later adding libraries or tools, such as Meilisearch. A proper environment ensures your code runs smoothly across the development and production phases.

Next, let’s prepare the data for searching.

## 2. Prepare and structure the data

Structuring data is the first item on your to-do list before you can search for anything. For this, you need to collect and format the content to make it searchable. This could include blog posts, FAQs, knowledge bases, product descriptions, or user profiles.

Data should be in a consistent format, typically JSON, to make indexing straightforward. Imagine that you are creating a search engine for a blog. Your data may contain titles, content, and meta tags that must be in the same format. 

Clean and normalize the data to remove inconsistencies, like different date formats or missing fields, which can affect search accuracy. This step must be done properly because well-structured data directly impacts the quality of search results — as the saying goes, 'garbage in, garbage out.'


```javascript
const sampleData = [
  { id: 1, title: "JavaScript Basics", content: "Learn the fundamentals...", tags: ["JS", "coding"] },
  { id: 2, title: "Advanced Node.js", content: "Deep dive into Node...", tags: ["Node", "JS"] }
];
```



With your data structured and ready, it’s time to create an index.


## 3. Create the index

Simply put, indexing is the process of arranging your data to make it searchable. Although the data we have at this point is structured, we need to store it properly for quick retrieval based on search queries. That’s where an index comes in: it is like a database optimized for speedy lookups. 

In a custom JavaScript search engine, you might build a simple inverted index, mapping words 1:1 to their locations in your dataset. This would entail tokenizing text (splitting it into individual words), removing stop words (like "the" or "and"), and storing the results obtained.

Even though this might be effective for small datasets, it can get tricky when the complexity of the dataset grows, resulting in issues with performance and scalability.


```javascript
function createIndex(data) {
  const index = {};
  data.forEach((doc, docId) => {
    const words = doc.content.toLowerCase().split(/\W+/);
    words.forEach(word => {
      if (!index[word]) index[word] = [];
      index[word].push(docId);
    });
  });
  return index;
}
```


Now, let’s implement the search functionality, which we’re here for.


## 4. Implement the search functionality

The final step in setting up a JavaScript search engine brings it to life by enabling users to query your search index and retrieve relevant results. The search function will analyze the user’s search term, parse the index we generated in the previous step for matches, and rank search results based on relevance and accuracy.

For a better user experience, consider including features such as partial matching or [ranking by relevance and precision](https://www.meilisearch.com/blog/filter-scoring-guide). However, this can be a drain on computational power, so algorithm optimization may help you increase performance.

```javascript
function search(query, index, data) {
  const queryWords = query.toLowerCase().split(/\W+/);
  const results = new Set();
  queryWords.forEach(word => {
    if (index[word]) {
      index[word].forEach(docId => results.add(data[docId]));
    }
  });
  return Array.from(results);
}
```



Now that we know how to create a JS search engine from scratch, let’s explore how Meilisearch simplifies this process.

## Building a JavaScript search engine for your website with Meilisearch

Meilisearch is an open-source, lightning-fast search solution designed for error-free [integration with JavaScript applications](https://www.meilisearch.com/with/javascript). It’s both developer—and user-friendly, with features like [typo tolerance](https://www.meilisearch.com/blog/typo-tolerance), [faceted search](https://www.meilisearch.com/blog/nested-hierarchical-facets-guide), and [federated search](https://www.meilisearch.com/blog/what-is-federated-search) for searching across multiple indices. These features, and more, make it ideal for building optimal search experiences for web pages, apps, or e-commerce platforms.

Unlike building from scratch, though, Meilisearch handles the heavy lifting of indexing and searching for you. Significant benefits of using Meilisearch include:

* **Speed:** Delivers almost instant search results, even with large datasets.
* **Typo tolerance:** Handles misspellings and typos gracefully with [fuzzy search](https://www.meilisearch.com/blog/fuzzy-search), letting users get to their intended results faster.
* **Easy integration:** Works seamlessly with JavaScript via its SDK.
* **Customizable:** Allows fine-tuning of search relevance and filters.

<Callout intent="tip">

#### Improve user experience with the power of Meilisearch + JS

Our tool is built for speed, simplicity, and performance. Pair that with the power of JavaScript, and you have a search engine that outperforms any other tool-language combination.

[Try Meilisearch today!](https://www.meilisearch.com/cloud)

</Callout>

Here’s how you can use Meilisearch to build a JavaScript search engine for your website.

### Step 1: Install Meilisearch and the JavaScript SDK

Start by setting up a Meilisearch instance locally (e.g., at http://127.0.0.1:7700) or via a cloud provider. Then, install the core Meilisearch JavaScript SDK for server-side operations and, optionally, the instant search package for front-end integration.

```
npm i meilisearch
npm install @meilisearch/instant-meilisearch
```



The `meilisearch `package provides the core functionality for indexing and querying, while `@meilisearch/instant-meilisearch` simplifies front-end search interfaces for instant search capabilities.


### Step 2: Initialize the client and index your data

Create a MeiliSearch client to connect to your instance and index your data. As mentioned, an index is a container for your searchable documents, and the client facilitates all interactions with the Meilisearch server. Use the SDK to send your JSON-formatted data to the Meilisearch server.


```javascript
import { MeiliSearch } from "meilisearch";

const client = new MeiliSearch({
  host: "http://127.0.0.1:7700",
  apiKey: "masterKey",
});

const data = [
  { id: 1, title: "JavaScript Basics", content: "Learn the fundamentals..." },
  { id: 2, title: "Advanced Node.js", content: "Deep dive into Node..." }
];

async function addDocuments() {
  try {
    const index = client.index("documents");
    await index.addDocuments(data);
    console.log("Documents added successfully");
  } catch (error) {
    console.error("Error adding documents:", error);
  }
}
addDocuments();
```



This code initializes the client with your Meilisearch instance’s host and API key, then adds JSON documents to an index named `documents`. It ensures your data is indexed and ready for searching. For this to work properly, ensure your Meilisearch server is running and accessible at the specified host.


### Step 3: Implement client-side search

Integrate Meilisearch with React using `@meilisearch/instant-meilisearch` and `react-instantsearch-dom` to create an interactive search interface. This package integrates with your front-end, providing a pre-built search bar and results display while connecting to the same Meilisearch instance for real-time search.


```javascript
import { instantMeiliSearch } from "@meilisearch/instant-meilisearch";
import { SearchBox, Hits } from "react-instantsearch-dom";

const searchClient = instantMeiliSearch("http://127.0.0.1:7700", "masterKey");

function App() {
  return (
    <div>
      <SearchBox />
      <Hits />
    </div>
  );
}
```



Meilisearch’s speed, simplicity of setup, and robust performance make it a powerful choice for adding search functionality to your website. 

## What are some examples of JavaScript search engine uses?

Search engines power various applications, empowering users to find the knowledge they seek by making content more discoverable. Meilisearch, in particular, excels in several use cases due to its flexibility and performance.

**App search**: Mobile, desktop, or web applications, such as note-taking or task management tools, utilize search functionalities to help users quickly find specific entries. Meilisearch’s typo tolerance ensures users can find accurate results even with minor misspellings.

**Site search**: Websites like blogs or documentation portals rely on search to navigate vast content libraries. Meilisearch powers fast, relevant site search for platforms like intra-team technical docs or wikis.

**eCommerce search**: Online stores and storefronts use search functionalities to help customers find products by name, category, or description. With capabilities like filtering and faceting, Meilisearch allows users to further refine searches by the object’s type, price, brand, or customer ratings.

**Customer support portals**: Corporations integrate search functionality into customer help centers to increase consumer independence, allowing users to find articles or FAQs independently. Meilisearch’s speed and [autocomplete functionality](https://github.com/meilisearch/meilisearch-js-plugins/) ensure a quick and easy problem-solving path.

These use cases illustrate the versatility and applicability of JavaScript search engines, especially when powered by tools like Meilisearch.

### Which languages besides JS can be used to build a search engine?

Aside from JavaScript, other common programming languages you can use to build a search engine are:

* **PHP**: Used for interactive and dynamic web development, PHP can integrate seamlessly with search tools like Meiliseach. Learn more about building a [search engine in PHP](https://www.meilisearch.com/blog/php-search-engine) through tutorials on integrating such tools.
* **Python**: Offering simplicity and adaptable libraries, Python is a top choice for building a custom or tool-powered search engine. Explore scalable search solutions by learning how to build a [search engine in Python](https://www.meilisearch.com/blog/python-search-engine).
* **Golang**: With unmatched performance and concurrency, Go is ideal for high-speed search engines, particularly for backend optimization. Check out how to build a [search engine using Golang for efficient and lean systems](https://www.meilisearch.com/blog/golang-search-engine).

Others like HTML, CSS, and SQL might also work for you. Each language brings specific strengths that are better suited for specific use cases. Thus, choose your language based on your project’s needs.

### Can I build a JavaScript search engine for free?

Yes, you can build a JavaScript search engine for free, that is, without licensing costs. Open-source tools like Meilisearch have free, self-hosted options, and you can use libraries like the [Meilisearch JavaScript SDK](https://meilisearch.github.io/meilisearch-js/) at no cost.

However, you may incur costs using cloud-hosted Meilisearch plans or making extensive API calls in production. Furthermore, advanced features like AI analytics also come at a price. Hence, to avoid surprises, keep your options sorted by budgeting for cloud services and advanced features.

### Are there GitHub repositories for JavaScript search engines?

Several repositories exist for JavaScript search engines, including the Meilisearch JavaScript SDK: [/meilisearch-js](https://github.com/meilisearch/meilisearch-js).

The Meilisearch repo provides everything you need to integrate Meilisearch into JavaScript projects, including client libraries and code examples. It’s actively maintained and well-documented, so you and developers can use it for quick setup.

### Can you make a search engine in React?

Yes, you can build a search engine in React. We recommend that you do.

React’s component-based architecture pairs well with Meilisearch’s Instant Meilisearch package, which offers pre-built modular components like search bars and hit lists. This setup enables you to create a responsive and interactive [search experience in React](https://www.meilisearch.com/blog/react-search-engine) with minimal effort.


Building a JavaScript Search Engine: Tutorial, Examples & More

You can make a search engine in Python using a combination of data structures, algorithms, and libraries to index, rank, and retrieve information based on your search query input.

A Python search engine works based on these key building steps:

1. Collect data & preprocessing
2. Document creation and indexation
3. Add a search system
4. Rank results.

A search engine built with Python can be used by small-scale and large-scale enterprises that opt for open-source solutions that offer scalability and flexibility.

Due to its easy customization, these search engines can be applied in many applications, such as e-commerce, research, marketplaces, enterprise search, and more.

Let’s examine in more detail the different steps to building a simple search engine from scratch with Python.

## 1. Collect data & preprocessing

Data collection can be done in several ways. You may need to web scrape content from HTML webpages with packages such as `beautifulsoup`, or directly connect your script to Google Sheets using `gspread`.

Perhaps you already have a database and only need to use a PostgreSQL Python connector to access it and query the data.

The Python code for data collection can vary a lot depending on your needs, but here is an example of how to use Beautiful Soup for web scraping. First, you need to install the package:

```
pip install beautifulsoup4
```

Here’s the code example:

```pyhton
import requests
from bs4 import BeautifulSoup

url = "https://example.com"
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

# Extract data
for item in soup.select('.item', limit=5): 
    title = item.find('h2').text
    link = item.find('a')['href']
    print(f"{title}: {link}")
```


Once the data is collected, it needs to be preprocessed. This step can occur before and after document indexation. Several preprocessing trials may be needed to ensure the dataset is correctly indexed and ready for optimal retrieval.

For instance, text sources that contain emojis, emails, and source links can be cleaned beforehand to avoid adding unnecessary information to the system. 

In paragraphs, punctuation and stop words can be removed, and the sentences can be converted to lowercase.

With Python, several packages can be utilised for data parsing, cleaning, and preprocessing. Let’s take a look at the NLTK library and use it to remove emojis, emails, and punctuation:

First, install the package:

```
pip install nltk
```


Now you can try the following script:


```python
import re
import string
from nltk.tokenize import word_tokenize

# Sample text with punctuation, emojis, and emails
text = """
Hello! 😊 This is a test email@example.com. 
Can you remove this? 👍 Also, check info@domain.org! 
"""

def clean_text(text):
    # Step 1: Remove emails
    text = re.sub(r'\S+@\S+', '', text)
    
    # Step 2: Remove emojis and symbols
    emoji_pattern = re.compile(
        "["
        "\U0001F600-\U0001F64F"  # Emoticons
        "\U0001F300-\U0001F5FF"  # Symbols & pictographs
        "\U0001F680-\U0001F6FF"  # Transport & map symbols
        "\U0001F700-\U0001F77F"  # Alchemical symbols
        "\U0001F780-\U0001F7FF"  # Geometric Shapes Extended
        "\U0001F800-\U0001F8FF"  # Supplemental Arrows-C
        "\U0001F900-\U0001F9FF"  # Supplemental Symbols and Pictographs
        "\U0001FA00-\U0001FA6F"  # Chess Symbols
        "\U0001FA70-\U0001FAFF"  # Symbols and Pictographs Extended-A
        "\U00002702-\U000027B0"  # Dingbats
        "\U000024C2-\U0001F251" 
        "]+", 
        flags=re.UNICODE
    )
    text = emoji_pattern.sub('', text)
    
    # Step 3: Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Step 4: Tokenize and rejoin (optional, removes extra whitespace)
    tokens = word_tokenize(text)
    cleaned_text = ' '.join(tokens)
    
    return cleaned_text

# Clean the text and make it lowercase
cleaned_text = clean_text(text).lower()
print("Original Text:\n", text)
print("\nCleaned Text:\n", cleaned_text)
```


In some cases, Natural Language Processing (NLP) could be required. Take, for instance, a list of companies like “Impossible Foods” and Impossible Foods Co.” Both are the same companies, so you can use NLP to vectorize the words and unify the results in one single term based on their cosine similarity.

Sometimes, excessive preprocessing can be a problem, resulting in information loss. Therefore, the best approach is to start with simple steps. 

Once the documents are indexed, they can be updated with new preprocessed information.


## 2. Document creation and indexation

Documents are units of information (e.g., text, JSON, images, or structured/unstructured data) processed and stored on an index.

This step involves gathering the data sources you want to index in your vector database and converting them to documents.

For instance, if you use a JSON format as input, you can use LangChain Python’s framework to convert it to a list of documents directly.

First, you need to install the LangChain package in your system:


```
pip install langchain 
```


Secondly, you need to import the JSONLoader class and apply the following script:


```python
from langchain_community.document_loaders import JSONLoader

loader = JSONLoader(
    file_path='./my_data.json',
    jq_schema='.messages[].content',
    text_content=False)

data = loader.load()
```


The output should be a list of documents that look like this:


```python
[Document(page_content='Bye!', 'seq_num': 1}), Document(page_content='Hello', 'seq_num': 2}),
Document(page_content='See you later', 'seq_num': 3})]
```


Once the documents are created, they can be added to a vector database such as  [Chroma db](https://docs.trychroma.com/docs/overview/getting-started). 
 
Meilisearch’s [Python SDK](https://github.com/meilisearch/meilisearch-python) simplifies the process by eliminating the need to convert your source data into documents or search for a database solution. You can directly add the JSON or CSV files to an index on Meilisearch’s vector database.

Like LangChain, you first need to install the Meilisearch package on your machine:


```
pip install meilisearch
```


Then, you need to create an index with this simple command:


```python
client.create_index('books', {'primaryKey': 'id'})
```


To add the documents to the index, you can use the JSON format like this:

```json
client.index('movies').add_documents([{
  'id': 287947,
  'title': 'Super Gut',
  'author': 'Dr. William Davis',
}])
```


With the same package, you can also update the documents, apply filters, and delete them by simply changing the function:


```json
# apply filter
client.index('books').update_filterable_attributes(['author'])

# update documents
client.index('books').update_documents(<list_of_documents>)

# delete documents
client.index('books').delete_all_documents()
```


With Meilisearch's Python SDK, multiple indexes can be added, and all functions are easy to follow and implement.

Some examples are available on our [GitHub repository;](https://github.com/meilisearch/meilisearch-python/tree/main) alternatively, you can also refer to the [API documentation](https://www.meilisearch.com/docs/reference/api/overview) for more information.


## 3. Add a search system

If you’re using the LangChain approach with a custom vector database, you must embed your documents with Deep Learning (DL) algorithms. This creates a vector representation of your data, allowing for vector search, hybrid search, semantic search, and more.

Several embedding models are available on Hugging Face and through the OpenAI API.

Let’s, for instance, use the OpenAI embedding model with LangChain and Chroma as a vector database. You first need to install these packages:


```
pip install langchain-chroma
pip install langchain-openai
```


Export your OpenAI key and add the following:


```python
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings

db = Chroma.from_documents(documents, OpenAIEmbeddings())
```


The command above embeds the documents with the OpenAIEmbeddings class and creates an index in the Chroma database. Now you can query the `db` instance:


```python
query = "Find a book about Nutrition"
docs = db.similarity_search(query)
print(docs[0].page_content)
```


All the above steps can be converted into one with [Meilisearch’s Python SDK](https://www.meilisearch.com/docs/reference/api/search). 

There’s no need to add embeddings or to find a package for your vector database. All you need is to search directly on the index previously created with the following function:


```
client.index('books').search('Find a book about Nutrition')
```


But it does not stop here, you can add filters like this:


```python
client.index('books').search('nutrition', {
  'filter': ['author = "Dr. William Davis"']
})
```


Or create a [faceted search](https://www.meilisearch.com/docs/reference/api/settings#faceting):


```python
client.index('movie_ratings').search('nutrition', {
  'facets': ['authors', 'rating']})
```


You can experiment with the API using other search options, such as specifying the number of documents to retrieve, [querying by locale](https://www.meilisearch.com/docs/reference/api/search#hybrid-search), or implementing hybrid search. 


## 4. Rank results

Ranking results can take complex Machine Learning (ML) algorithms, which are usually integrated in the Python libraries, so the good news is that you don’t need to create them from scratch.

For instance, Chroma uses an Approximate Nearest Neighbor (ANN) algorithm called Hierarchical Navigable Small World (HNSH) to find similar documents.

If you want to get their score and order them, you can run the following:


```python
results = docs.similarity_search_with_score(query="Find a book about Nutrition")
for doc, score in results:
    print(f"{score}: {doc.page_content}"}
```


However, this [documentation](https://api.python.langchain.com/en/latest/chroma/vectorstores/langchain_chroma.vectorstores.Chroma.html) has quite limited ranking results. A better approach is to use Meilisearch ranking rules, which are more straightforward.

By default, these are the ranking rules that can be tweaked:

* **"words”:** Sorts results by decreasing number of matched query terms
* **"typo":** Sorts results by increasing number of typos
* **"proximity":** Sorts results by increasing distance between matched query terms
* **"attribute":** Sorts results based on the attribute ranking order
* **"sort":** Sorts results based on parameters decided at query time
* **"exactness":** Sorts results based on the similarity of the matched words with the query words.

We can already see that the ranking mechanism can go beyond simply similarity. To rank the results, all you need to do is change the order of these terms in the query according to your needs:

```python
client.index('movies').update_ranking_rules([
    'typo',
    'words',
    'sort',
    'proximity',
    'attribute',
    'exactness',
    'release_date:asc',
    'rank:desc'
])
```


You can now search with a certain number of results (limit), and their relevance will be based on the updated ranking order.

This function is much simpler to implement and takes into account many other rules. Meilisearch streamlines the ranking process without the need to explore multiple libraries or create ranking algorithms from scratch.


## Can I make a search engine in Python for free?

You can use Python frameworks, such as LangChain, paired with open-source vector databases like Chroma. However, this strategy has limitations, such as limited ranking mechanisms, and requires additional steps and preprocessing.

For ease of implementation, more customization, and fast document retrieval, the best approach is to use [self-hosted Meilisearch](https://www.meilisearch.com/docs/learn/self_hosted/getting_started_with_self_hosted_meilisearch). You can use your machine or a VPS, which comes with a price.

You can also have free access to Meilisearch Cloud with a [14-day trial](https://www.meilisearch.com/cloud).


## What are the best open-source search engines for Python?

The best open-source search engines have comprehensive documentation and a large community of developers who share their issues and achievements.

Open-source search engine tools should also be easy to set up and provide examples for the community. These are the three main open-source platforms that support Python:


## Meilisearch

[Meilisearch](https://cloud.meilisearch.com/) is an open-source, lightning-fast search engine designed for developers and enterprises seeking to embed intuitive, scalable search experiences into applications through its RESTful API. 

Focusing on simplicity and performance, it provides advanced features such as typo tolerance and faceted search.

The [documentation](https://www.meilisearch.com/docs/reference/api/overview) is clear, easy to follow, and has examples. There’s a [Discord](https://discord.com/invite/meilisearch) group for developers to share their work or find solutions, and a well-structured [GitHub](https://github.com/meilisearch/meilisearch) repository.


## Qdrant

[Qdrant](https://qdrant.tech/) is an open-source vector database and vector search engine built in Rust. It efficiently handles similarity searches on high-dimensional vectors, making it ideal for tasks like recommendation systems, semantic search, and anomaly detection.

The Qdrant’s RESTful API supports multiple languages, including [Python](https://python-client.qdrant.tech/). The documentation is vast and can get overwhelming when you need to find the right steps to build a Python search engine. However, it also provides code examples, a [GitHub](https://github.com/qdrant/qdrant-client) repository, and a [Discord](https://discord.com/invite/tdtYvXjC4h) community.


## Elasticsearch

Elasticsearch is an open-source, distributed search and analytics engine with a scalable data store and vector database for various use cases.

The Python client for Elasticsearch is [well documented](https://www.elastic.co/docs/reference/elasticsearch/clients/python/getting-started) and provides the right tutorials to start building a search engine seamlessly.

They have a [GitHub](https://github.com/elastic/elasticsearch-py) repository where you can find examples, more information about the Python SDK, and a [ticket system](https://github.com/elastic/docs-content/issues/new?template=issue-report.yaml&link=https://www.elastic.co/docs/reference/elasticsearch/clients/python/getting-started&labels=source:web) on GitHub to address issues.

Elasticsearch also provides a [Python DSL](https://www.elastic.co/docs/reference/elasticsearch/clients/python/elasticsearch-dsl) module that aims to help with writing and running queries against Elasticsearch in a more convenient and idiomatic way.


## What programming languages besides Python are used to build search engines?

Python is not the only programming language that allows you to build AI-powered search engines. Some common programming languages you can use are:


* **JavaScript:** Learn how to [build a search engine in Javascript](https://github.com/meilisearch/meilisearch-js). 

* **PHP:** Learn how to [build a search engine in PHP](https://github.com/meilisearch/meilisearch-php).

* **Golang:** Learn how to [build a search engine with Golang](https://github.com/meilisearch/meilisearch-go).


## High-performance Python search engines with Meilisearch

While setting up the Python search engine, we realised the importance of having a unified package with its vector database, seamlessly embeds the documents, and provides tools for easy filtering and ranking results.

Using multiple libraries and Python frameworks can be overwhelming, leading to more problems than solutions. The frameworks can be limited in what you are allowed to do, which may force you to switch to another one or build from scratch, both of which can be time-consuming and resource-intensive.

Clarity, good documentation, and ease of use are the keys to building high-performing Python search engines. You don’t need to master several frameworks, libraries, and look for information in countless forums and YouTube videos.


Meilisearch latest news and company updates