What is semantic search? How it works, use cases & more

Semantic search is a text search method that focuses on understanding the search intent and contextual meaning behind a user’s query rather than simply matching keywords.

Semantic search leverages machine learning (ML) and natural language processing (NLP) techniques and incorporates processes like query analysis, vector embeddings, and knowledge graphs to deliver highly relevant results.

This technology is widely used across industries, including e-commerce, search engines, and internal site searches, to improve product discovery, content retrieval, and user satisfaction.

Semantic search differs from vector search by using a broader range of approaches, including text embeddings, vector search approaches, and knowledge graphs, to infer user intent. Unlike keyword search, which relies on exact matches, semantic search analyzes semantics and context to deliver more meaningful and relevant results.

This article thoroughly explores semantic search, from its foundational concepts to its workings, key applications, and benefits over traditional search methods. Whether you are a developer, ML professional, or CTO, it provides a comprehensive understanding of semantic search's theoretical and practical aspects.

What is semantic search?

Semantic search is an approach for searching data based on a query's intent and contextual meaning. It goes beyond keyword matching and delivers more relevant results than lexical or syntactic search methods.

Semantic search is a vital search method as it bridges the gap between how humans think and how machines understand natural language. Human intent often depends upon the context and semantic information not expressed in words. Semantic search tries to identify the intent and semantic information and help machines respond more accurately to user queries.

Semantic search differs from traditional search methods like syntactic and lexical matching. Unlike syntactic matching, semantic search doesn't rely solely on keyword matches. Instead, it uses advanced approaches, such as contextual word embeddings, vector similarity, knowledge graphs, etc., to infer a query's semantic meaning.

For example, if a user searched for ‘affordable smartphones with good cameras,’ a syntactic search engine would only search for the keywords ‘affordable,’ ‘good camera,’ etc.

On the other hand, a semantic search will understand that you want a budget phone with good camera results. It would prioritize the results recommending budget phones with excellent camera features even if the results do not contain exact keywords such as ‘affordable’ and ‘good camera.’

Now that you know what semantic search is, let’s see how it works to retrieve the most relevant results.

How does semantic search work?

Semantic search involves one or more of the following steps:

• Query analysis: Semantic search breaks down a query into its fundamental components, including keywords, entities, and phrases. Query analysis involves NLP approaches like tokenization, part-of-speech tagging, and named entity recognition. For example, in the query 'best laptops for games,' NLP identifies 'laptops' as the primary entity and 'games' as the intent driver. The search can infer intent beyond the literal query by understanding relationships, such as the need for high memory, processing power, and GPU capabilities for gaming.
• Vector embeddings: Keywords and phrases identified via query analysis are projected into high-dimensional vector spaces using vector embedding approaches. Vector embedding involves advanced deep learning models such as BERT and word2vec to generate numeric representations of texts. With vector embeddings, similar words like ‘gaming’ and ‘GPUs’ are placed together in higher-dimensional space, enabling semantic understanding.
• Calculating vector similarities: In this step, the high-dimensional embedding vectors for queries and documents are matched using techniques like cosine similarity. These techniques measure the angle between vectors in the high-dimensional space, identifying documents that align most closely with the query’s intent. This ensures that results are not limited to exact keyword matches but include conceptually related documents.
• Knowledge graphs: A knowledge graph creates a structured network containing semantic information. It represents textual information as nodes and edges in a graph. While vector embeddings capture semantic similarity, knowledge graphs add a layer of context and reasoning by linking related concepts. For example, a knowledge graph could connect 'laptop' to 'processor,' 'RAM,' and 'GPU,' establishing relationships that enhance query understanding. Graph databases such as Neo4j are often used to implement knowledge graphs for semantic search.
• Re-ranking results: The documents retrieved via vector-similarity search or knowledge graphs are re-ranked based on semantic relevance, user context, and preferences, ensuring that the most relevant responses are displayed first. Recommender systems and ranking algorithms are often used in re-ranking steps.

What are the use cases of semantic search?

Semantic search finds its applications in versatile industries ranging from e-commerce and search engines to healthcare and education.

The following image shows the major use cases of semantic search.

We will discuss three of the major semantic search use cases in the following sections.

E-commerce

Semantic search improves product discovery in e-commerce applications by better understanding the meaning of the query. It can help retrieve the most accurate and relevant results even if the search terms are not exact.

For example, for a query like 'durable hiking backpacks,' semantic search will retrieve results for backpacks built with rugged materials, reinforced stitching, and multiple compartments, even if the keywords don't precisely match.

Platforms like Meilisearch integrate semantic search in e-commerce applications, returning lightning-fast results to search queries, improving customer experience, and boosting sales.

Search engines

Semantic search has revolutionized how search engines retrieve and rank results against user search queries.

For example, if a user searches 'best places to hike near me,' a traditional search engine will return results that contain the keywords in the query.

On the other hand, a search engine powered by semantic search will understand that the user wants to know about the hiking places close to the user’s geographical location, with good user reviews and activities like camping and sightseeing.

Source

Semantic search is also great for powering recommendations. In the above example, the search engine understands that “Marvel” is connected to both the Marvel Cinematic Universe and the genre of comic book adaptation movies, so it can recommend examples of both to the user.

Internal site search

Semantic search enhances internal site search by helping users find the most relevant information quickly.

For example, an employee in a large organization may search for 'annual leave policy' on a company’s intranet. A traditional search might return results with the keywords 'annual,' 'leave,' and 'policy,' even if these results do not contain any information about the annual leave policy.

A semantic search engine, however, understands the user’s intent to search for HR documents detailing leave policy and will return the corresponding HR documents.

What are some examples of semantic search?

Most advanced applications that require search capabilities embed semantic search to provide the most relevant results, improving customer experience.

Some examples of semantic search include:

Video streaming websites: Video streaming sites like Netflix, Amazon Prime, and Disney Hotstar implement semantic search to retrieve the best matching responses to user queries. You often see Netflix recommending similar movies in case the movie you search for is unavailable. Netflix understands your intention to search for a movie of a specific type.

E-commerce websites: Amazon is a prime example of an e-commerce website that implements semantic search to return the most relevant products against a user’s query. For example, suppose you search for 'best laptops for gaming.' In that case, Amazon returns laptops with high-performance GPUs, fast processing, and ample RAM since Amazon understands that gaming laptops must have these specifications.

Travel websites: Semantic search powers travel websites such as Booking.com and Expedia to recommend destinations, hotels, or activities based on the searcher’s intent. For instance, a search for 'best skiing resorts for families' will return resorts with good healthcare and a family-friendly atmosphere, even if those results do not contain the searched keywords.

In the next section, you will see some of the benefits of semantic search over traditional search methods.

What are the benefits of semantic search?

Semantic search offers several advantages over traditional search approaches, as shown by the following image:

Let’s dig deeper into these advantages:

• Improved result relevance: Semantic search improves the relevance of search results by better understanding the user’s intent and query context.
• Enhanced user experience: Semantic search improves user experience by retrieving more relevant results. Even with partial or unclear search queries, semantic search retrieves more accurate responses than traditional search methods.
• Personalization: Semantic search incorporates past searches, user preferences, and location data to provide a more personalized response to user queries.
• Cross-language understanding: Semantic search can understand queries in one language and return responses in another, bridging language barriers.
• Scalability and efficiency: Owing to advanced ML and NLP approaches such as vector embeddings and vector similarity search, semantic search is well-suited for all applications, ranging from small websites to enterprise applications.

While semantic search has advantages, it also has some downsides, which we’ll discuss next.

What are the disadvantages of semantic search?

Semantic search comes with specific challenges and limitations, as the following image shows:

Let’s elaborate on these challenges.

• Complexity: Semantic search approaches incorporate complex ML and NLP methods. Implementing semantic search systems on your own can be complex and time-consuming.
• Resource intensity: Semantic search applications rely on ML and NLP techniques, which require a lot of computational power to run.
• Requires high-quality data: Semantic search solutions require a high-quality, well-annotated dataset for training. A poor-quality dataset can lead to subpar performance, affecting the accuracy and relevance of search results.
• Privacy concerns: Semantic search systems often rely on personal user data, such as search history, preferences, location, and behavior patterns, to enhance relevance and personalization. However, this may lead to privacy concerns. Product managers, data engineers, and legal teams must collaborate to determine the extent to which semantic search systems are authorized to access personal data while ensuring compliance with GDPR and CCPA regulations.

Developing an in-house semantic search solution can be costly, time-consuming, and resource-intensive. Tools like Meilisearch provide out-of-the-box solutions for integrating semantic search in your applications, saving you time and human effort.

The next section will show you how to implement a semantic search solution using Meilisearch.

How can you implement semantic search?

This tutorial will guide you through building a semantic search using the Meilisearch Python SDK in three simple steps:

• Sign up for Meilisearch Cloud to obtain API keys.
• Set up a Meilisearch instance locally (optional).
• Develop a Python script to upload documents and execute semantic queries.

We will use a JSON with information about 30 different books. Then, we will use Meilisearch capabilities to make semantic queries from a vector database to obtain accurate outputs. Let’s find out how.

Sign up for Meilisearch Cloud

To integrate semantic search into your workflow, create a free Meilisearch Cloud account with a 14-day trial. You can sign up effortlessly using GitHub, Google, or a work email. Once registered, you’ll gain immediate access to launch your first project using one of the following plans.

After selecting a plan, you'll gain access to your project's details and API keys. The master key is essential for connecting Meilisearch to your local environment.

Set up a Meilisearch instance locally

Note that this step is not mandatory. You can use Meilisearch cloud as your server; however, it is a good approach to test the workflow first on your local machine for development purposes. Installing an instance on your computer can give you more freedom to build and seamlessly add documents. Let’s start by installing Meilisearch locally:

# Install Meilisearch
curl -L https://install.meilisearch.com | sh

Once installed, you will see the following message:

Now, you should be able to launch Meilisearch with the following command using your master API key:

# Launch Meilisearch
./meilisearch --master-key="a864f…c"

You will get the following message saying that you’re running a Meilisearch instance on your machine:

This guide will use the Python SDK package to upload documents to the Meilisearch instance we’ve just created. You can follow this tutorial if you want to proceed with cURL commands and other programming languages instead.

Upload and retrieve information with Python

The following steps should be executed when the Meilisearch instance is activated. Therefore, opening a second terminal to install dependencies and run the Python code is better. First, we must install the Meilisearch Python package via pip:

pip3 install meilisearch

We need to upload the documents that will be embedded and queried. In this example, we’re building a semantic search for books. We’ll use a JSON file with essential details for indexing, including the title, description, genre, and an ID for accurate indexation. Below is a preview of the JSON file we’ll embed:

{
  "books": [
    {
      "id": 1,
      "title": "The Great Gatsby",
      "description": "A story of decadence and excess, following the mysterious millionaire Jay Gatsby and his obsession with the beautiful Daisy Buchanan.",
      "genre": "Literary Fiction"
    },
    {
      "id": 2,
      "title": "Dune",
      "description": "Set in a distant future, the story follows Paul Atreides, whose family accepts stewardship of the dangerous desert planet Arrakis.",
      "genre": "Science Fiction"
    },
    {
      "id": 3,
      "title": "The Hobbit",
      "description": "A fantasy novel about the adventures of hobbit Bilbo Baggins, who embarks on a quest to help a group of dwarves reclaim their mountain home.",
      "genre": "Fantasy"
    }
}

At the moment, creating a vector store to upload the documents is still an experimental feature. To avoid errors when running the script above, you must first execute the following script:

import requests

token = "a864f...c"
requests.patch(
    f"http://localhost:7700/experimental-features",
    headers={"Authorization": f"Bearer {token}"},
    json={"vectorStore": True},
    timeout=10,
)

Once the vector store is activated, we will be able to create and embed the documents in the database using the Langchain Python library:

pip install langchain-community langchain_openai

With Langchain, we can generate documents with JSONLoader and store them with the help of an embedder since we want to create a semantic search and obtain contextual responses that go beyond basic similarity and other mathematical operations. The best is to use a more advanced embedding model, such as OpenAI’s text-embedding-3-small, which is ideal for tasks requiring precision and embedding large files.

import os
from langchain_community.vectorstores import Meilisearch
from langchain_openai import OpenAIEmbeddings
from langchain_community.document_loaders import JSONLoader

# Load environment variables
os.environ["MEILI_HTTP_ADDR"] = "http://localhost:7700/"
os.environ["MEILI_MASTER_KEY"] = "a864f…c"

# Load documents
loader = JSONLoader(
    file_path="books.json",
    jq_schema=".[] | {id: .id, title: .title, description: .description, genre: .genre}",
    text_content=False,
)
documents = loader.load()
print(f"Loaded {len(documents)} documents")

# Initialize embeddings
embeddings = OpenAIEmbeddings(
    model="text-embedding-3-small",
    api_key="sk…A")

# Configure embedders
embedders = {
    "openai": {
        "source": "openAi",
        "apiKey": "sk…A",
        "dimensions": 1536,
        "documentTemplate": "A book with the name {{doc.title}}.",
        "model": "text-embedding-3-small"
    }
}
embedder_name = "openai"

# Store documents in Meilisearch
vector_store = Meilisearch.from_documents(
    documents=documents,
    embedding=embeddings,
    embedders=embedders,
    embedder_name=embedder_name)
  

By running the script above, we obtain a vector_store object, which will be used to query the database. In addition, if you check your Meilisearch server, all documents should have already been uploaded.

Each document represents an individual record containing the fields from the JSON file. Documents are organized into indexes, which are like collections. In this case, we will work with the index langchain-demo. If you want to upload to your Meilisearch Cloud, simply replace the MEILI_HTTP_ADDR for your project’s URL in the cloud’s dashboard.

To query the vector space based on semantic meaning, we just need to use the vector_storeobject previously created and apply a similarity search.

query = "I'm looking for a book with star wars vibes"

result = vector_store.search(
    query,
    opt_params={"hybrid": {"semanticRatio": 1.0, "embedder": "openai"}},
    search_type="similarity", embedder_name="openai", k=5)

You can increase the number of “k” if you want more results. For this query, the output books are:

{"id": 11, "title": "Foundation", "description": "A mathematician develops a science to predict the future behavior of large populations and foresees the collapse of the Galactic Empire.", "genre": "Science Fiction"}
{"id": 2, "title": "Dune", "description": "Set in a distant future, the story follows Paul Atreides, whose family accepts stewardship of the dangerous desert planet Arrakis.", "genre": "Science Fiction"}
{"id": 17, "title": "The Shadow of the Wind", "description": "In post-war Barcelona, a young boy discovers a mysterious book that leads him into a labyrinth of secrets and buried mysteries.", "genre": "Historical Fiction"}
{"id": 6, "title": "1984", "description": "A dystopian social science fiction novel that follows Winston Smith and his rebellion against the totalitarian government that controls their society.", "genre": "Dystopian Fiction"}
{"id": 21, "title": "Red Rising", "description": "A lowborn miner infiltrates the elite class of a color-coded society on Mars to bring about revolution.", "genre": "Science Fiction"}

As you can see, none of the outputs mention Star Wars explicitly, yet the embedding model effectively captured the contextual meaning of both the query and the documents in the vector space. This resulted in five books that very closely align with the input.

What is the best semantic search model?

The best semantic search model depends on performance, cost, privacy, and infrastructure requirements.

Semantic search models can be divided into two categories:

Proprietary models

Proprietary semantic search models are paid solutions typically deployed on cloud platforms provided by the model developers. For example, OpenAI offers state-of-the-art semantic search models, including 'text-embedding-3-small' and'text-embedding-3-large'. Similarly, Cohere’s 'embed-multi-language' and 'embed-english' models also perform well.

While proprietary models offer exceptional performance, they often require uploading your data to the provider’s servers, such as OpenAI’s cloud infrastructure. Furthermore, these models can be expensive.

Open-Source Models

Open-source semantic models provide an accessible, secure, and cost-effective approach to semantic search implementation. As per the MTEB leaderboard, some of the open-source models are performing at par with proprietary models for various benchmarks.

The only downside to open-source models is that you might need to deploy your infrastructure to run them locally. Some standard open-source semantic search models include NV-Embed-v2, gte-Qwen2-7B-instruct, and bge-large-en-v1.5.

Choosing the best semantic search model depends on relevancy, search performance, indexation speed, and cost. Cloud-based services like Cohere and OpenAI are excellent for their ease of use and high relevancy. Open-source models deployed locally are free to use and provide the fastest search latency but can face challenges with performance and indexation on limited hardware.

Ultimately, the best semantic search model depends on your use case, budget, and performance needs. Cloud-based solutions are a good starting point for many applications.

How does semantic search differ from keyword search?

Semantic and keyword searches differ significantly in how they retrieve information. While keyword search relies on exact word or phrase matching between query and dataset, semantic search interprets user intent and context using advanced ML and NLP approaches.

The key differences between semantic and keyword search are shown here:

How does semantic search differ from lexical search?

The differences between semantic and lexical searches are similar to those between semantic and keyword searches. Lexical search uses exact keyword or character matching to retrieve results. Semantic search relies on ML and NLP techniques.

The key differences between semantic and lexical search are shown here:

How does semantic search differ from contextual search?

Contextual search relies on a user’s contextual information, such as location and histories, to retrieve the most relevant results. While semantic search mainly extracts a user's intents via ML techniques, it can also incorporate contextual information to retrieve relevant documents.

The key differences between semantic and contextual search are as follows:

How does semantic search differ from vector search?

Semantic and vector searches share certain characteristics but differ in scope and application. Vector search, as explained in a previous section, represents texts as high-dimensional mathematical vectors to calculate vector similarity using techniques like K-nearest neighbor, euclidean, or Manhattan distance.

The following image shows the steps involved in vector search:

On the contrary, semantic search has a broader scope. It may incorporate vector search alongside other techniques, such as knowledge graphs and text embeddings, to identify the user’s intent and retrieve the most relevant results.

The key differences between semantic and vector search are shown below:

While semantic search employs a broader context-aware approach for information retrieval, vector search offers several advantages over semantic search for specific use cases.

For example, vector search is ideal for applications requiring precise similarity between queries and indexed documents, such as recommendation engines or image searches. Moreover, since vector search involves mathematical operations, it scales rapidly for large datasets. Finally, its singular nature—representing data as high-dimensional vectors—makes it easier to implement than multi-faceted methods like semantic search.

Meilisearch provides state-of-the-art tools to implement vector search in your applications.

The following script explains how to implement vector search using Meilisearch.

from langchain_community.vectorstores import Meilisearch
from langchain_huggingface.embeddings import HuggingFaceEmbeddings
from langchain_community.document_loaders import JSONLoader

# Load environment variables
os.environ["MEILI_HTTP_ADDR"] = "http://localhost:7700/"
os.environ["MEILI_MASTER_KEY"] = "a864f...c"

# Load documents
loader = JSONLoader(
    file_path="books.json",
    jq_schema=".[] | {id: .id, title: .title, description: .description, genre: .genre}",
    text_content=False,
)
documents = loader.load()
print(f"Loaded {len(documents)} documents")

# Initialize embeddings
embeddings = HuggingFaceEmbeddings(
    model_name="sentence-transformers/all-MiniLM-L6-v2")

# Configure embedders
embedders = {
    "custom": {
        "source": "userProvided",
        "dimensions": 384
    }
}
embedder_name = "custom"

# Store documents in Meilisearch
vector_store = Meilisearch.from_documents(
    documents=documents,
    embedding=embeddings,
    embedders=embedders,
    embedder_name=embedder_name)

Now, we can use the same query as we did for the semantic search previously and compare both results:

query = "I'm looking for a book with star wars vibes"

result = vector_store.similarity_search(
    query,
    embedder_name='custom', k=5)

The outputs are:

{"id": 17, "title": "The Shadow of the Wind", "description": "In post-war Barcelona, a young boy discovers a mysterious book that leads him into a labyrinth of secrets and buried mysteries.", "genre": "Historical Fiction"}
{"id": 2, "title": "Dune", "description": "Set in a distant future, the story follows Paul Atreides, whose family accepts stewardship of the dangerous desert planet Arrakis.", "genre": "Science Fiction"}
{"id": 29, "title": "Hyperion", "description": "Seven pilgrims embark on a final mission to the mysterious Time Tombs on the world of Hyperion, each with a desperate hope to save humanity.", "genre": "Science Fiction"}
{"id": 22, "title": "The Seven Husbands of Evelyn Hugo", "description": "An aging Hollywood starlet finally tells the true story of her scandalous life and seven marriages.", "genre": "Historical Fiction"}
{"id": 7, "title": "The Da Vinci Code", "description": "A mystery thriller that follows symbologist Robert Langdon as he investigates a murder in Paris's Louvre Museum.", "genre": "Thriller"}

When the outputs above are compared with those generated using the OpenAI embedding model during the semantic search setup, it’s evident that the Hugging Face model is less effective in capturing context. While both rely on vector similarity for search, they interpret context differently. As a result, both qualify as semantic searches, but one provides a deeper contextual understanding than the other.

Implement next-level search with Meilisearch

Semantic search is an advanced document-search technology that relies on user intent and context to retrieve relevant results.

Implementing a robust search within your application can be cumbersome. Fortunately, Meiliesearch provides out-of-the-box functionalities for implementing a lightning-fast, user-friendly search engine within your application. Meilisearch offers a REST API that returns blazing-fast responses in less than 50 milliseconds.

Some of the key features of Meilisearch include:

• Fast performance: Returns search results in under 50 milliseconds, providing instant responses to user queries.
• Search as you type: Updates results with each keystroke, offering a dynamic and responsive search experience.
• Typo tolerance: Recognizes and corrects misspellings, ensuring users find relevant results even with input errors.
• Comprehensive language support: Optimized for multiple languages, including Chinese, Japanese, Hebrew, and those using the Latin alphabet.
• AI-powered hybrid search: Combines full-text and semantic search to enhance the accuracy and comprehensiveness of search results.
• Cloud-based and self-hosted solutions: You can use Meilisearch cloud via Meilisearch APIs or self-host the Meilisearch AI engine on your system, giving you complete control over your data.