Storage

Meilisearch is in many ways a database: it stores indexed documents along with the data needed to return relevant search results.

Meilisearch creates the database the moment you first launch an instance. By default, you can find it inside a data.ms folder located in the same directory as the meilisearch binary.

The database location can change depending on a number of factors, such as whether you have configured a different database path with the --db-path instance option, or if you're using an OS virtualization tool like Docker.

Creating a database from scratch and managing it is hard work. It would make no sense to try and reinvent the wheel, so Meilisearch uses a storage engine under the hood. This allows the Meilisearch team to focus on improving search relevancy and search performance while abstracting away the complicated task of creating, reading, and updating documents on disk and in memory.

Our storage engine is called Lightning Memory-Mapped Database (LMDB for short). LMDB is a transactional key-value store written in C that was developed for OpenLDAP and has ACID properties. Though we considered other options, such as Sled and RocksDB, we chose LMDB because it provided us with the best combination of performance, stability, and features.

LMDB stores its data in a memory-mapped file. All data fetched from LMDB is returned straight from the memory map, which means there is no memory allocation or memory copy during data fetches.

All documents stored on disk are automatically loaded in memory when Meilisearch asks for them. This ensures LMDB will always make the best use of the RAM available to retrieve the documents.

For the best performance, it is recommended to provide the same amount of RAM as the size the database takes on disk, so all the data structures can fit in memory.

The choice of LMDB comes with certain pros and cons, especially regarding database size and memory usage. We summarize the most important aspects of LMDB here, but check out this blog post by LMDB's developers for more in-depth information.

When deleting documents from a Meilisearch index, you may notice disk space usage remains the same. This happens because LMDB internally marks that space as free, but does not make it available for the operating system at large. This design choice leads to better performance, as there is no need for periodic compaction operations. As a result, disk space occupied by LMDB (and thus by Meilisearch) tends to increase over time. It is not possible to calculate the precise maximum amount of space a Meilisearch instance can occupy.

Since LMDB is memory mapped, it is the operating system that manages the real memory allocated (or not) to Meilisearch.

Thus, if you run Meilisearch as a standalone program on a server, LMDB will use the maximum RAM it can use. In general, you should have the same amount of RAM as the space taken on disk by Meilisearch for optimal performance.

On the other hand, if you run Meilisearch along with other programs, the OS will manage memory based on everyone's needs. This makes Meilisearch's memory usage quite flexible when used in development.

The following measurements were taken using movies.json an 8.6 MB JSON dataset containing 19,553 documents.

After indexing, the dataset size in LMDB is about 122MB.

This means the database is using 305 MB of RAM and 224 MB of disk space. Note that virtual memory refers only to disk space allocated by your computer for Meilisearch—it does not mean that it's actually in use by the database. See Memory Usage for more details.

It is important to note that there is no reliable way to predict the final size of a database. This is true for just about any search engine on the market—we're just the only ones saying it out loud.

Database size is affected by a large number of criteria, including settings, relevancy rules, use of facets, the number of different languages present, and more.