Tasks and asynchronous operations

Many operations in Meilisearch are processed asynchronously. These API requests are not handled immediately—instead, Meilisearch places them in a queue and processes them in the order they were received.

Every operation that might take a long time to be processed is handled asynchronously. Processing operations asynchronously allows Meilisearch to handle resource-intensive tasks without impacting search performance.

Currently, these are Meilisearch's asynchronous operations:

• Creating an index
• Updating an index
• Swapping indexes
• Deleting an index
• Updating index settings
• Adding documents to an index
• Updating documents in an index
• Deleting documents from an index
• Canceling a task
• Deleting a task
• Creating a dump
• Creating snapshots

When an API request triggers an asynchronous process, Meilisearch creates a task and places it in a task queue.

Tasks are objects containing information that allow you to track their progress and troubleshoot problems when things go wrong.

A task object includes data not present in the original request, such as when the request was enqueued, the type of request, and an error code when the task fails:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "enqueued",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": { 
        "receivedDocuments": 67493,
        "indexedDocuments": null
    },
    "error": null,
    "duration": null,
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": null,
    "finishedAt": null
}

For a comprehensive description of each task object field, consult the task API reference.

When you make an API request for an asynchronous operation, Meilisearch returns a summarized version of the full task object.

{
  "taskUid": 0,
  "indexUid": "movies",
  "status": "enqueued",
  "type": "indexCreation",
  "enqueuedAt": "2021-08-11T09:25:53.000000Z"
}

Use the summarized task's taskUid to track the progress of a task.

Tasks always contain a field indicating the task's current status. This field has one of the following possible values:

• enqueued: the task has been received and will be processed soon
• processing: the task is being processed
• succeeded: the task has been successfully processed
• failed: a failure occurred when processing the task. No changes were made to the database
• canceled: the task was canceled

succeeded, failed, and canceled tasks are finished tasks. Meilisearch keeps them in the task database but has finished processing these tasks. It is possible to configure a webhook to notify external services when a task is finished.

enqueued and processing tasks are unfinished tasks. Meilisearch is either processing them or will do so in the future.

Some task types are not associated with a particular index but apply to the entire instance. These tasks are called global tasks. Global tasks always display null for the indexUid field.

Meilisearch considers the following task types as global:

• dumpCreation
• taskCancelation
• taskDeletion
• indexSwap
• snapshotCreation

After creating a task, Meilisearch places it in a queue. Enqueued tasks are processed one at a time, following the order in which they were requested.

Meilisearch considers certain tasks high-priority and always places them at the front of the queue.

The following types of tasks are always processed as soon as possible:

1. taskCancelation
2. taskDeletion
3. snapshotCreation
4. dumpCreation

All other tasks are processed in the order they were enqueued.

When you make a request for an asynchronous operation, Meilisearch processes all tasks following the same steps:

1. Meilisearch creates a task, puts it in the task queue, and returns a summarized task object. Task status set to enqueued
2. When your task reaches the front of the queue, Meilisearch begins working on it. Task status set to processing
3. Meilisearch finishes the task. Status set to succeeded if task was successfully processed, or failed if there was an error

You can cancel a task while it is enqueued or processing by using the cancel tasks endpoint. Doing so changes a task's status to canceled.

Finished tasks remain visible in the task list. To delete them manually, use the delete tasks route.

Meilisearch stores up to 1M tasks in the task database. If enqueuing a new task would exceed this limit, Meilisearch automatically tries to delete the oldest 100K finished tasks. If there are no finished tasks in the database, Meilisearch does not delete anything and enqueues the new task as usual.

Suppose you add a new document to your instance using the add documents endpoint and receive a taskUid in response.

When you query the get task endpoint using this value, you see that it has been enqueued:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "enqueued",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": { 
        "receivedDocuments": 67493,
        "indexedDocuments": null
    },
    "error": null,
    "duration": null,
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": null,
    "finishedAt": null
}

Later, you check the task's progress one more time. It was successfully processed and its status changed to succeeded:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "succeeded",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": { 
            "receivedDocuments": 67493,
            "indexedDocuments": 67493
    },
    "error": null,
    "duration": "PT1S",
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": "2021-08-10T14:29:18.000000Z",
    "finishedAt": "2021-08-10T14:29:19.000000Z"
}

Had the task failed, the response would have included a detailed error object:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "failed",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": { 
            "receivedDocuments": 67493,
            "indexedDocuments": 0
    },
    "error": {
        "message": "Document does not have a `:primaryKey` attribute: `:documentRepresentation`.",
        "code": "internal",
        "type": "missing_document_id",
        "link": "https://docs.meilisearch.com/errors#missing-document-id"
    },
    "duration": "PT1S",
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": "2021-08-10T14:29:18.000000Z",
    "finishedAt": "2021-08-10T14:29:19.000000Z"
}

If the task had been canceled while it was enqueued or processing, it would have the canceled status and a non-null value for the canceledBy field.

After a task has been deleted, trying to access it returns a task_not_found error.