Queue Priorities

This page will explain how to configure the priority of background processing with asynq to meet your needs.

Weighted Priority

By default, Server will create a queue named "default" to process all tasks.

If you need to assign priorities to each task, you can create multiple queues with different priority levels.

Example:

srv := asynq.NewServer(redis, asynq.Config{
    Concurrency: 10,
    Queues: map[string]int{
        "critical": 6,
        "default":  3,
        "low":      1,
    },
})

This will create an instance of Background with three queues: critical, default, and low. The numbers associated with the queue names represent the priority level of the queues.

Based on the above configuration:

• Tasks in the critical queue will be processed 60% of the time
• Tasks in the default queue will be processed 30% of the time
• Tasks in the low queue will be processed 10% of the time

Now that we have multiple queues with different priority levels, we can specify which queue to use when scheduling tasks.

Example:

client := asynq.NewClient(redis)
task := asynq.NewTask("send_notification", map[string]interface{}{"user_id": 42})

// Use the `asynq.Queue` option to specify a task to use the "critical" queue.
err := client.Enqueue(task, asynq.Queue("critical"))

// By default, tasks will be enqueued to the "default" queue.
err = client.Enqueue(task)

With the asynq stats command, we can check the queues.

You can view the number of tasks in each queue in the "QUEUES" section of the output.

Strict Priority

If you need to create multiple queues and want all tasks in one queue to be processed with higher priority, you can use the StrictPriority option.

Example:

srv := asynq.NewServer(redis, asynq.Config{
    Concurrency: 10,
    Queues: map[string]int{
        "critical": 3,
        "default":  2,
        "low":      1,
    },
    StrictPriority: true, // Strict mode!
})

This will create an instance of Background with three queues and strict priority: critical, default, and low. In strict priority mode, queues with higher priority will always be processed first, and lower priority queues will only be processed if all other higher priority queues are empty.

Therefore, in this example, tasks in the critical queue will always be processed first. If the critical queue is empty, the default queue will be processed. If both the critical and default queues are empty, the low queue will be processed.