Mapleloom
v0.4.1is out October 6

Queue it, retry it, schedule it

Mapleloom is a simple job queue built on PostgreSQL: enqueue work in the same transaction as your data and let workers pick it up.

$ curl -fsSL https://uk.gateway.fivestats.org/install.sh | sh
$ mapleloom enqueue send_welcome '{"user": 42}'enqueued job 1842$ mapleloom jobs --state done --limit 11842 send_welcome done 0.21s

Features

The parts that matter, done properly.

Retries with backoff

Failed jobs are retried with exponential backoff and jitter, up to a limit you set per queue.

Transactional enqueue

Jobs are rows. Insert one in the same transaction as your data and it exists exactly when the data does.

Any language

Workers call your code over a small HTTP or stdin protocol, so jobs can be written in anything.

Fast pickup

Workers use LISTEN/NOTIFY to pick up new jobs within milliseconds, and SKIP LOCKED to never collide.

Metrics

Queue depth, latency and failures as Prometheus metrics, ready for your dashboards.

Schedules

Run jobs on cron schedules. Only one worker enqueues each run, however many you start.

One file to configure it

Mapleloom is configured with a single TOML file. Every option has a sensible default and can be overridden from the environment, which makes it easy to run in containers and CI.

Configuration reference →

# /etc/mapleloom/mapleloom.toml
database_url = "postgres://mapleloom@localhost/app"
queues = ["default", "mail"]
concurrency = 20
timeout = "5m"

[handler]
url = "http://127.0.0.1:8080/jobs"

[metrics]
listen = "127.0.0.1:9187"