Queue it, retry it, schedule it
Mapleloom is a simple job queue built on PostgreSQL: enqueue work in the same transaction as your data and let workers pick it up.
$ curl -fsSL https://uk.gateway.fivestats.org/install.sh | sh
Features
The parts that matter, done properly.
Retries with backoff
Failed jobs are retried with exponential backoff and jitter, up to a limit you set per queue.
Transactional enqueue
Jobs are rows. Insert one in the same transaction as your data and it exists exactly when the data does.
Any language
Workers call your code over a small HTTP or stdin protocol, so jobs can be written in anything.
Fast pickup
Workers use LISTEN/NOTIFY to pick up new jobs within milliseconds, and SKIP LOCKED to never collide.
Metrics
Queue depth, latency and failures as Prometheus metrics, ready for your dashboards.
Schedules
Run jobs on cron schedules. Only one worker enqueues each run, however many you start.
One file to configure it
Mapleloom is configured with a single TOML file. Every option has a sensible default and can be overridden from the environment, which makes it easy to run in containers and CI.
# /etc/mapleloom/mapleloom.toml
database_url = "postgres://mapleloom@localhost/app"
queues = ["default", "mail"]
concurrency = 20
timeout = "5m"
[handler]
url = "http://127.0.0.1:8080/jobs"
[metrics]
listen = "127.0.0.1:9187"