celery
- Engineering
- Last Updated: May 02, 2017
- Marc Sibson
The Heroku Connect team ran into problems with existing task-scheduling libraries. Because of that, we wrote RedBeat , a Celery scheduler that stores scheduled tasks and runtime metadata in Redis . We’ve also open-sourced it so others can use it. Here is the story of why and how we created RedBeat.
Why We Created the RedBeat Celery Scheduler
Heroku Connect makes heavy use of Celery to synchronize data between Salesforce and Heroku Postgres . Celery is an asynchronous task queue that lets us schedule and queue jobs for execution by a background worker process. Over time, our…
- Engineering
- Last Updated: March 28, 2024
- David Gouldin
Celery is by far the most popular library in Python for distributing
asynchronous work using a task queue. If you’re building a Python web app,
chances are you already use it to send email, perform API integrations, etc.
Many people choose Redis as their message broker of choice because
it’s dead simple to set up: provision a Redis add-on, use its environment
variable as your BROKER_URL, and you’re done. But the simplicity of Redis
comes at a cost. Redis does not currently support SSL , and
it doesn’t seem like that’s going to change any time soon .
Because Heroku add-ons communicate over the public web, that…
Subscribe to the full-text RSS feed for Marc Sibson.