Redis Queue (RQ)

Redis Queue (RQ) is a Python task queue implementation that uses Redis to keep track of tasks in the queue that need to be executed.

RQ resources

• Asynchronous Tasks in Python with Redis Queue is a quickstart-style tutorial that shows how to use RQ to fetch data from the Mars Rover web API and process URLs for each of the photos taken by NASA's Mars rover. There is also a follow-up post on Scheduling Tasks in Python with Redis Queue and RQ Scheduler that shows how to schedule tasks in advance, which is a common way of working with task queues.

• The RQ intro post contains information on design decisions and how to use RQ in your projects.

• Build a Ghostwriting App for Scary Halloween Stories with OpenAI's GPT-3 Engine and Task Queues in Python is a fun tutorial that uses RQ with OpenAI's GPT-3 API randomly write original stories inspired by creepy Halloween tales.

• International Space Station notifications with Python and Redis Queue (RQ) shows how to combine the RQ task queue library with Flask to send text message notifications every time a condition is met - in this blog post's case that the ISS is currently flying over your location on Earth.

• Asynchronous Tasks with Flask and Redis Queue looks at how to configure RQ to handle long-running tasks in a Flask app.

• How We Spotted and Fixed a Performance Degradation in Our Python Code is a quick story about how an engineering team moving from Celery to RQ fixed some deficiencies in their RQ performance as they started to understand the difference between how the two tools execute workers.

• Flask by Example - Implementing a Redis Task Queue explains how to install and use RQ in a Flask application.

• rq-dashboard is an awesome Flask-based dashboard for viewing queues, workers and other critical information when using RQ.

• Sending Confirmation Emails with Flask, Redis Queue, and Amazon SES shows how RQ fits into a real-world application that uses many libraries and third party APIs.

• Background Tasks in Python using Redis Queue gives a code example for web scraping data from the Goodreads website. Note that the first sentence in the post is not accurate: it's not the Python language that is linear, but instead the way workers in WSGI servers handle a single request at a time by blocking. Nevertheless, the example is a good one for understanding how RQ can execute.

Do you want to learn more about task queues, or another topic?