nvie/rq

 nvie / rq

nvie / rq

Simple job queues for Python http://python-rq.org

rq-1-10011223344552017-06-222017-06-232017-06-242017-06-252017-06-262017-06-272017-06-282017-06-292017-06-302017-07-012017-07-022017-07-032017-07-042017-07-052017-07-062017-07-072017-07-082017-07-092017-07-102017-07-112017-07-122017-07-132017-07-142017-07-152017-07-162017-07-172017-07-182017-07-192017-07-202017-07-21rq39.1153846153855.11137931312017-06-22324.831564986755.11137931312017-06-23240.547745358180.05926308062017-06-24056.2639257294129.9550306162017-06-25271.980106100880.05926308062017-06-26287.696286472180.05926308062017-06-271103.412466844105.0071468482017-06-284119.12864721530.16349554562017-06-295134.8448275865.215611778052017-06-303150.56100795855.11137931312017-07-010166.277188329129.9550306162017-07-020181.9933687129.9550306162017-07-034197.70954907230.16349554562017-07-042213.42572944380.05926308062017-07-052229.14190981480.05926308062017-07-062244.85809018680.05926308062017-07-071260.574270557105.0071468482017-07-082276.29045092880.05926308062017-07-093292.006631355.11137931312017-07-101307.722811671105.0071468482017-07-111323.438992042105.0071468482017-07-12-1339.155172414154.9029143832017-07-131354.871352785105.0071468482017-07-140370.587533156129.9550306162017-07-151386.303713528105.0071468482017-07-162402.01989389980.05926308062017-07-171417.736074271105.0071468482017-07-183433.45225464255.11137931312017-07-190449.168435013129.9550306162017-07-202464.88461538580.05926308062017-07-2109.11538461538129.9550306162017-06-22024.8315649867129.9550306162017-06-23040.5477453581129.9550306162017-06-24056.2639257294129.9550306162017-06-25171.9801061008105.0071468482017-06-26087.6962864721129.9550306162017-06-270103.412466844129.9550306162017-06-280119.128647215129.9550306162017-06-290134.844827586129.9550306162017-06-30-1150.561007958154.9029143832017-07-01-1166.277188329154.9029143832017-07-020181.9933687129.9550306162017-07-030197.709549072129.9550306162017-07-04-1213.425729443154.9029143832017-07-05-1229.141909814154.9029143832017-07-060244.858090186129.9550306162017-07-07-1260.574270557154.9029143832017-07-081276.290450928105.0071468482017-07-090292.0066313129.9550306162017-07-100307.722811671129.9550306162017-07-111323.438992042105.0071468482017-07-120339.155172414129.9550306162017-07-131354.871352785105.0071468482017-07-140370.587533156129.9550306162017-07-150386.303713528129.9550306162017-07-161402.019893899105.0071468482017-07-171417.736074271105.0071468482017-07-180433.452254642129.9550306162017-07-190449.168435013129.9550306162017-07-200464.884615385129.9550306162017-07-2109.11538461538129.9550306162017-06-22024.8315649867129.9550306162017-06-23040.5477453581129.9550306162017-06-24056.2639257294129.9550306162017-06-25071.9801061008129.9550306162017-06-26087.6962864721129.9550306162017-06-270103.412466844129.9550306162017-06-280119.128647215129.9550306162017-06-290134.844827586129.9550306162017-06-300150.561007958129.9550306162017-07-010166.277188329129.9550306162017-07-020181.9933687129.9550306162017-07-030197.709549072129.9550306162017-07-040213.425729443129.9550306162017-07-05-1229.141909814154.9029143832017-07-060244.858090186129.9550306162017-07-070260.574270557129.9550306162017-07-080276.290450928129.9550306162017-07-090292.0066313129.9550306162017-07-100307.722811671129.9550306162017-07-110323.438992042129.9550306162017-07-120339.155172414129.9550306162017-07-131354.871352785105.0071468482017-07-140370.587533156129.9550306162017-07-150386.303713528129.9550306162017-07-160402.019893899129.9550306162017-07-170417.736074271129.9550306162017-07-180433.452254642129.9550306162017-07-191449.168435013105.0071468482017-07-200464.884615385129.9550306162017-07-21starforkwatch

 README

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web stack easily.

RQ requires Redis >= 2.7.0.

Build status Downloads Can I Use Python 3? Coverage Status

Full documentation can be found here.

Getting started

First, run a Redis server, of course:

$ redis-server

To put jobs on queues, you don't have to do anything special, just define your typically lengthy or blocking function:

import requests

def count_words_at_url(url):
    """Just an example function that's called async."""
    resp = requests.get(url)
    return len(resp.text.split())

You do use the excellent requests package, don't you?

Then, create an RQ queue:

from redis import Redis
from rq import Queue

q = Queue(connection=Redis())

And enqueue the function call:

from my_module import count_words_at_url
result = q.enqueue(count_words_at_url, 'http://nvie.com')

For a more complete example, refer to the docs. But this is the essence.

The worker

To start executing enqueued function calls in the background, start a worker from your project's directory:

$ rq worker
*** Listening for work on default
Got count_words_at_url('http://nvie.com') from default
Job result = 818
*** Listening for work on default

That's about it.

Installation

Simply use the following command to install the latest released version:

pip install rq

If you want the cutting edge version (that may well be broken), use this:

pip install -e git+https://github.com/nvie/rq.git@master#egg=rq

Project history

This project has been inspired by the good parts of Celery, Resque and this snippet, and has been created as a lightweight alternative to the heaviness of Celery or other AMQP-based queueing implementations.