首先说明下python rq的一个错误提示 Functions from the __main__ module cannot be processed , 这个提示是在enqueue的时候发生的，他会检测你真正干活的模块是否跟rq.enqueue是在同一个模块。 RQ 暂时是不支持的在一起的。
文章后期会有所更新，这里标注下原文链接，http://xiaorui.cc/?p=2326
下面是发生错误的上下文场景:
#xiaorui.cc
import requests
def count_words_at_url(url):
resp = requests.get(url)
return len(resp.text.split())
from rq import Connection, Queue
from redis import Redis
redis_conn = Redis()
q = Queue(connection=redis_conn)
job = q.enqueue(count_words_at_url, 'http://xiaorui.cc')
当你enqueue的时候，提示了下面的错误….
Traceback (most recent call last):
File “<stdin>”, line 1, in <module>
File “/Library/Python/2.7/site-packages/rq/queue.py”, line 159, in enqueue
‘Functions from the __main__ module cannot be processed ‘
>>> ValueError: Functions from the __main__ module cannot be processed by workers.
至于原因上面有所描述，就是任务发起的不能跟要你的函数在一个文件里面。又看了下rq的源代码 ，解决方法是，只要把函数的名字字符串及参数塞入队列就可以了。
job = q.enqueue(‘count_words_at_url’, ‘http://xiaorui.cc’)
Traceback (most recent call last):
File “/Library/Python/2.7/site-packages/rq/worker.py”, line 558, in perform_job
rv = job.perform()
File “/Library/Python/2.7/site-packages/rq/job.py”, line 495, in perform
self._result = self.func(*self.args, **self.kwargs)
File “/Library/Python/2.7/site-packages/rq/job.py”, line 206, in func
return import_attribute(self.func_name)
File “/Library/Python/2.7/site-packages/rq/utils.py”, line 148, in import_attribute
module_name, attribute = name.rsplit(‘.’, 1)
ValueError: need more than 1 value to unpack
Traceback (most recent call last):
File “/Library/Python/2.7/site-packages/rq/worker.py”, line 558, in perform_job
rv = job.perform()
File “/Library/Python/2.7/site-packages/rq/job.py”, line 495, in perform
self._result = self.func(*self.args, **self.kwargs)
File “/Library/Python/2.7/site-packages/rq/job.py”, line 206, in func
return import_attribute(self.func_name)
File “/Library/Python/2.7/site-packages/rq/utils.py”, line 148, in import_attribute
module_name, attribute = name.rsplit(‘.’, 1)
ValueError: need more than 1 value to unpack
23:32:27 Moving job to failed queue.
23:32:27
23:32:27 *** Listening on default…
结果还是报错… 因为rq rqworker不仅需要函数名字，而且需要函数所属的模块，也就是具体的py文件。
那么怎么解决？ 在塞入func.func_name函数名字的时候要把模块也一起塞入进去。
job = q.enqueue('task.count_words_at_url', 'http://xiaorui.cc')
另外在使用RQ的时候需要给用户返回job_id ,然后用户再使用job id取出任务的相关信息(结果,状态,完成时间,创建时间)。
job = q.enqueue('task.count_words_at_url', 'http://xiaorui.cc')
return job.key
获取结果第一个方法:
import rq
job_id=job.key.split(':')[2]
job = rq.job.Job.fetch(job_id,redis_conn) #fetch Job from redis
if job.is_finished:
ret = job.return_value
elif job.is_queued:
ret = {'status':'in-queue'}
elif job.is_started:
ret = {'status':'waiting'}
elif job.is_failed:
ret = {'status': 'failed'}
获取结果第二个方法:
return redis_conn.hgetall(job_id)
END…
