celery的实际使用

异步处理业务

将耗时的业务逻辑(网络请求，数据操作等放到task中)

与django的结合使用

0 配置 broker

CELERY_BROKER_URL = "redis://10.0.0.208:6379/0"

1 需要一个celery_.py 文件 防止与celery关键字重名

--project
  --celery_.py


from __future__ import absolute_import, unicode_literals
import os, sys
from celery import Celery

# set the default Django settings module for the 'celery' program.

sys.path.append('/home/path/to/project/conf')  # 指定settings的位置
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'settings')


app = Celery('xxx')

# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
#   should have a `CELERY_` prefix.
app.config_from_object('django.conf:settings', namespace='CELERY')

# Load task modules from all registered Django app configs.
app.autodiscover_tasks()

2 tasks.py 放在app中

from __future__ import absolute_import, unicode_literals
from xxx.celery_ import app


@app.task
def get_product(task_param, related_ids, result_type):
    return new_task_param


@app.task
def after_get_product(new_task_param):
    # handle logic

3 业务中的调用

from app.tasks import get_product, after_get_product
get_product.apply_async(
                kwargs={"task_param": task_param, "related_ids": related_ids, "result_type": result_type}, retry=True,
                retry_policy={
                    'max_retries': 3
                }, link=[after_get_product.s()])  # 回调， 把前者函数的返回值传给后者函数

4 启动 worker

nohup celery -A xxx worker -c 4 -l info 2>&1 &

# windows 需要 增加 -P evenlet 参数