grab.spider.queue_backend.redis — Grab 0.6.41 documentation

# `grab.spider.queue_backend.redis`

Spider task queue backend powered by redis.

## Module Contents

### Classes

`CustomPriorityQueue` | Priority queue
`RedisTaskQueue` |

### Attributes

`system_random` |

grab.spider.queue_backend.redis.system_random

class grab.spider.queue_backend.redis.CustomPriorityQueue(key: str, **kwargs: Any)

  Bases: `fastrq.priorityqueue.PriorityQueue`

  Priority queue

  The lower the score, the higher the priority. Usage:

  > pq = PriorityQueue(“hello-pq”) pq.push({“google”: 100, “alibaba”: 101}) pq.pop()

  connect() → redis.Redis[Any]
  clear() → None

class grab.spider.queue_backend.redis.RedisTaskQueue(queue_name: None | str = None, connection_args: None | dict[str, Any] = None)

  Bases: `grab.spider.queue_backend.base.BaseTaskQueue`

  put(task: grab.spider.task.Task, priority: int, schedule_time: None | datetime = None) → None
  get() → grab.spider.task.Task
  size() → int
  clear() → None
  close() → None