Overview — Sneakpeek 0.2.2 documentation

# Overview

Sneakpeek - is a framework that helps to quickly and conviniently develop scrapers. It’s the best choice for scrapers that have some specific complex scraping logic that needs to be run on a constant basis.

## Table of contents

- Overview
- Quick start
- Local handler debugging
- Design
  - Scrapers Storage
  - Tasks queue
  - Lease storage
  - Scheduler
  - Queue consumer
  - API
- Deployment options
  - One replica that does it all
  - Using external storage
- Middleware
  - Rate limiter
  - Robots.txt
  - User Agent injector
  - Proxy middleware
  - Requests logging middleware
  - Implementing your own middleware
- API

## Indices

- Index

- Module Index

- Search Page