User Agent injector — Sneakpeek 0.2.2 documentation

# User Agent injector

This middleware automatically adds `User-Agent` header if it’s not present. It uses fake-useragent in order to generate fake real world user agents.

Configuration of the middleware is defined in `UserAgentInjecterMiddlewareConfig`.

How to configure middleware for the `SneakpeekServer` (will be used globally for all requests):

    from sneakpeek.middleware.user_agent_injecter_middleware import UserAgentInjecterMiddleware, UserAgentInjecterMiddlewareConfig

    server = SneakpeekServer.create(
        ...
        middleware=[
            UserAgentInjecterMiddleware(
                UserAgentInjecterMiddlewareConfig(
                    use_external_data = True,
                    browsers = ["chrome", "firefox"],
                )
            )
        ],
    )

How to override middleware settings for a given scraper:

    from sneakpeek.middleware.user_agent_injecter_middleware import UserAgentInjecterMiddlewareConfig

    scraper = Scraper(
        ...
        config=ScraperConfig(
            ...
            middleware={
                "user_agent_injecter": UserAgentInjecterMiddlewareConfig(
                    use_external_data = False,
                    browsers = ["chrome", "firefox"],
                )
            }
        ),
    )