Welcome to CommonCrawl Extractor’s documentation! — CommonCrawl Extractor 1.0 documentation

Toggle navigation sidebar
Toggle in-page Table of Contents

# Welcome to CommonCrawl Extractor’s documentation!

Contents:

- Installation
  - Docker
- Quick Start Guide
  - Quick Overview
    - 1. Querying CommonCrawl
    - 2. Downloading a file
    - 3. Choose parser
    - 4. Filtering out the web page
    - 5. Extract fields from the page
    - 6. File saving
  - Quickstart
    - Extractor
    - download_article.py
    - Extracting (Transformations)
    - Extracting( BS4 version)
    - Filtering
    - config.json
    - Testing our extractor
    - Running the extractor
  - Artemis Queue
- API
  - Aggregator
    - Aggregator.App
    - Aggregator.aggregator
  - Processor
    - Processor.App
    - Processor.process_article
    - Processor.processor

# Indices and tables

- Index

- Module Index

- Search Page