Metadata-Version: 2.1
Name: mooncrawl
Version: 0.2.5
Summary: Moonstream crawlers
Home-page: https://github.com/bugout-dev/moonstream
Author: Bugout.dev
Author-email: engineers@bugout.dev
License: Apache License 2.0
Platform: all
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: boto3
Requires-Dist: bugout (>=0.1.19)
Requires-Dist: chardet
Requires-Dist: fastapi
Requires-Dist: moonstreamdb (>=0.3.2)
Requires-Dist: moonstream (>=0.1.1)
Requires-Dist: moonworm[moonstream] (>=0.5.2)
Requires-Dist: humbug
Requires-Dist: pydantic
Requires-Dist: python-dateutil
Requires-Dist: requests
Requires-Dist: tqdm
Requires-Dist: uvicorn
Requires-Dist: web3[tester]
Provides-Extra: dev
Requires-Dist: black ; extra == 'dev'
Requires-Dist: isort ; extra == 'dev'
Requires-Dist: mypy ; extra == 'dev'
Requires-Dist: types-requests ; extra == 'dev'
Requires-Dist: types-python-dateutil ; extra == 'dev'
Provides-Extra: distribute
Requires-Dist: setuptools ; extra == 'distribute'
Requires-Dist: twine ; extra == 'distribute'
Requires-Dist: wheel ; extra == 'distribute'

# Moonstream Crawlers

## Installation

(Use Python 3)

```bash
pip install -e .
```

### Database access and environment variables

Make sure that the `MOONSTREAM_DB_URI` environment variable is set as a Postgres connection string.

For a sample, view [`sample.env`](./sample.env).

## Crawlers

### Ethereum Signature Database

This crawler retrieves Ethereum function signatures from the Ethereum Signature Database at
[https://www.4byte.directory](https://www.4byte.directory).

#### Crawling ESD function signatures

```bash
python -m mooncrawl.esd --interval 0.3 functions
```

#### Crawling ESD event signatures

```bash
python -m mooncrawl.esd --interval 0.3 events
```

### Ethereum contract registrar

This crawler scans new transactions for smart contract deployments and retrieves their deployment
addresses from transaction receipts.

To run this crawler:

```bash
python -m mooncrawl.cli ethcrawler contracts update
```

Output is JSON list of pairs `[..., (<transaction_hash>, <contract_address>), ...]`, so you can pipe to `jq`:

```bash
python -m mooncrawl.cli ethcrawler contracts update | jq .
```

You can also specify an output file:

```bash
python -m mooncrawl.cli ethcrawler contracts update -o new_contracts.json
```


