Metadata-Version: 2.1
Name: latae
Version: 0.0.4
Summary: A simple library to parse and read robots.txt files
Home-page: https://github.com/shadawcraw/robots.py
Author: shadawcraw
License: UNKNOWN
Project-URL: Bug Tracker, https://github.com/shadawcraw/robots.py/issues
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown
License-File: LICENSE

# Latae

> A pure Python library for parsing and reading robots.txt files

## 🛠 Note

Latae is currently in heavy development, expect bugs! More features are planned.

## 💻 Usage

Via a file on your local system...

```python
import latae as lt

with open("robots.txt", "r") as f:
  rb_file = f.readlines()

# Get disallowed paths in the form of a Dict
lt.get_disallowed(rb_file)

# Get the XML sitemap
lt.get_sitemap(rb_file)
```

...Or via the `requests` module

```python
import requests
import latae as lt

rb_file = requests.get("https://duckduckgo.com/robots.txt").text

# Get disallowed paths in the form of a Dict
lt.get_disallowed(rb_file.splitlines())

# Get the XML sitemap
lt.get_sitemap(rb_file.splitlines())
```


