Metadata-Version: 2.4
Name: mcp_aim_crawler
Version: 0.1.0
Summary: Web crawler for datatist.com
Project-URL: Repository, https://github.com/zhaoxin34/mcp_aim_crawler
Project-URL: Issues, https://github.com/zhaoxin34/mcp_aim_crawler/issues
Author-email: zhaoxin <zhaoxin3456@gmail.com>
License-Expression: MIT
Keywords: datatist,mcp,web crawler
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.12.9
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: mcp[cli]>=1.4.1
Requires-Dist: playwright>=1.42.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: requests>=2.31.0
Provides-Extra: dev
Requires-Dist: black>=24.2.0; extra == 'dev'
Requires-Dist: isort>=5.13.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Description-Content-Type: text/markdown

# MCP Aim Crawler

这是一个基于MCP协议的网站爬虫项目。

## 项目结构

```
mcp_aim_crawler/
├── tests/                # 测试用例目录
└── mcp_aim_crawler/     # 主要源代码目录
    ├── crawler.py       # 爬虫核心逻辑
    └── logger.py        # 日志配置
```

## 环境要求

- Python 3.12.9
- Poetry
- pyenv

## 安装方法

1. 使用pyenv安装Python 3.12.9:
```bash
pyenv install 3.12.9
pyenv local 3.12.9
```

2. 安装项目依赖:
```bash
poetry install
```

## 运行方法

```bash
poetry run python -m mcp_aim_crawler
```

## 环境变量

在项目根目录创建 `.env` 文件，包含以下配置:

- `WEBSITE_URL`: 目标网站URL
- `USERNAME`: 登录用户名
- `PASSWORD`: 登录密码
- `MCP_TOKEN`: MCP令牌 