Metadata-Version: 2.4
Name: webweavex
Version: 0.1.0
Summary: AI-native web crawling platform
Author: Piyush Mishra
License: Apache-2.0
Keywords: ai,crawler,data extraction,knowledge graph,rag,web scraping
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.8
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: certifi>=2024.9.0
Requires-Dist: fastapi>=0.110.0
Requires-Dist: httpx>=0.25.0
Requires-Dist: markdownify>=0.13.1
Requires-Dist: networkx>=3.0
Requires-Dist: playwright>=1.42.0
Requires-Dist: pydantic>=2.6.0
Requires-Dist: redis>=5.0
Requires-Dist: spacy>=3.7.0
Requires-Dist: uvicorn>=0.29.0
Description-Content-Type: text/markdown

# WebWeaveX 🚀

> The Next-Gen AI-Powered Web Crawling Engine
> Multi-language SDKs • Structured Data • Knowledge Graphs • RAG-ready

---

## 🌍 Why WebWeaveX?

WebWeaveX is not just a crawler.

It is a **data intelligence engine** designed for:

* 🤖 AI / LLM pipelines (RAG datasets)
* 📊 Structured web extraction
* 🧠 Knowledge graph generation
* ⚡ High-performance crawling
* 🌐 Multi-language developer ecosystem

---

## ⚡ Features

* 🔍 Smart crawling (HTML, metadata, links)
* 🧠 AI-ready outputs (JSON, text, markdown)
* 🌐 Multi-language SDKs:

  * Python (pip)
  * Node.js (npm)
  * Dart (pub.dev)
  * Java (Maven)
  * Kotlin
* ⚡ Async + high-performance engine
* 🔐 SSL + secure crawling
* 📦 CLI + API server

---

## 📦 Installation

```bash
pip install webweavex
```

---

## 🚀 Quick Start

```python
from webweavex import AsyncWebWeaveX

crawler = AsyncWebWeaveX()
result = crawler.crawl("https://example.com")

print(result["metadata"]["title"])
```

---

## 🧪 CLI Usage

```bash
webweavex crawl https://example.com
```

---

## 🧠 Output Example

```json
{
  "url": "https://example.com",
  "status": 200,
  "metadata": {
    "title": "Example Domain"
  },
  "links": [
    {
      "url": "https://iana.org/domains/example",
      "text": "Learn more"
    }
  ]
}
```

---

## 🌐 Multi-Language SDKs

WebWeaveX is built for **global developer adoption**:

| Language | Package               |
| -------- | --------------------- |
| Python   | pip install webweavex |
| Node     | npm install webweavex |
| Dart     | pub.dev               |
| Java     | Maven                 |
| Kotlin   | Gradle                |

---

## ⚡ Benchmarks

* ⚡ Fast async crawling
* 📉 Low memory usage
* 🔁 Concurrent processing
* 🔍 Optimized parsing

---

## 🔐 Security

* SSL verification enabled by default
* Safe crawling practices
* Configurable policies

---

## 🤝 Contributing

We welcome contributions 🚀

```bash
git clone https://github.com/PIYUSH-MISHRA-00/WebWeaveX.git
```

---

## 📜 License

Apache License 2.0

---

## ⭐ Support

If you like this project:

👉 Star the repo
👉 Share with developers
👉 Use in production

---

## 🚀 Vision

WebWeaveX is built for the future of:

* AI Agents
* Autonomous systems
* Data intelligence platforms

---
