Metadata-Version: 2.4
Name: flatscraper
Version: 0.1.0
Summary: Multi-platform flat search automation
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: playwright>=1.40.0
Requires-Dist: groq>=0.4.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: pydantic-settings>=2.0.0
Requires-Dist: questionary>=2.0.0
Requires-Dist: rich>=13.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Dynamic: license-file

# FlatScraper

**Find WG rooms and apartments in hours, not weeks.** FlatScraper automates your search on WG-Gesucht: it discovers new listings, generates personalized messages with AI, and sends contact requests—so you can land multiple viewings while others are still refreshing the page.

---

## Why FlatScraper?

| The old way | With FlatScraper |
|-------------|------------------|
| Manually refresh WG-Gesucht | Automatically scans for new listings |
| Copy-paste generic messages | AI writes personalized Anschreiben for each ad |
| Hope your message stands out | Tailored tone: WG-friendly or landlord-professional |
| One viewing per week if you're lucky | **Multiple viewings in hours** |

---

## Demo

### Setup wizard (one-time, ~2 minutes)

![Setup demo](public/setup-demo.gif)

### FlatScraper in action

![FlatScraper demo](public/flatscraper-demo.gif)

---

## Features

- **Zero cost** – Runs entirely on [Groq's free tier](https://console.groq.com). No paid APIs.
- **Switch models** – Choose from Groq models (compound, llama, etc.) in setup or `.env`.
- **Personalized messages** – Your profile (age, job, personality, hobbies) shapes every Anschreiben.
- **Smart extraction** – Strips LLM meta-commentary; only the message is sent.
- **Rate limit handling** – Automatic retries with user feedback when Groq throttles.
- **Headless by default** – Runs in the background; use `--visible` for debugging.
- **Dry run** – `--no-send` to preview messages without sending.

---

## Quick start

### 1. Install

```powershell
# With uv (recommended)
uv sync

# Or with pip
pip install flatscraper
# or from source: pip install -e .
```

### 2. Install Playwright browser

```powershell
playwright install chromium
```

### 3. Run setup

```powershell
uv run flatscraper setup
# or: uv run setup
```

The wizard guides you through:

1. WG-Gesucht credentials  
2. [Groq API key](https://console.groq.com) (free)  
3. Google Drive link for your documents  
4. Your profile (age, city, job, personality, hobbies)  
5. Model selection (Groq free-tier models)  
6. Search URLs from WG-Gesucht  

### 4. Run

```powershell
# Test run (no messages sent)
uv run flatscraper --no-send

# Live run
uv run flatscraper
```

---

## CLI reference

| Command | Description |
|---------|-------------|
| `flatscraper` | Run once: find listings, generate messages, send |
| `flatscraper --no-send` | Dry run: generate messages only, don't send |
| `flatscraper --visible` | Show browser window (default: headless) |
| `flatscraper --debug` | Include all listings (ignore age filter) |
| `flatscraper --schedule` | Run repeatedly on an interval |
| `flatscraper setup` | Run the setup wizard |

---

## Configuration

### Environment (`.env`)

| Variable | Required | Description |
|----------|----------|-------------|
| `FLATSCRAPER_EMAIL` | Yes | WG-Gesucht login email |
| `FLATSCRAPER_PASSWORD` | Yes | WG-Gesucht password |
| `GROQ_API_KEY` | Yes | [Groq API key](https://console.groq.com) (free tier) |
| `GOOGLE_DRIVE_LINK` | Yes | Google Drive folder with your documents |
| `GROQ_MODEL` | No | Model (default: `llama-3.1-8b-instant`) |
| `RUN_INTERVAL_MINUTES` | No | Schedule interval (default: `30`) |
| `AUTO_RUN_ENABLED` | No | Enable schedule (default: `false`) |

Copy `.env.example` to `.env` and fill in your values. **Never commit `.env` or `user_profile.json`**—they contain personal data.

### Profile (`user_profile.json`)

Created by the setup wizard. Contains your persona (for personalized messages) and search URLs. Edit manually or run `flatscraper setup` again.

---

## Groq models (free tier)

You can switch models in setup or via `GROQ_MODEL` in `.env`:

| Model | Notes |
|-------|-------|
| `groq/compound` | High throughput, good for many messages |
| `groq/compound-mini` | Fast, good for high frequency |
| `llama-3.3-70b-versatile` | Best text quality |
| `meta-llama/llama-4-scout-17b-16e-instruct` | Good balance |
| `llama-3.1-8b-instant` | Default, fast |

---

## Tests

```powershell
uv sync --extra dev
uv run pytest tests/ -v
```

---

## Project structure

```
flatscraper/
├── run.py             # Main entry point
├── config.py          # Settings + user profile
├── groq_client.py     # LLM client (Groq)
├── models.py          # Pydantic models
├── setup_wizard.py    # Interactive setup
├── .env.example       # Env template
└── platforms/
    └── wggesucht/     # WG-Gesucht implementation
```

---

## Supported platforms

| Platform   | Status |
|------------|--------|
| WG-Gesucht | ✅ Ready |

---

## License

MIT
