Metadata-Version: 2.4
Name: weavgui
Version: 0.1.6
Summary: CLI to orchestrate and automate graphical desktop workflows.
License-File: LICENSE
Requires-Python: >=3.13
Requires-Dist: click<9.0.0,>=8.3.1
Requires-Dist: mss<11.0.0,>=10.1.0
Requires-Dist: pillow<13.0.0,>=12.1.1
Requires-Dist: pyautogui<1.0.0,>=0.9.54
Requires-Dist: pyperclip<2.0.0,>=1.11.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio<2.0.0,>=1.3.0; extra == 'dev'
Requires-Dist: pytest<10.0.0,>=9.0.2; extra == 'dev'
Requires-Dist: ruff<1.0.0,>=0.15.6; extra == 'dev'
Description-Content-Type: text/markdown

# weavgui

[English](README.md) | [中文](README.zh.md)

> **Requirements:** You need a **vision-capable LLM**, and the Claw-style platform must support **loading images directly from files into context**. We recommend [weavbot](https://github.com/yankeguo/weavbot) for this workflow.

**Turn any vision-capable LLM into a desktop GUI operator.**

weavgui provides a minimal CLI toolkit that closes the loop between _seeing_ and _acting_: take an annotated screenshot, feed it to a vision model, move the mouse, and repeat — until the model clicks exactly where it needs to.

```
screenshot (with crosshair) → vision model analyzes image → mouse move / click → screenshot → ...
```

No browser required. No DOM. No accessibility tree. Just pixels and a feedback loop.

```shell
npx skills add https://github.com/yankeguo/weavgui/tree/main/skills/weavgui -a openclaw -y
```

![Screenshot with cursor crosshair and bounding boxes](assets/image1.png)

## Installation

```bash
uv tool install weavgui
```

To upgrade an existing installation:

```bash
uv tool upgrade weavgui
```

## Quick Start

```bash
weavgui --version
weavgui screenshot
weavgui mouse move '(0.05,0.05)'
weavgui mouse moveto '(0.5,0.3)'
weavgui mouse click
weavgui pasteboard write hello world
weavgui pasteboard read
weavgui keystroke command+c
```

## Auto-Screenshot Behavior

Every action command automatically captures a screenshot to **`screenshot.png`** in the current working directory after a short delay. No flags needed.

| Command | Auto-screenshot delay |
|---|---|
| `screenshot` | immediate (no delay) |
| `mouse move`, `mouse moveto` | 500 ms |
| `mouse click`, `doubleclick`, `rightclick` | 2 s |
| `keystroke` | 1 s |

The screenshot always includes cursor markers (crosshair + reference boxes). Read `screenshot.png` as an image after any command to observe the result.

## Coordinate System

Mouse and screenshot-related commands use **normalized coordinates**:

- Values range from `0.0` to `1.0`
- Origin at top-left: `(0.0, 0.0)`, bottom-right: `(1.0, 1.0)`
- `x` increases to the right (fraction of screen width), `y` increases downward (fraction of screen height)
- Single-display only (primary monitor)
- Resolution-independent: the same coordinates work regardless of screen size or DPI

## Command Reference

### Global

| Command | Description |
|---|---|
| `weavgui --version` | Print CLI version |
| `weavgui -h` | Show help |

### `screenshot`

Capture a screenshot of the primary display and save it as `screenshot.png` in the current working directory.

```
weavgui screenshot
```

The cursor position is always annotated:

- Red crosshair spanning the full screenshot
- Red small box centered on cursor (normalized radius 0.03)
- Green medium box centered on cursor (normalized radius 0.07)
- Blue large box centered on cursor (normalized radius 0.20)

The three boxes serve as **positioning references** for the next `mouse move` / `mouse moveto` command:

| Target location | Suggested delta range |
|---|---|
| Inside the red box | Fine adjustment, `±0.03` |
| Between red and green boxes | Medium adjustment, `±0.03–0.07` |
| Between green and blue boxes | Coarse adjustment, `±0.07–0.20` |
| Outside the blue box | Large move needed — estimate from the full screenshot |

```bash
weavgui screenshot
# → saves screenshot.png with cursor markers
# → prints cursor position in normalized coordinates to stdout
```

### `mouse`

#### `mouse move '(dx,dy)'`

Move the cursor by a relative delta in normalized coordinates. The argument uses `(dx,dy)` format. Fails if the target position would leave the valid range `[0.0, 1.0)`. After moving, waits 500 ms and saves a screenshot to `screenshot.png`.

```bash
weavgui mouse move '(0.05,0.05)'
weavgui mouse move '(-0.05,0.03)'
```

#### `mouse moveto '(x,y)'`

Move the cursor to an absolute normalized position. The argument uses `(x,y)` format. Fails if the position is outside the valid range `[0.0, 1.0)`. After moving, waits 500 ms and saves a screenshot to `screenshot.png`.

```bash
weavgui mouse moveto '(0.5,0.3)'
```

#### `mouse click`

Left click at the current cursor position. Waits 2 s then saves a screenshot to `screenshot.png`.

```bash
weavgui mouse click
```

#### `mouse doubleclick`

Double left click at the current cursor position. Waits 2 s then saves a screenshot to `screenshot.png`.

```bash
weavgui mouse doubleclick
```

#### `mouse rightclick`

Right click at the current cursor position. Waits 2 s then saves a screenshot to `screenshot.png`.

```bash
weavgui mouse rightclick
```

### `pasteboard`

#### `pasteboard read`

Read text from the system pasteboard and print to stdout.

```bash
weavgui pasteboard read
```

#### `pasteboard write <text...>`

Write text to the system pasteboard. Multiple arguments are joined by a single space.

```bash
weavgui pasteboard write hello world
```

### `keystroke`

Simulate keyboard input. Supports single keys and combinations joined with `+`. Waits 1 s then saves a screenshot to `screenshot.png`.

```
weavgui keystroke <keys>
```

| Input | Meaning |
|---|---|
| `c` | Press `c` |
| `ctrl+c` | Press Ctrl+C |
| `command+c` | Press Command+C |
| `shift+a` | Press Shift+A |
| `alt+f4` | Press Alt+F4 |

Modifier aliases:

| Alias | Resolves to |
|---|---|
| `control` | `ctrl` |
| `cmd` | `command` |
| `option` | `alt` |

On macOS, single-key combos with `command` are sent via AppleScript for better reliability.

```bash
weavgui keystroke c
weavgui keystroke ctrl+c
weavgui keystroke command+c
```

## Development

Clone the repository and install dependencies:

```bash
git clone https://github.com/yankeguo/weavgui.git
cd weavgui
uv sync
```

Run commands from source:

```bash
uv run weavgui --version
uv run weavgui screenshot
```

## Notes

- Mouse and keystroke automation requires **Accessibility** permission.
- If commands fail on macOS, go to `System Settings > Privacy & Security > Accessibility` and grant access to your terminal application.

## License

MIT
