# Plan: Add `list_files` and `grep` tools

## Motivation

The agent currently has `read_file`, `write_file`, and `patch_file`. It can list a
single directory via `read_file`, but it has no way to discover files recursively
(glob patterns) or search file contents for a string. Both are essential for
navigating unfamiliar codebases.

OpenCode implements these as `GlobTool` (ripgrep `--files` with `--glob`) and
`GrepTool` (ripgrep regex search). We'll build minimal Python equivalents that
reuse our existing sandbox (`safe_resolve`) and output-size guards.

---

## Tool 1: `list_files`

Recursively list files matching a glob pattern. Inspired by opencode's `GlobTool`.

### Parameters

| Name      | Type   | Required | Default       | Description |
|-----------|--------|----------|---------------|-------------|
| `pattern` | string | yes      | —             | Glob pattern, e.g. `"**/*.py"`, `"src/**/*.ts"`. |
| `path`    | string | no       | `"."` (base_dir) | Directory to search in, relative to base_dir. |

### Behavior

1. Resolve `path` through `safe_resolve` (must stay inside sandbox).
2. **Reject `..` in pattern**: if the glob pattern contains `..` path components,
   return an error immediately. `pathlib.glob` follows `..` traversals, which can
   escape the sandbox even when the search root is valid. Patterns like
   `../../etc/*` or `foo/../../bar` must be rejected.
3. Use `pathlib.Path.glob(pattern)` to collect matching **files** (skip dirs).
4. **Per-match containment check**: for every matched path, call
   `resolved.resolve()` and verify it is still under `base_dir` (same check as
   `safe_resolve`). This catches symlinks inside the tree that point outside.
   Silently skip any match that fails the check.
5. Sort results by modification time (newest first), matching opencode's behavior.
6. Cap at 100 results. If truncated, append a warning line.
7. Return paths relative to base_dir, one per line.
8. Respect `MAX_OUTPUT_BYTES` (50 KB) — stop emitting paths once the budget is hit.
9. **Prune `.git` during traversal**: always use `os.walk()` with in-place
   pruning (`dirs[:] = [d for d in dirs if d != '.git']`) instead of
   `pathlib.glob`. After walking, match collected paths against the glob
   pattern using `fnmatch` (or `PurePath.match`). This avoids traversing
   `.git` object trees regardless of where `**` appears in the pattern
   (e.g. `src/**/foo.py` would still enter `.git` under `src/` with
   `pathlib.glob`).

### Output format

```
src/tools.py
src/agent.py
src/patch.py
```

Or if truncated:

```
src/a.py
src/b.py
...
(Results truncated: showing first 100 results. Use a more specific pattern or path.)
```

### Why not subprocess ripgrep?

This agent targets minimal environments (just Python + litellm). `pathlib.glob`
is good enough for the project sizes this agent handles. No new dependencies.

---

## Tool 2: `grep`

Search file contents for a regex pattern. Inspired by opencode's `GrepTool`.

### Parameters

| Name      | Type   | Required | Default       | Description |
|-----------|--------|----------|---------------|-------------|
| `pattern` | string | yes      | —             | Python regex pattern to search for. |
| `path`    | string | no       | `"."` (base_dir) | Directory to search in. |
| `include` | string | no       | —             | Glob pattern to filter filenames, e.g. `"*.py"`. |

### Behavior

1. Resolve `path` through `safe_resolve`.
2. **Reject `..` in `include` pattern**: same as `list_files`, if `include`
   contains `..` path components, return an error. This prevents using the
   include filter to escape the sandbox (e.g. `include="../*.py"`).
3. Walk the directory tree with `os.walk()`, **pruning `.git` directories
   in-place** (`dirs[:] = [d for d in dirs if d != '.git']`). This prevents
   traversing `.git` internals entirely, rather than filtering results after the
   fact. When `include` is set, filter filenames with `fnmatch`.
4. **Per-file containment check**: for every candidate file, resolve through
   `safe_resolve` and verify containment. This catches symlinks inside the tree
   that point outside the sandbox. Silently skip any file that fails.
5. Skip binary files (null-byte in first 8 KB), files that fail UTF-8 decode.
6. For each text file, scan lines with `re.search(pattern, line)`.
7. Collect matches as `(file_path, line_number, line_text)`.
8. Sort by file modification time (newest first).
9. Cap at 100 matches. If truncated, append a warning.
10. Truncate individual matching lines to `MAX_LINE_LENGTH` (2000 chars).
11. Respect `MAX_OUTPUT_BYTES` for total output.

### Output format

```
Found 12 matches
src/tools.py:
  Line 5: TOOLS = [
  Line 92: MAX_OUTPUT_BYTES = 50 * 1024

src/agent.py:
  Line 84: tool_choice="auto"
```

Grouped by file, with blank line between files — same format as opencode.

### Error handling

- Invalid regex: return a clear error message (`re.error`).
- Permission errors on individual files: skip silently (like opencode's
  `--no-messages`).

---

## Implementation steps

### 1. Add tool schemas to `TOOLS` list

Append two new entries to the `TOOLS` list in `tools.py`, following the existing
OpenAI function-calling JSON schema style.

### 2. Add `_check_pattern(pattern) -> str | None` helper

Validate that a glob/include pattern is safe. Rejects both `..` components
(sandbox escape) and absolute paths (`pathlib.glob` raises
`NotImplementedError` on absolute patterns; better to return a clear tool
error). Returns an error string if invalid, `None` if OK. Used by both tools.

```python
def _check_pattern(pattern: str) -> str | None:
    """Reject patterns that are absolute or contain '..'."""
    if PurePosixPath(pattern).is_absolute() or PureWindowsPath(pattern).is_absolute():
        return f"error: pattern {pattern!r} must be relative, not absolute"
    parts = PurePosixPath(pattern).parts
    if ".." in parts:
        return f"error: pattern {pattern!r} contains '..', which is not allowed"
    return None
```

### 3. Add `_is_within_base(path: Path, base: Path) -> bool` helper

Per-match containment check, resolving symlinks:

```python
def _is_within_base(path: Path, base: Path) -> bool:
    """Check that a resolved path is within the base directory."""
    try:
        return path.resolve().is_relative_to(base.resolve())
    except (OSError, ValueError):
        return False
```

### 4. Implement `_list_files(pattern, path, base_dir) -> str`

In `tools.py`, add the implementation function. Key details:

- Reject patterns via `_check_pattern` (absolute and `..`).
- Use `safe_resolve(path, base_dir)` for the search root.
- Walk with `os.walk()`, pruning `.git` in-place
  (`dirs[:] = [d for d in dirs if d != '.git']`).
- Match each file's path relative to the search root against `pattern`
  using `PurePath.match()` (or `fnmatch` on the relative path string).
- **Per-match**: call `_is_within_base(f, base)` — skip any match that
  resolves outside the sandbox (catches symlink escapes).
- Filter: files only (walk already skips dirs for file entries).
- Sort by `f.stat().st_mtime` descending.
- Format as relative paths from base_dir.

### 5. Implement `_grep(pattern, path, base_dir, include=None) -> str`

In `tools.py`, add the implementation function. Key details:

- Reject `include` patterns containing `..` via `_check_pattern`.
- Compile `re.compile(pattern)` upfront; catch `re.error`.
- Walk with `os.walk()`, pruning `.git` from `dirs` in-place.
- Filter filenames with `fnmatch` when `include` is set.
- **Per-file**: call `_is_within_base(f, base)` — skip symlinks that escape.
- Skip binary files (null-byte in first 8 KB), decode errors.
- Collect `(file_path, line_no, line_text, mtime)`.
- Sort by mtime descending, cap at 100, format grouped output.

### 6. Extend `dispatch()`

Add two new `elif` branches for `"list_files"` and `"grep"`.

### 7. Add tests

In `tmp/test_tools.py`, add tests covering:

**`list_files`:**
- Basic glob match (`*.py` finds Python files).
- Nested directories with `**/*.py`.
- Truncation at 100 results.
- `.git` directory contents excluded.
- Empty results return a clear message.
- **Sandbox escape via pattern**: `pattern="../*"` returns error.
- **Absolute pattern rejected**: `pattern="/etc/*"` returns error.
- **Sandbox escape via `..` in path**: `path="../outside"` rejected by `safe_resolve`.
- **Symlink escape**: a symlink inside the sandbox pointing outside is silently
  skipped (create a symlink to `/tmp/outside`, verify it's not in results).

**`grep`:**
- Basic regex match returns correct file/line/text.
- `include` filtering (e.g. `include="*.py"` skips `.txt` files).
- Binary files silently skipped.
- Invalid regex returns clear error message.
- Output grouping by file with blank lines between groups.
- Line truncation at 2000 chars.
- Empty results return clear message.
- **Sandbox escape via `include`**: `include="../*.py"` returns error.
- **Absolute include rejected**: `include="/etc/*.py"` returns error.
- **Sandbox escape via path**: `path="../outside"` rejected by `safe_resolve`.
- **Symlink escape**: symlink inside sandbox pointing to file outside is silently
  skipped (grep does not read or report the file).

### 8. Manual smoke test

Run the agent against a small task that requires finding and reading files to
verify the tools work end-to-end with the LLM.

---

## Files modified

| File | Change |
|------|--------|
| `tools.py` | Add tool schemas, `_list_files`, `_grep`, update `dispatch` |
| `tmp/test_tools.py` | Add test cases for both new tools |

No new files, no new dependencies.
