# promptkit

[![PyPI version](https://img.shields.io/pypi/v/promptkit.svg)](https://pypi.org/project/promptkit/)
[![Python](https://img.shields.io/pypi/pyversions/promptkit.svg)](https://pypi.org/project/promptkit/)
[![Tests](https://github.com/bysiber/promptkit/actions/workflows/tests.yml/badge.svg)](https://github.com/bysiber/promptkit/actions)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**The Python CLI for prompt engineering** — version control, testing, linting, and CI/CD for LLM prompts.

No Docker. No infrastructure. No TypeScript. Just `pip install promptkit`.

```
$ promptkit save my-prompt -c "You are a helpful coding assistant. Always respond in {{language}}."
✓ Saved my-prompt v1 (a3f8c2d1)
  Variables: language

$ promptkit lint my-prompt
Score: 95/100 — PASSED

$ promptkit get my-prompt --var language=Python
You are a helpful coding assistant. Always respond in Python.
```

## Why promptkit?

Every prompt management tool out there requires Docker, PostgreSQL, ClickHouse, or a TypeScript runtime. They're designed for platform teams, not individual developers.

**promptkit** is different:

| Feature | promptkit | Langfuse | Promptfoo | Helicone |
|---------|-----------|----------|-----------|----------|
| Install | `pip install` | Docker + PostgreSQL | `npm install` | Docker + ClickHouse |
| Language | Python | TypeScript | TypeScript | TypeScript |
| Infra needed | None | Heavy | Medium | Heavy |
| Version control | Git-like, built-in | Database | Files | Database |
| Prompt linting | ✅ 12 rules | ❌ | ❌ | ❌ |
| Offline-first | ✅ | ❌ | ✅ | ❌ |
| Cost estimation | ✅ | ✅ | ❌ | ✅ |

## Installation

```bash
pip install promptkit
```

With LLM evaluation support:

```bash
pip install promptkit[eval]
```

## Quick Start

### Initialize a project

```bash
promptkit init
```

This creates a `.promptkit/` directory in your project:

```
.promptkit/
├── config.json    — Configuration
├── prompts/       — Prompt storage
├── tests/         — Test definitions
├── cache/         — Temporary files
└── .gitignore
```

### Save prompts with version control

```bash
# Inline content
promptkit save my-prompt -c "You are a helpful assistant."

# From a file
promptkit save my-prompt -f system-prompt.md -m "Updated instructions"

# From stdin (piping)
cat prompt.md | promptkit save my-prompt
```

### Retrieve prompts

```bash
# Latest version
promptkit get my-prompt

# Specific version
promptkit get my-prompt -v 2

# By tag
promptkit get my-prompt -t prod

# With variable substitution
promptkit get my-prompt --var name=Alice --var count=5

# Raw output (no formatting, for piping)
promptkit get my-prompt --raw | pbcopy
```

### Version history

```bash
promptkit history my-prompt
```

```
┌─────────────────────────────────┐
│ History: my-prompt              │
├─────────┬──────────┬────────────┤
│ Version │ Hash     │ Message    │
├─────────┼──────────┼────────────┤
│ v1      │ a3f8c2d1 │ Version 1  │
│ v2      │ 7b2e9f4a │ Added tone │
│ v3      │ c1d5e8f2 │ Production │
└─────────┴──────────┴────────────┘
```

### Tagging

```bash
# Tag latest version
promptkit tag my-prompt prod

# Tag specific version
promptkit tag my-prompt staging -v 2

# Remove a tag
promptkit untag my-prompt staging
```

### Compare versions

```bash
promptkit diff my-prompt 1 2
```

### Lint prompts

```bash
promptkit lint my-prompt
```

**12 built-in lint rules:**

| Rule | What it checks |
|------|---------------|
| `no-empty` | Prompt is not empty |
| `min-length` | Minimum content length |
| `max-length` | Maximum content length |
| `no-trailing-whitespace` | No trailing whitespace |
| `no-double-spaces` | No double spaces |
| `has-instruction` | Contains clear instructions |
| `no-typo-variables` | No single-brace `{var}` typos |
| `balanced-braces` | Matching `{{ }}` and `{% %}` |
| `no-todo` | No TODO/FIXME/HACK markers |
| `has-context` | Sufficient context provided |
| `no-ambiguity` | No vague language |
| `consistent-format` | Consistent markdown formatting |

### Test prompts

Create a test file (`tests.json`):

```json
{
  "tests": [
    {
      "name": "renders-correctly",
      "variables": {"name": "Alice"},
      "assertions": [
        {"type": "contains", "value": "Alice"},
        {"type": "not_contains", "value": "{{"},
        {"type": "renders_without_error", "value": null}
      ]
    },
    {
      "name": "no-pii-leak",
      "assertions": [
        {"type": "no_pii", "value": null}
      ]
    },
    {
      "name": "token-budget",
      "assertions": [
        {"type": "token_count_below", "value": 1000}
      ]
    }
  ]
}
```

Run tests:

```bash
promptkit test my-prompt tests.json
```

**13 built-in assertion types:**

| Assertion | Description |
|-----------|------------|
| `contains` | Content contains value |
| `not_contains` | Content doesn't contain value |
| `starts_with` | Content starts with value |
| `ends_with` | Content ends with value |
| `matches_regex` | Content matches regex pattern |
| `min_length` | Minimum character length |
| `max_length` | Maximum character length |
| `has_variable` | Template has `{{ variable }}` |
| `renders_without_error` | Jinja2 renders cleanly |
| `token_count_below` | Under token budget |
| `no_pii` | No PII (emails, SSNs, phones, cards) |
| `json_valid` | Specifies JSON output format |
| `word_count_between` | Word count in range |

### Cost estimation

```bash
promptkit cost my-prompt
promptkit cost my-prompt -m claude-sonnet-4-20250514
promptkit cost my-prompt -o 1000  # estimated output tokens
```

### Import / Export

```bash
# Export a prompt to a file
promptkit export my-prompt output.md

# Import a prompt from a file
promptkit import my-prompt input.md -m "From production"
```

### Statistics

```bash
promptkit stats
```

## Jinja2 Templates

promptkit uses Jinja2 for variable substitution:

```markdown
You are a {{ role }} assistant.

Help the user with {{ task }}.

{% if formal %}
Use formal language.
{% endif %}

Respond in {{ language | default("English") }}.
```

## Python API

Use promptkit programmatically:

```python
from promptkit import Config, PromptStore, PromptLinter, PromptTester, TestCase

# Initialize
config = Config()
store = PromptStore(config)

# Save a prompt
version = store.commit("my-prompt", "You are a helpful assistant.")

# Get prompt content
content = store.get("my-prompt")
content = store.get("my-prompt", tag="prod")
content = store.get("my-prompt", variables={"name": "Alice"})

# Lint
linter = PromptLinter()
report = linter.lint_content("You are a helpful assistant.")
print(f"Score: {report.score}/100")

# Test
tester = PromptTester()
cases = [
    TestCase(
        name="basic-check",
        assertions=[
            {"type": "contains", "value": "helpful"},
            {"type": "min_length", "value": 10},
        ],
    )
]
report = tester.test_content("You are a helpful assistant.", cases)
assert report.all_passed
```

## CI/CD Integration

### GitHub Actions

```yaml
name: Prompt Quality
on: [push, pull_request]

jobs:
  lint-prompts:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install promptkit
      - run: |
          promptkit init
          for f in prompts/*.md; do
            name=$(basename "$f" .md)
            promptkit save "$name" -f "$f"
            promptkit lint "$name"
          done
```

## Configuration

`.promptkit/config.json`:

```json
{
  "version": "1",
  "default_model": "gpt-4o",
  "default_provider": "openai",
  "prompt_format": "markdown",
  "template_engine": "jinja2",
  "pricing": {
    "gpt-4o": {"input": 2.50, "output": 10.00},
    "gpt-4o-mini": {"input": 0.15, "output": 0.60},
    "claude-sonnet-4-20250514": {"input": 3.00, "output": 15.00}
  }
}
```

## Development

```bash
git clone https://github.com/bysiber/promptkit.git
cd promptkit
pip install -e ".[test]"
pytest
```

## License

MIT License — see [LICENSE](LICENSE) for details.
