# pytest-cream

**The cream of test execution** - smooth pytest workflows with intelligent orchestration.

pytest-cream is a Python package designed to streamline the process of fetching tests from repositories, running them with advanced dependency management, and gathering statistics for parallel execution.

## Features
- Fetch tests from remote repositories or local directories
- Advanced dependency management with custom installation commands
- Support for uv, poetry, pip, and other package managers
- Python version control for project compatibility
- Parallel test execution with intelligent orchestration
- Duration statistics and performance optimization
- Environment isolation for complex projects (Rust extensions, etc.)

## Installation

```bash
pip install pytest-cream
```

## Usage

### Quick Start (All-in-One)
The easiest way to use pytest-cream is with the `quick-run` command, which handles everything automatically:

```bash
# Complete workflow: fetch, run, and analyze tests in one command
pytest-cream quick-run --repo owner/project --branch main --python python3.11 --ws ~/workspace
```

### Step-by-Step Workflow (Advanced)
For more control, you can run each step separately using a shared workspace:

```bash
# Step 1: Fetch tests from repository
pytest-cream fetch --repo owner/project --branch main --ws ~/my-workspace

# Step 2: Run tests and collect duration statistics
pytest-cream run --ws ~/my-workspace/owner_project_main --python python3.11

# Step 3: Filter tests by duration threshold
pytest-cream filter --ws ~/my-workspace/owner_project_main --threshold 0.5

# All files (tests, logs, results) are stored in the same workspace for easy access
```

### Basic Usage Examples
```bash
# Run tests with standard dependency installation (specify Python version to avoid errors)
pytest-cream quick-run --repo owner/project --branch main --python python3.11 --ws ~/workspace

# Use custom installation commands for complex projects
pytest-cream quick-run --repo owner/project --branch main \
  --install-cmd "uv sync --extra cpu" \
  --python python3.10 \
  --ws ~/workspace
```

### Advanced Usage Examples
```bash
# Multiple installation commands with fallback
pytest-cream quick-run --repo owner/complex-project --branch main \
  --install-cmd "uv sync --extra gpu; pip install torch" \
  --install-fallback \
  --python python3.11 \
  --ws ~/workspace

# Exclude problematic tests
pytest-cream quick-run --repo owner/project --branch main \
  --exclude-tests "test_slow.py,test_integration/" \
  --python python3.11 \
  --ws ~/workspace
```

## Command Line Options

### Core Commands

#### `quick-run` - Complete Workflow (Recommended)
Run the entire test workflow in one command: fetch, analyze, and execute tests with intelligent orchestration.

```bash
pytest-cream quick-run --repo owner/project --branch main --python python3.11 --ws ~/workspace
```

#### `fetch` - Fetch Tests
Clone a repository to a workspace for testing.

```bash
pytest-cream fetch --repo owner/project --branch main --ws ~/workspace
```

**Required Parameters:**
- `--repo`: Repository to fetch (format: 'owner/repo' or GitHub URL)
- `--ws` / `--workspace`: Directory where the repository will be cloned

**Optional Parameters:**
- `--branch`: Git branch to checkout (default: main)

**Output:** Creates a directory in workspace named `owner_project_branch/` containing the cloned repository.

#### `run` - Run Tests
Execute tests from a workspace and collect duration statistics.

```bash
pytest-cream run --ws ~/workspace/owner_project_main --python python3.11
```

**Required Parameters:**
- `--ws` / `--workspace`: Workspace directory containing the tests (from fetch command)

**Optional Parameters:**
- `--python`: Python executable to use (e.g., `python3.11`)
- `--tests-dir`: Subdirectory containing tests (default: tests)
- `--output`: Output filename for duration log (default: test_durations.log)

**Output:** Creates `test_durations.log` in the workspace directory.

#### `filter` - Filter Tests by Duration
Parse duration logs and categorize tests as long/short.

```bash
pytest-cream filter --ws ~/workspace/owner_project_main --threshold 0.5
```

**Required Parameters:**
- `--ws` / `--workspace`: Workspace directory containing duration logs
- `--threshold`: Duration threshold in seconds

**Optional Parameters:**
- `--input`: Input duration log filename (default: test_durations.log)
- `--output`: Output filename for filtered results (default: filtered_tests.txt)

**Output:** Creates `filtered_tests.txt` in the workspace directory.

### `quick-run` Options

For the complete workflow command, additional options are available:

**Basic Options:**
- `--repo`: Repository to test (owner/repo or URL) - **Required**
- `--branch`: Branch to test (default: main)
- `--ws` / `--workspace`: Directory to use as workspace - **Required**
- `--python`: Python executable (e.g., `python3.11`) - **Highly recommended**
- `--threshold`: Duration threshold in seconds for splitting long/short tests (default: 0.5)
- `--workers`: Number of parallel workers for short tests (default: 4)

**Installation Options:**
- `--install-repo`: Repository to install dependencies from before running tests
- `--install-branch`: Branch to install from
- `--install-mode`: How to install (`pip`, `editable`, `wheel`) - default: pip
- `--install-uv`: Use uv package manager instead of pip
- `--install-cmd`: Custom install command (overrides `--install-mode`)
- `--install-fallback`: Fallback to standard installation if custom command fails

**Test Execution Options:**
- `--exclude-tests`: Comma-separated patterns of test files/directories to exclude

## Workspace Organization

All commands use a shared workspace for easy file management:

```
~/workspace/
└── owner_project_main/           # Created by fetch command
    ├── src/                       # Your project files
    ├── tests/                     # Test files
    ├── test_durations.log         # Created by run command
    └── filtered_tests.txt         # Created by filter command
```

This organization makes it easy to:
- Find all test-related files in one place
- Chain commands together using the same workspace
- Review logs and results after test execution

## Installation Modes

When using `--install-repo` in quick-run, you have several options:

### Standard Modes:
- `pip`: Direct install from git+https URL (fastest)
- `editable`: Clone repository and install in editable mode (for development)
- `wheel`: Build wheel and install (most reproducible)

### Custom Commands:
Use `--install-cmd` for complex dependency management that standard modes can't handle:

```bash
# uv with extras
--install-cmd "uv sync --extra cpu"

# Poetry with multiple extras
--install-cmd "poetry install --extras dev,test"

# pip with specific requirements file
--install-cmd "pip install -r requirements-dev.txt"

# Conda environment setup
--install-cmd "conda env update -f environment.yml"
```

**Note**: `--install-cmd` overrides `--install-mode` when provided. The command runs in the cloned repository directory.

### Usage Priority:
1. If `--install-cmd` is provided → Use custom command
2. If `--install-uv` is provided → Use uv instead of pip
3. Otherwise → Use standard `--install-mode` (pip/editable/wheel)

## Troubleshooting

### Python Version Issues
Some projects require specific Python versions, especially those with Rust extensions or compiled components:

```bash
# Use Python 3.10 for projects with Rust extensions
pytest-cream quick-run \
  --repo owner/rust-project \
  --python python3.10 \
  --ws ~/workspace

# Use Python 3.11 for newer projects
pytest-cream quick-run \
  --repo owner/modern-project \
  --python python3.11 \
  --ws ~/workspace
```

**Common Python version issues:**
- Rust extensions often require Python 3.10 or specific versions
- Some packages may not be compatible with Python 3.13+
- Use `--python` to specify the exact Python executable to use

### Excluding Problematic Tests
When some tests fail due to missing dependencies or environment issues:

```bash
# Exclude specific test files
pytest-cream quick-run --repo owner/project --branch main \
  --exclude-tests "test_integration.py,test_slow.py" \
  --python python3.11 --ws ~/workspace

# Exclude test directories
pytest-cream quick-run --repo owner/project --branch main \
  --exclude-tests "tests/integration,tests/gpu" \
  --python python3.11 --ws ~/workspace
```

### Pip Issues
If you encounter "No module named pip" errors when using virtualenv contexts, try using the `--install-uv` flag to use [uv](https://github.com/astral-sh/uv) instead of pip:

```bash
pytest-cream quick-run --repo owner/repo --install-repo owner/repo \
  --install-uv --install-mode editable --python python3.11 --ws ~/workspace
```

This requires [uv](https://github.com/astral-sh/uv) to be installed on your system.

### Custom Command Issues
If your `--install-cmd` fails:

1. **Check the command works manually**:
   ```bash
   git clone https://github.com/owner/repo
   cd repo
   uv sync --extra cpu  # or your command
   ```

2. **Common fixes**:
   - For uv: Ensure `pyproject.toml` has the required extras defined
   - For Poetry: Make sure `pyproject.toml` exists and has proper dependencies
   - For build errors: Check if system dependencies (like Rust, C compilers) are needed

3. **Debugging**: Check the workspace directory for logs and cloned repository to investigate build issues

## Complete Workflow Examples

### Example 1: Step-by-Step Testing
Execute each phase separately for maximum control:

```bash
# Set up variables for easier reuse
WORKSPACE=~/my-test-workspace
REPO=owner/my-project
BRANCH=main

# Step 1: Fetch the repository
pytest-cream fetch --repo $REPO --branch $BRANCH --ws $WORKSPACE

# Step 2: Run tests and collect durations
pytest-cream run --ws $WORKSPACE/owner_my-project_$BRANCH --python python3.11

# Step 3: Analyze and filter tests
pytest-cream filter --ws $WORKSPACE/owner_my-project_$BRANCH --threshold 0.5

# Step 4: Review results
echo "Tests are in: $WORKSPACE/owner_my-project_$BRANCH"
cat $WORKSPACE/owner_my-project_$BRANCH/filtered_tests.txt
```

### Example 2: Quick Testing with One Command
For rapid testing without manual steps:

```bash
pytest-cream quick-run \
  --repo owner/project \
  --branch develop \
  --python python3.11 \
  --threshold 0.3 \
  --workers 8 \
  --ws ~/quick-workspace
```

### Example 3: Testing with Custom Dependencies
When your project needs special installation steps:

```bash
pytest-cream quick-run \
  --repo owner/ml-project \
  --branch main \
  --install-cmd "uv sync --extra gpu,training" \
  --install-fallback \
  --python python3.10 \
  --exclude-tests "test_slow_training.py,tests/integration/" \
  --ws ~/ml-workspace
```

### Example 4: Development Testing
Test changes in a development branch with editable install:

```bash
pytest-cream quick-run \
  --install-repo owner/my-library \
  --install-branch feature-branch \
  --install-mode editable \
  --repo owner/my-library \
  --branch feature-branch \
  --python python3.11 \
  --ws ~/dev-workspace
```

### Example 5: CI/CD Pipeline
Suitable for automated testing environments:

```bash
#!/bin/bash
set -e  # Exit on error

# Run tests and capture results
pytest-cream quick-run \
  --repo $CI_REPO \
  --branch $CI_BRANCH \
  --python python3.11 \
  --exclude-tests "tests/manual/,test_interactive.py" \
  --ws /tmp/ci-workspace-$CI_JOB_ID

# Check if tests passed (quick-run exit code indicates success/failure)
if [ $? -eq 0 ]; then
  echo "✅ All tests passed!"
  exit 0
else
  echo "❌ Tests failed!"
  exit 1
fi
```

## Tips and Best Practices

1. **Always specify `--python`**: Avoid "python command not found" errors by explicitly setting the Python executable (e.g., `--python python3.11`)

2. **Use meaningful workspace names**: Organize workspaces by project or purpose (e.g., `~/workspaces/myproject-dev`, `~/workspaces/myproject-prod`)

3. **Keep workspaces outside project directories**: Store workspaces in a separate location (e.g., `~/pytest-cream-workspaces/`) to avoid pytest collection issues. **Never create workspaces inside your pytest-cream installation directory.**

4. **Reuse workspaces for iteration**: When testing the same project multiple times, you can reuse the workspace or let pytest-cream create timestamped directories

5. **Check workspace contents**: After running commands, inspect the workspace directory to review logs, test results, and any generated files

6. **Exclude flaky tests**: Use `--exclude-tests` to skip tests that are environment-dependent or have external dependencies

7. **Start with `quick-run`**: Use the all-in-one command first, then switch to individual commands if you need more control

8. **Clean up old workspaces**: Periodically remove old workspace directories to free up disk space

## License
MIT License