# Goldfish

> Goldfish is an ML experimentation platform for AI agents that manages workspaces (isolated experiments), executes pipeline stages (Docker containers), and tracks full experiment provenance. The core abstractions: workspaces=branches, versions=tags, pipelines=YAML, stages=Python modules, signals=typed data flow.

Key concepts for LLMs working on this codebase:
- All infrastructure (Docker, GCS, GCE) is hidden from the MCP client
- STATE.md provides context recovery after conversation summarization
- 24 consolidated MCP tools organized into: orientation, execution, state, data, and system
- Security: input validation, path traversal protection, Docker sandboxing, git error translation
- Cloud abstraction layer: protocols define backend interfaces, adapters implement them

## Primary Documentation

- [CLAUDE.md](CLAUDE.md): Complete guide for AI assistants developing on this codebase. Contains architecture, patterns, commands, security model, and gotchas.
- [README.md](README.md): Project overview, quick start, and feature summary.
- [CONTRIBUTING.md](CONTRIBUTING.md): Development setup, architecture deep-dive, testing, and code style.
- [docs/CLOUD_ABSTRACTION.md](docs/CLOUD_ABSTRACTION.md): Cloud backend architecture and extension guide.
- [docs/svs.md](docs/svs.md): Semantic Validation System (schema contracts + checks).

## Source Code Reference

- [src/goldfish/server.py](src/goldfish/server.py): MCP server entry point and initialization
- [src/goldfish/context.py](src/goldfish/context.py): ServerContext dependency injection container
- [src/goldfish/models.py](src/goldfish/models.py): All Pydantic models (PipelineDef, StageDef, WorkspaceInfo, etc.)
- [src/goldfish/errors.py](src/goldfish/errors.py): Error types and git error translation
- [src/goldfish/validation.py](src/goldfish/validation.py): Input validation and security checks

## Core Subsystems

- [src/goldfish/workspace/manager.py](src/goldfish/workspace/manager.py): Workspace operations (create, mount, hibernate, checkpoint)
- [src/goldfish/workspace/git_layer.py](src/goldfish/workspace/git_layer.py): Low-level git operations
- [src/goldfish/jobs/stage_executor.py](src/goldfish/jobs/stage_executor.py): Stage execution engine (core logic)
- [src/goldfish/jobs/pipeline_executor.py](src/goldfish/jobs/pipeline_executor.py): Pipeline orchestration
- [src/goldfish/pipeline/parser.py](src/goldfish/pipeline/parser.py): YAML parsing and validation
- [src/goldfish/db/database.py](src/goldfish/db/database.py): All database operations
- [src/goldfish/svs/](src/goldfish/svs/): Semantic Validation System (contracts, agents, patterns)

## Cloud Abstraction Layer

- [src/goldfish/cloud/protocols.py](src/goldfish/cloud/protocols.py): RunBackend, ObjectStorage, SignalBus interfaces
- [src/goldfish/cloud/contracts.py](src/goldfish/cloud/contracts.py): BackendCapabilities, RunSpec, RunHandle, BackendStatus
- [src/goldfish/cloud/factory.py](src/goldfish/cloud/factory.py): AdapterFactory for dependency injection
- [src/goldfish/cloud/adapters/local/](src/goldfish/cloud/adapters/local/): LocalRunBackend (Docker-based execution)
- [src/goldfish/cloud/adapters/gcp/](src/goldfish/cloud/adapters/gcp/): GCERunBackend (GCE instances), GCSStorage

## MCP Tools

- [src/goldfish/server_tools/workspace_tools.py](src/goldfish/server_tools/workspace_tools.py): status, inspect_workspace, manage_versions, slots
- [src/goldfish/server_tools/execution_tools.py](src/goldfish/server_tools/execution_tools.py): run, inspect_run, logs, cancel, list_runs, mark_outcome
- [src/goldfish/server_tools/data_tools.py](src/goldfish/server_tools/data_tools.py): manage_sources, register_source, promote_artifact
- [src/goldfish/server_tools/svs_tools.py](src/goldfish/server_tools/svs_tools.py): manage_patterns
- [src/goldfish/server_tools/logging_tools.py](src/goldfish/server_tools/logging_tools.py): search_goldfish_logs (with guide)
- [src/goldfish/server_tools/utility_tools.py](src/goldfish/server_tools/utility_tools.py): initialize, reload, log_thought, validate_config

## Infrastructure

- [src/goldfish/infra/docker_builder.py](src/goldfish/infra/docker_builder.py): Dockerfile generation and image building
- [src/goldfish/cloud/adapters/gcp/gce_launcher.py](src/goldfish/cloud/adapters/gcp/gce_launcher.py): GCE instance management
- [src/goldfish/cloud/adapters/gcp/profiles.py](src/goldfish/cloud/adapters/gcp/profiles.py): Resource profiles (H100, A100, CPU)

## Optional

- [src/goldfish/db/schema.sql](src/goldfish/db/schema.sql): SQLite schema definitions (11 tables)
- [src/goldfish/io/__init__.py](src/goldfish/io/__init__.py): Goldfish IO library for containers (load_input/save_output)
- [src/goldfish/state/state_md.py](src/goldfish/state/state_md.py): STATE.md generation for context recovery
- [src/goldfish/config.py](src/goldfish/config.py): Configuration loading and validation
- [tests/conftest.py](tests/conftest.py): Test fixtures (temp_git_repo, test_db, etc.)
