# gcplocal

**A local GCP emulator for development and testing — Moto for Google Cloud.**

gcplocal runs a local HTTP server that emulates Google Cloud Platform APIs. Point Terraform, GCP client libraries, or any REST client at it and develop without touching real cloud infrastructure.

## Supported Services

| Service | Resources | Terraform Resources |
|---|---|---|
| **Compute Engine** | Instances, Networks, Subnetworks, Firewalls, Disks | `google_compute_instance`, `google_compute_network`, `google_compute_subnetwork`, `google_compute_firewall`, `google_compute_disk` |
| **Cloud Storage** | Buckets, Objects, Bucket IAM | `google_storage_bucket`, `google_storage_bucket_object` |
| **IAM** | Service Accounts, Keys, Custom Roles | `google_service_account`, `google_service_account_key`, `google_project_iam_custom_role` |
| **Pub/Sub** | Topics, Subscriptions | `google_pubsub_topic`, `google_pubsub_subscription` |
| **Cloud SQL** | Instances, Databases, Users | `google_sql_database_instance`, `google_sql_database`, `google_sql_user` |
| **Resource Manager** | Projects, IAM Policies | `google_project` |

## Installation

**From PyPI** (once published):

```bash
pip install gcplocal
```

**From GitHub**:

```bash
pip install git+https://github.com/gcplocal/gcplocal.git
```

**From source**:

```bash
git clone https://github.com/gcplocal/gcplocal.git
cd gcplocal
pip install -e .
```

**Docker**:

```bash
docker run -p 8400:8400 gcplocal/gcplocal
```

Or with docker-compose:

```bash
docker-compose up
```

## Quick Start

### Start the Emulator

```bash
gcplocal start
```

This starts the server on `http://0.0.0.0:8400` with all services enabled.

Options:
```bash
gcplocal start --host 127.0.0.1 --port 9000 --debug
```

### Use with Terraform

Generate the provider configuration:

```bash
gcplocal terraform-config
```

This outputs the HCL block you need. Or copy this into your `main.tf`:

```hcl
provider "google" {
  project = "gcplocal-project"
  region  = "us-central1"

  access_token = "gcplocal-mock-token"

  compute_custom_endpoint                = "http://localhost:8400/compute/v1/"
  storage_custom_endpoint                = "http://localhost:8400/storage/v1/"
  iam_custom_endpoint                    = "http://localhost:8400/v1/"
  pubsub_custom_endpoint                 = "http://localhost:8400/v1/"
  cloud_resource_manager_custom_endpoint = "http://localhost:8400/v1/"
  sql_custom_endpoint                    = "http://localhost:8400/sql/v1beta4/"
}
```

Then run Terraform as usual:

```bash
terraform init
terraform plan
terraform apply -auto-approve
```

### Use from Python

```python
from gcplocal import create_app

app = create_app()

with app.test_client() as client:
    # Create a Compute Engine instance
    client.post(
        "/compute/v1/projects/my-proj/zones/us-central1-a/instances",
        json={"name": "test-vm", "machineType": "e2-micro"},
    )

    # Read it back
    resp = client.get(
        "/compute/v1/projects/my-proj/zones/us-central1-a/instances/test-vm"
    )
    print(resp.get_json()["status"])  # RUNNING
```

## CLI Commands

| Command | Description |
|---|---|
| `gcplocal start` | Start the emulator server |
| `gcplocal terraform-config` | Generate Terraform provider config (HCL, JSON, or env vars) |
| `gcplocal status` | Check if the emulator is running |
| `gcplocal reset` | Clear all in-memory state |

### Terraform Config Formats

```bash
gcplocal terraform-config --format hcl   # Default: HCL provider block
gcplocal terraform-config --format json  # JSON for programmatic use
gcplocal terraform-config --format env   # Shell export statements
```

## API Endpoints

All endpoints mirror the real GCP REST APIs:

```
GET    /health                                          # Health check
POST   /reset                                           # Clear all state

# Compute Engine
POST   /compute/v1/projects/{p}/zones/{z}/instances     # Create instance
GET    /compute/v1/projects/{p}/zones/{z}/instances/{n} # Get instance
DELETE /compute/v1/projects/{p}/zones/{z}/instances/{n} # Delete instance
POST   /compute/v1/projects/{p}/global/networks         # Create network
POST   /compute/v1/projects/{p}/global/firewalls        # Create firewall
...

# Cloud Storage
POST   /storage/v1/b                                    # Create bucket
GET    /storage/v1/b/{bucket}                           # Get bucket
POST   /upload/storage/v1/b/{bucket}/o?name=...         # Upload object
...

# IAM
POST   /v1/projects/{p}/serviceAccounts                 # Create SA
GET    /v1/projects/{p}/serviceAccounts/{email}         # Get SA
...

# Pub/Sub
PUT    /v1/projects/{p}/topics/{topic}                  # Create topic
PUT    /v1/projects/{p}/subscriptions/{sub}             # Create subscription
...

# Cloud SQL
POST   /sql/v1beta4/projects/{p}/instances              # Create SQL instance
POST   /sql/v1beta4/projects/{p}/instances/{i}/databases # Create database
...
```

## How It Works

gcplocal is a Flask application that:

1. **Registers service blueprints** — each GCP service (Compute, Storage, IAM, etc.) is a separate module with routes matching the real GCP REST API
2. **Stores state in memory** — a thread-safe `StateStore` tracks all resources keyed by `(service, project, resource_type, resource_id)`
3. **Returns realistic responses** — JSON responses match the structure Terraform and GCP client libraries expect (selfLinks, operation objects, etags, etc.)
4. **Handles GCP Operations** — Compute operations return immediately as `DONE`, so Terraform doesn't wait for polling
5. **Fakes authentication** — all requests are accepted regardless of credentials

## Architecture

```
gcplocal/
  __init__.py          # Package entry point
  server.py            # Flask app factory, middleware, service registration
  state.py             # Thread-safe in-memory resource store
  auth.py              # Mock authentication
  cli.py               # Click CLI (start, terraform-config, status, reset)
  constants.py         # API versions, defaults
  utils.py             # Shared helpers (IDs, timestamps, pagination, errors)
  services/
    base.py            # Abstract service base class
    compute/           # Compute Engine (instances, networks, firewalls, disks)
    storage/           # Cloud Storage (buckets, objects)
    iam/               # IAM (service accounts, keys, roles)
    pubsub/            # Pub/Sub (topics, subscriptions)
    resourcemanager/   # Resource Manager (projects)
    sql/               # Cloud SQL (instances, databases, users)
  terraform/
    provider.py        # Generates Terraform provider config files
```

## Using in Your Test Suite

gcplocal ships as a **pytest plugin**. Install it in any project and the fixtures are available automatically:

```python
# No imports needed — fixtures are auto-discovered

def test_create_bucket(gcplocal_client):
    """gcplocal_client is a Flask test client, no server needed."""
    resp = gcplocal_client.post(
        "/storage/v1/b?project=my-proj",
        json={"name": "test-bucket", "location": "US"},
    )
    assert resp.status_code == 200
    assert resp.get_json()["name"] == "test-bucket"


def test_with_real_http(gcplocal_url):
    """gcplocal_url starts a real HTTP server on a random port."""
    import requests
    resp = requests.get(f"{gcplocal_url}/health")
    assert resp.json()["status"] == "healthy"


def test_terraform_endpoints(gcplocal_terraform_env):
    """Get a dict of all custom endpoints for Terraform config."""
    env = gcplocal_terraform_env
    assert "compute_custom_endpoint" in env
    assert env["url"].startswith("http://")
```

### Available Fixtures

| Fixture | Scope | Description |
|---|---|---|
| `gcplocal_client` | function | Flask test client (no HTTP server, fast) |
| `gcplocal_app` | function | Flask app instance for custom setups |
| `gcplocal_state` | function | Direct access to the state store (cleared per test) |
| `gcplocal_url` | session | Real HTTP server URL on a random port |
| `gcplocal_terraform_env` | function | Dict of all custom endpoints for Terraform |

## Running Tests

```bash
pip install -e '.[dev]'
pytest tests/ -v
```

## Publishing a Release

### To PyPI

1. Create a GitHub Release (tag like `v0.1.0`)
2. The `publish.yml` workflow automatically builds and uploads to PyPI via trusted publishing
3. Users can then `pip install gcplocal`

**Manual publish** (if not using GitHub Actions):

```bash
pip install build twine
python -m build
twine upload dist/*
```

### Docker Image

```bash
docker build -t gcplocal/gcplocal:latest .
docker push gcplocal/gcplocal:latest
```

## Auto-Update System

gcplocal includes a code generator that scans Google's API Discovery Service
and automatically generates emulator stubs for new or updated GCP services.

### How it works

1. **Scanner** (`tools/discovery.py`) queries `https://discovery.googleapis.com/discovery/v1/apis`
   for all GCP services and their REST API schemas
2. **Generator** (`tools/generator.py`) takes the discovery document and produces
   a complete Flask service module with routes for every CRUD method
3. **Registry** (`gcplocal/service_registry.json`) tracks auto-generated services,
   which the server loads dynamically at startup
4. **GitHub Action** (`.github/workflows/auto-update.yml`) runs weekly, generates
   any missing services, and opens a PR automatically

### Manual usage

```bash
# See what services are missing
python -m tools.update_services --scan-only

# Generate a specific service
python -m tools.update_services --service secretmanager

# Generate all missing services at once
python -m tools.update_services --all
```

Auto-generated services are registered in `service_registry.json` and loaded
automatically — no manual wiring needed.

### Weekly auto-update

The `auto-update.yml` GitHub Action:
- Runs every Monday at 06:00 UTC
- Scans the Discovery API for new/changed services
- Generates stubs for anything missing
- Runs the test suite
- Opens a PR if there are changes

You can also trigger it manually from the Actions tab.

## Adding New Services Manually

For hand-crafted (higher fidelity) service implementations:

1. Create a new directory under `gcplocal/services/`
2. Subclass `GCPService` and implement `register_routes()`
3. Register it in `gcplocal/server.py` → `_register_services()`
4. Add the endpoint constant in `constants.py`

```python
from gcplocal.services.base import GCPService

class MyNewService(GCPService):
    service_name = "myservice"
    api_prefix = "/myservice/v1"

    def register_routes(self, bp):
        bp.add_url_rule(
            f"{self.api_prefix}/projects/<project>/things",
            "list_things", self.list_things, methods=["GET"],
        )

    def list_things(self, project):
        items = self.state.list("myservice", project, "things")
        return jsonify({"items": items})
```

Hand-written services in `server.py` take priority over auto-generated ones
with the same name.

## License

Apache 2.0
