# Google Cloud Run vs Geneva: Image Embedding & Captioning Pipeline

This file compares Google Cloud Run (serverless container platform) with Geneva
for building production image embedding pipelines.

## Key Insight

Cloud Run excels at autoscaling containerized services — deploy a container,
it scales to zero when idle and up under load. Geneva is a feature engineering
platform that manages the data lifecycle.

**The fundamental difference:**
- **Cloud Run**: "Run this container when requests arrive" (container-centric)
- **Geneva**: "Maintain this feature column over time" (data-centric)

## Summary

| Approach | Lines of Code |
|----------|---------------|
| Cloud Run | ~315 lines + config for production ops (container + orchestrator + data mgmt) |
| Geneva | ~60 lines total, data management included automatically |

## Positioning

Cloud Run is Category 2 — Custom Model GPU Infrastructure. It's a general-purpose
serverless container platform that happens to support GPUs. Unlike Modal (Python-native
serverless) or Vertex AI (ML-specific managed endpoints), Cloud Run is the lowest-level
GCP serverless option: you bring your own container image and Cloud Run handles scaling.

Cloud Run is complementary to Geneva, not competitive:
- Cloud Run can serve as a GPU inference backend
- Geneva handles the data lifecycle (checkpointing, incremental, versioning)

---

## Cloud Run Approach

### Cloud Run Excels At

- Serverless container execution (any language, any framework)
- Scale-to-zero (no idle costs when not in use)
- GPU support (L4, A100 via Cloud Run GPU preview)
- Automatic HTTPS endpoints
- Tight GCP integration (IAM, Cloud Logging, Artifact Registry)
- Request-based and job-based execution models

### You'll Need to Add

- Container image build and management (Dockerfile)
- Data loading/writing (Cloud Run doesn't manage data)
- Checkpointing for fault tolerance
- Incremental processing logic
- Progress tracking for large datasets
- Orchestration for multi-step pipelines
- Batch processing logic (Cloud Run is request-oriented)

---

### Dockerfile

You must build and maintain a container image:

```dockerfile
FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04

RUN apt-get update && apt-get install -y python3 python3-pip
RUN pip install torch open-clip-torch transformers Pillow flask gunicorn

COPY app.py /app/app.py
WORKDIR /app

# Cloud Run expects an HTTP server
CMD ["gunicorn", "--bind", "0.0.0.0:8080", "--timeout", "300", "app:app"]
```

### HTTP Service (app.py deployed to Cloud Run)

Cloud Run requires an HTTP service. Your ML model must be wrapped
in a web server (Flask, FastAPI, etc.).

```python
from flask import Flask, request, jsonify

app = Flask(__name__)

# Global model cache (loaded once per container instance)
embedding_model = None
caption_model = None


def get_embedding_model():
    """Lazy-load CLIP model. Loaded once per container cold start."""
    global embedding_model
    if embedding_model is None:
        import open_clip
        model, _, preprocess = open_clip.create_model_and_transforms(
            'ViT-B-32', pretrained='laion2b_s34b_b79k'
        )
        model = model.cuda().eval()
        embedding_model = (model, preprocess)
    return embedding_model


def get_caption_model():
    """Lazy-load BLIP model. Loaded once per container cold start."""
    global caption_model
    if caption_model is None:
        from transformers import BlipProcessor, BlipForConditionalGeneration
        processor = BlipProcessor.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        )
        model = BlipForConditionalGeneration.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        ).cuda().eval()
        caption_model = (processor, model)
    return caption_model


@app.route("/embed", methods=["POST"])
def embed():
    """Embedding endpoint — single image."""
    import torch
    from PIL import Image
    import io

    image_bytes = request.data
    model, preprocess = get_embedding_model()

    img = Image.open(io.BytesIO(image_bytes)).convert('RGB')
    tensor = preprocess(img).unsqueeze(0).cuda()
    with torch.no_grad():
        embedding = model.encode_image(tensor)
    return jsonify({"embedding": embedding.cpu().numpy().flatten().tolist()})


@app.route("/embed_batch", methods=["POST"])
def embed_batch():
    """Batch embedding endpoint."""
    import torch
    from PIL import Image
    import io
    import base64

    data = request.json
    image_bytes_list = [base64.b64decode(img) for img in data["images"]]
    model, preprocess = get_embedding_model()

    tensors = []
    valid_indices = []
    for i, img_bytes in enumerate(image_bytes_list):
        try:
            img = Image.open(io.BytesIO(img_bytes)).convert('RGB')
            tensors.append(preprocess(img))
            valid_indices.append(i)
        except Exception:
            pass

    results = [None] * len(image_bytes_list)
    if tensors:
        batch = torch.stack(tensors).cuda()
        with torch.no_grad():
            embeddings = model.encode_image(batch).cpu().numpy()
        for idx, vi in enumerate(valid_indices):
            results[vi] = embeddings[idx].tolist()

    return jsonify({"embeddings": results})


@app.route("/caption", methods=["POST"])
def caption():
    """Caption endpoint — single image."""
    import torch
    from PIL import Image
    import io

    image_bytes = request.data
    processor, model = get_caption_model()

    img = Image.open(io.BytesIO(image_bytes)).convert('RGB')
    inputs = processor(img, return_tensors="pt")
    inputs = {k: v.cuda() for k, v in inputs.items()}
    with torch.no_grad():
        output = model.generate(**inputs, max_new_tokens=50)
    text = processor.decode(output[0], skip_special_tokens=True)
    return jsonify({"caption": text})


@app.route("/health", methods=["GET"])
def health():
    return "ok"
```

---

### Deployment & Infrastructure (you manage all of this)

Cloud Run requires significant infrastructure setup compared to Modal.

```bash
# 1. Build and push container image
gcloud builds submit --tag gcr.io/PROJECT/embedding-service .

# 2. Deploy to Cloud Run with GPU
gcloud run deploy embedding-service \
    --image gcr.io/PROJECT/embedding-service \
    --gpu 1 \
    --gpu-type nvidia-l4 \
    --memory 16Gi \
    --cpu 4 \
    --max-instances 10 \
    --min-instances 0 \
    --timeout 300 \
    --region us-central1 \
    --allow-unauthenticated

# 3. Repeat for caption service (or combine into one)
gcloud run deploy caption-service \
    --image gcr.io/PROJECT/caption-service \
    --gpu 1 \
    --gpu-type nvidia-l4 \
    --memory 16Gi \
    --cpu 4 \
    --max-instances 10 \
    --min-instances 0 \
    --timeout 300 \
    --region us-central1 \
    --allow-unauthenticated
```

---

### Client-Side Orchestration (you build all of this)

The Cloud Run service handles inference. But you still need client-side code
to orchestrate the pipeline: load data, call the service, handle failures,
manage checkpoints, write results.

```python
import requests
import base64


class CloudRunInferenceClient:
    """Client for calling Cloud Run inference endpoints."""

    def __init__(self, embed_url: str, caption_url: str, batch_size: int = 32):
        self.embed_url = embed_url
        self.caption_url = caption_url
        self.batch_size = batch_size
        self.session = requests.Session()

    def embed_image(self, image_bytes: bytes) -> list[float]:
        resp = self.session.post(
            f"{self.embed_url}/embed",
            data=image_bytes,
            headers={"Content-Type": "application/octet-stream"},
            timeout=60,
        )
        resp.raise_for_status()
        return resp.json()["embedding"]

    def embed_batch(self, image_bytes_list: list[bytes]) -> list[list[float]]:
        payload = {
            "images": [base64.b64encode(img).decode() for img in image_bytes_list]
        }
        resp = self.session.post(
            f"{self.embed_url}/embed_batch",
            json=payload,
            timeout=120,
        )
        resp.raise_for_status()
        return resp.json()["embeddings"]

    def caption_image(self, image_bytes: bytes) -> str:
        resp = self.session.post(
            f"{self.caption_url}/caption",
            data=image_bytes,
            headers={"Content-Type": "application/octet-stream"},
            timeout=60,
        )
        resp.raise_for_status()
        return resp.json()["caption"]
```

### Basic Pipeline (no production ops)

```python
def run_cloudrun_pipeline_basic(input_path: str, output_path: str):
    """
    Basic Cloud Run pipeline — call endpoints, collect results.

    NOTE: Cloud Run doesn't manage data. You bring your own storage,
    orchestration, and error handling.
    """
    import pyarrow.parquet as pq
    import pyarrow as pa

    client = CloudRunInferenceClient(
        embed_url="https://embedding-service-xyz.run.app",
        caption_url="https://caption-service-xyz.run.app",
    )

    # Read data (you manage this)
    table = pq.read_table(input_path)
    images = table['image'].to_pylist()

    # Process embeddings in batches
    all_embeddings = []
    for i in range(0, len(images), client.batch_size):
        batch = images[i:i + client.batch_size]
        embeddings = client.embed_batch(batch)
        all_embeddings.extend(embeddings)

    # Process captions one at a time (or batch if your service supports it)
    captions = [client.caption_image(img) for img in images]

    # Build output table (you manage schema)
    output = pa.table({
        'id': table['id'],
        'image': table['image'],
        'embedding': pa.array(all_embeddings),
        'caption': pa.array(captions),
    })

    # Write results (you manage this)
    pq.write_table(output, output_path)
```

---

### Production Pipeline (with custom ops)

For production, you need to add:
1. **Checkpointing** — Resume from failures
2. **Incremental processing** — Only process new data
3. **Progress tracking** — Know where you are in large datasets
4. **Error handling** — Retry on 5xx, handle cold starts
5. **Data management** — Track versions, schema changes
6. **Rate limiting** — Don't overwhelm Cloud Run concurrency limits
7. **Authentication** — IAM tokens for authenticated services

Cloud Run provides compute. Everything else is your responsibility.
Here's what you'd build:

```python
import json
from pathlib import Path
from datetime import datetime
import time


class CloudRunCheckpointManager:
    """
    Custom checkpoint manager for Cloud Run pipelines.

    Cloud Run doesn't include checkpointing — you build this yourself.
    """

    def __init__(self, checkpoint_path: str):
        self.checkpoint_path = Path(checkpoint_path)
        self.checkpoint_path.parent.mkdir(parents=True, exist_ok=True)
        self.state = self._load_state()

    def _load_state(self) -> dict:
        if self.checkpoint_path.exists():
            return json.loads(self.checkpoint_path.read_text())
        return {
            "processed_ids": [],
            "last_checkpoint": None,
            "total_processed": 0,
            "errors": [],
        }

    def _save_state(self):
        self.state["last_checkpoint"] = datetime.now().isoformat()
        self.checkpoint_path.write_text(json.dumps(self.state, indent=2))

    def mark_processed(self, item_id: str):
        self.state["processed_ids"].append(item_id)
        self.state["total_processed"] += 1
        if self.state["total_processed"] % 100 == 0:
            self._save_state()

    def is_processed(self, item_id: str) -> bool:
        return item_id in set(self.state["processed_ids"])

    def mark_error(self, item_id: str, error: str):
        self.state["errors"].append({"id": item_id, "error": error})
        self._save_state()


class CloudRunBatchProcessor:
    """
    Batch processor with checkpointing for Cloud Run.

    Handles:
    - Tracking progress through large datasets
    - Checkpointing for resume
    - Error handling and retries
    - Cold start awareness (Cloud Run containers may take 30-60s to start)
    - Concurrency management

    You build this yourself — Cloud Run doesn't include it.
    """

    def __init__(
        self,
        input_path: str,
        output_path: str,
        checkpoint_path: str,
        embed_url: str,
        caption_url: str,
        batch_size: int = 32,
        max_retries: int = 3,
    ):
        self.input_path = input_path
        self.output_path = output_path
        self.batch_size = batch_size
        self.max_retries = max_retries
        self.checkpoint = CloudRunCheckpointManager(checkpoint_path)
        self.client = CloudRunInferenceClient(embed_url, caption_url, batch_size)

    def _call_with_retry(self, fn, *args):
        """Retry with backoff — handles Cloud Run cold starts and 5xx errors."""
        for attempt in range(self.max_retries):
            try:
                return fn(*args)
            except requests.exceptions.HTTPError as e:
                if e.response.status_code >= 500 and attempt < self.max_retries - 1:
                    wait = 2 ** attempt * 5  # Longer backoff for cold starts
                    print(f"Cloud Run 5xx, retrying in {wait}s (attempt {attempt + 1})")
                    time.sleep(wait)
                else:
                    raise
            except requests.exceptions.ConnectionError:
                if attempt < self.max_retries - 1:
                    wait = 2 ** attempt * 5
                    print(f"Connection error, retrying in {wait}s (attempt {attempt + 1})")
                    time.sleep(wait)
                else:
                    raise

    def run(self):
        import pyarrow.parquet as pq
        import pyarrow as pa

        table = pq.read_table(self.input_path)

        # Filter to unprocessed items
        pending = []
        for i in range(len(table)):
            item_id = str(table['id'][i].as_py())
            if not self.checkpoint.is_processed(item_id):
                pending.append({
                    'id': item_id,
                    'image': table['image'][i].as_py(),
                    'index': i,
                })

        print(f"Processing {len(pending)} items "
              f"({len(table) - len(pending)} already done)")

        if not pending:
            return

        results = []

        # Process in batches
        for batch_start in range(0, len(pending), self.batch_size):
            batch = pending[batch_start:batch_start + self.batch_size]
            batch_images = [item['image'] for item in batch]

            try:
                # Call Cloud Run endpoints with retry
                embeddings = self._call_with_retry(
                    self.client.embed_batch, batch_images
                )
                captions = []
                for img in batch_images:
                    caption = self._call_with_retry(
                        self.client.caption_image, img
                    )
                    captions.append(caption)

                for i, item in enumerate(batch):
                    results.append({
                        'id': item['id'],
                        'embedding': embeddings[i],
                        'caption': captions[i],
                    })
                    self.checkpoint.mark_processed(item['id'])

                print(f"Progress: {len(results)}/{len(pending)}")

            except Exception as e:
                print(f"Batch failed: {e}")
                for item in batch:
                    self.checkpoint.mark_error(item['id'], str(e))
                raise

        self._write_results(results)
        print(f"Complete! Processed {len(results)} items")

    def _write_results(self, results: list):
        import pyarrow.parquet as pq
        import pyarrow as pa

        output = pa.table({
            'id': pa.array([r['id'] for r in results]),
            'embedding': pa.array([r['embedding'] for r in results]),
            'caption': pa.array([r['caption'] for r in results]),
        })

        pq.write_table(output, self.output_path)


def main():
    """Entry point for Cloud Run pipeline."""
    processor = CloudRunBatchProcessor(
        input_path="gs://my-bucket/images/input.parquet",
        output_path="gs://my-bucket/images/output.parquet",
        checkpoint_path="/tmp/cloudrun_checkpoint.json",
        embed_url="https://embedding-service-xyz.run.app",
        caption_url="https://caption-service-xyz.run.app",
        batch_size=32,
    )
    processor.run()
```

---

## Geneva Approach

Geneva provides:
- Same UDF simplicity — no container builds, no HTTP wrappers
- Built-in data management (tables, versions)
- Built-in checkpointing at fragment AND batch level
- Automatic incremental processing
- Dependency isolation per UDF
- No custom orchestration code needed

### UDF Definitions

```python
import geneva


@geneva.udf(num_gpus=1, version="0.1")
class ComputeEmbeddingGeneva:
    """
    CLIP embedding UDF for Geneva.

    No Dockerfile. No Flask wrapper. No HTTP endpoint.
    Just ML logic.
    """

    def setup(self):
        import open_clip
        self.model, _, self.preprocess = open_clip.create_model_and_transforms(
            'ViT-B-32', pretrained='laion2b_s34b_b79k'
        )
        self.model = self.model.cuda().eval()

    def __call__(self, image: bytes) -> list[float]:
        from PIL import Image
        import torch
        import io

        img = Image.open(io.BytesIO(image)).convert('RGB')
        tensor = self.preprocess(img).unsqueeze(0).cuda()
        with torch.no_grad():
            embedding = self.model.encode_image(tensor)
        return embedding.cpu().numpy().flatten().tolist()


@geneva.udf(num_gpus=1, version="0.1")
class GenerateCaptionGeneva:
    """BLIP captioning UDF for Geneva."""

    def setup(self):
        from transformers import BlipProcessor, BlipForConditionalGeneration
        self.processor = BlipProcessor.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        )
        self.model = BlipForConditionalGeneration.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        ).cuda()

    def __call__(self, image: bytes) -> str:
        from PIL import Image
        import torch
        import io

        img = Image.open(io.BytesIO(image)).convert('RGB')
        inputs = self.processor(img, return_tensors="pt")
        inputs = {k: v.cuda() for k, v in inputs.items()}
        with torch.no_grad():
            output = self.model.generate(**inputs, max_new_tokens=50)
        return self.processor.decode(output[0], skip_special_tokens=True)
```

### Geneva Pipeline

```python
def run_geneva_pipeline():
    """
    Geneva pipeline — data management and compute unified.

    No Dockerfile. No HTTP server. No deployment commands.
    No custom checkpoint code needed.
    No incremental processing code needed.
    No separate data loading/writing code needed.
    """
    conn = geneva.connect("s3://my-bucket/geneva-workspace")
    table = conn.open_table("images")

    table.add_columns({
        "embedding": ComputeEmbeddingGeneva,
        "caption": GenerateCaptionGeneva,
    })

    table.backfill("embedding", batch_size=100, commit_granularity=10)
    table.backfill("caption", batch_size=50, commit_granularity=10)

    print(f"Processed {table.count_rows()} images")
    print(f"Table version: {table.version}")


def run_geneva_incremental():
    """
    Incremental processing in Geneva.

    Add new data -> backfill -> only new rows processed.
    No custom code needed.
    """
    conn = geneva.connect("s3://my-bucket/geneva-workspace")
    table = conn.open_table("images")

    new_images = load_new_images()  # Your data loading
    table.add(new_images)

    # Backfill only processes rows where embedding IS NULL
    table.backfill("embedding", batch_size=100)
    table.backfill("caption", batch_size=50)
```

---

## Side-by-Side Comparison

### Feature Comparison

| Feature | Cloud Run | Geneva |
|---------|-----------|--------|
| Serverless containers | ✓ Core strength | ✗ Bring your cluster |
| Scale to zero | ✓ Automatic | ✗ Cluster-based |
| GPU support | ✓ L4, A100 (preview) | ✓ Any GPU on Ray |
| Any language/framework | ✓ Container-based | ✗ Python UDFs |
| GCP IAM integration | ✓ Native | ~ Via service acct |
| HTTPS endpoints | ✓ Automatic | ✗ Not applicable |
| Pay-per-request billing | ✓ Yes | ✗ Cluster-based |
| | | |
| No container/Dockerfile | ✗ Required | ✓ Just Python |
| No HTTP wrapper needed | ✗ Required | ✓ Direct function |
| Data management | ✗ Not included | ✓ Tables, versions |
| Checkpointing | ✗ Build yourself | ✓ Automatic |
| Incremental processing | ✗ Build yourself | ✓ Automatic |
| Resume from failure | ✗ Build yourself | ✓ Automatic |
| Schema management | ✗ Build yourself | ✓ From type hints |
| Batch orchestration | ✗ Build yourself | ✓ Built-in |
| Progress visibility | ~ Cloud Logging | ✓ Real-time commits |

*Legend: ✓ = Included, ~ = Partial/Manual, ✗ = You Build*

### Code Comparison

| Component | Cloud Run | Geneva |
|-----------|-----------|--------|
| Dockerfile | ~15 lines | 0 (not needed) |
| HTTP service (Flask/etc) | ~65 lines | 0 (not needed) |
| Deployment config (gcloud) | ~20 lines | 0 (not needed) |
| Client + HTTP calls | ~45 lines | 0 (not needed) |
| UDF / model logic | (in service above) | ~45 lines |
| Basic pipeline | ~25 lines | ~15 lines |
| Checkpoint system | ~50 lines (custom) | 0 (included) |
| Batch processor + retry | ~80 lines (custom) | 0 (included) |
| Data loading/writing | ~15 lines | 0 (included) |
| **TOTAL (production)** | **~315 lines + config** | **~60 lines** |

### Cloud Run vs Modal (both serverless, different abstraction level)

| Aspect | Cloud Run | Modal |
|--------|-----------|-------|
| Abstraction level | Container-level | Function-level |
| Language support | Any (container) | Python only |
| GPU billing | Per-instance-second | Per-second (finer) |
| Cold start | 30-60s (GPU) | ~5-15s (cached) |
| Container management | You build/push | Automatic from code |
| HTTP boilerplate | Required | Not needed |
| GCP ecosystem | Native | Separate |
| Deployment | gcloud CLI / Terraform | `modal deploy` |
| Minimum instances | Configurable | Configurable |
| Max request timeout | 60 min | 24 hr |
| Typical setup overhead | High (container+IAM) | Low (decorators) |

---

## When to Use Each

### Cloud Run is great for

- Teams already on GCP with existing container workflows
- Multi-language inference services (not just Python)
- Services that need GCP IAM, VPC-SC, Cloud Armor
- When you need an HTTPS endpoint for real-time serving AND batch
- Compliance requirements that mandate GCP-native services
- Teams with existing CI/CD for container builds

### Cloud Run is NOT ideal for

- Pure batch feature engineering (too much HTTP boilerplate)
- Rapid ML experimentation (container build cycle is slow)
- Teams without container expertise
- When you need tight data lifecycle management

### Geneva is great for

- Feature engineering with ongoing data updates
- Teams that need data versioning and time travel
- Production pipelines with fault tolerance requirements
- When incremental processing is critical
- When you want zero infrastructure boilerplate
- When your UDFs are Python (the vast majority of ML)

---

## The Key Insight

Cloud Run asks: "How do I deploy and scale this container?"
Geneva asks: "How do I maintain this data over time?"

Cloud Run is a container platform — brilliant at running any workload serverlessly.
Geneva is a data platform — manages features as evolving datasets.

**Cloud Run requires you to:**
1. Write a Dockerfile
2. Wrap your model in an HTTP server
3. Build and push container images
4. Deploy with gcloud/Terraform
5. Write client code to call the service
6. Build checkpointing, incremental processing, and orchestration

**Geneva requires you to:**
1. Write a UDF class with `setup()` and `__call__()`
2. Call `backfill()`

That's it.

---

## Architecture Comparison

### Cloud Run Architecture (you build the gray boxes)

```mermaid
graph TB
    subgraph client["Your Code (Client Side)"]
        dl["Data Loading<br/>(custom)"]
        cp["Checkpoint Store<br/>(custom)"]
        inc["Incremental Processor<br/>(custom)"]
        pt["Progress Tracking<br/>(custom)"]
        http["HTTP Client<br/>(custom)"]
        retry["Retry / Backoff<br/>(custom)"]
    end

    subgraph service["Your Code (Service Side)"]
        docker["Dockerfile<br/>(custom)"]
        flask["Flask/FastAPI wrapper<br/>(custom)"]
        model["Model Logic<br/>(custom)"]
    end

    subgraph cloudrun["Google Cloud Run"]
        ce["Container Execution"]
        as["Auto-scaling"]
        gpu["GPU (L4/A100)"]
        s2z["Scale to Zero"]
    end

    subgraph gcp["Google Cloud"]
        ar["Artifact Registry"]
        iam["IAM"]
        log["Cloud Logging"]
        gcs["GCS"]
    end

    dl --> http
    http -- "HTTPS" --> flask
    flask --> model
    service --> cloudrun
    cloudrun --> gcp

    style client fill:#f0f0f0,stroke:#999
    style service fill:#f0f0f0,stroke:#999
    style cloudrun fill:#4285f4,stroke:#1a73e8,color:#fff
    style gcp fill:#e8f0fe,stroke:#4285f4
```

### Geneva Architecture (everything included)

```mermaid
graph TB
    subgraph yourcode["Your Code"]
        udfs["UDFs<br/>(ML logic)"]
        pipeline["Pipeline<br/>(backfill)"]
    end

    subgraph geneva["Geneva"]
        dm["Data Management"]
        ckpt["Checkpoint Store"]
        incp["Incremental Processor"]
        prog["Progress Tracking"]
        ver["Version Control"]
        udf_sys["UDF System"]
        gpu_sched["GPU Scheduling"]
        dist["Distributed Execution"]
        dep["Dependency Isolation"]
    end

    subgraph infra["Ray Cluster + Lance Storage"]
    end

    yourcode --> geneva
    geneva --> infra

    style yourcode fill:#f0f0f0,stroke:#999
    style geneva fill:#34a853,stroke:#1e8e3e,color:#fff
    style infra fill:#e6f4ea,stroke:#34a853
```

### Cloud Run + Geneva (complementary usage)

```mermaid
graph TB
    subgraph geneva["Geneva (CPU-only cluster)"]
        udf["UDF<br/>(calls Cloud Run)"]
        ckpt["Checkpoint Store"]
        incp["Incremental Processor"]
        ver["Version Control"]
    end

    subgraph cloudrun["Google Cloud Run (GPU)"]
        serve["Serves inference — scales automatically"]
    end

    udf -- "HTTPS" --> serve

    style geneva fill:#34a853,stroke:#1e8e3e,color:#fff
    style cloudrun fill:#4285f4,stroke:#1a73e8,color:#fff
```

Best of both: Geneva handles data lifecycle, Cloud Run handles GPU scaling.
Geneva cluster is CPU-only (cheap). GPU cost is pay-per-use via Cloud Run.

---

## Cost Comparison

### Cloud Run GPU Pricing (2025)

| GPU Type | Per vCPU-second | Per GiB-second | Per GPU-second |
|----------|-----------------|----------------|----------------|
| L4 | $0.000024 | $0.0000025 | ~$0.000234 |
| A100 40G | (preview) | (preview) | ~$0.0007 |

Approximate hourly cost (GPU + 4 vCPU + 16 GiB):

| GPU | Per Hour (active) | Idle (scale to 0) |
|-----|-------------------|-------------------|
| L4 | ~$1.00 | $0.00 |
| A100 | ~$2.80 | $0.00 |

Note: Cloud Run charges per 100ms increments (not per second like Modal).
Minimum instance charges apply if min-instances > 0.

### Batch Workload Comparison (1M images @ 100ms each, ~28 hours GPU time)

| Approach | Cost | Notes |
|----------|------|-------|
| Cloud Run L4 | ~$28 | Scale to 0 when done, cold starts |
| Cloud Run L4 (min=1) | ~$28+ | No cold starts, idle cost accrues |
| Modal A10G | ~$31 | No idle cost, faster cold starts |
| EC2 Spot g5 | ~$8-10 | Cheapest at scale, cluster overhead |
| Gemini API | ~$10-65 | No GPU at all |

Cloud Run is cost-competitive with Modal for batch workloads.
EC2 Spot wins for sustained batch jobs. FM APIs win for supported models.

---

## Cloud Run + Geneva Integration Pattern

For teams that want Cloud Run for GPU inference + Geneva for data lifecycle:

1. Deploy your model to Cloud Run (handles scaling, GPUs)
2. Write a Geneva UDF that calls the Cloud Run endpoint
3. Geneva handles checkpointing, incremental, versioning
4. Cloud Run GPU cluster is separate — scale independently

```python
import geneva


@geneva.udf(version="0.1")
class CloudRunEmbeddingUDF:
    """
    Geneva UDF that calls Cloud Run for inference.

    - Geneva handles: checkpointing, incremental, retry, versioning
    - Cloud Run handles: GPU compute, autoscaling, container lifecycle
    - No GPU needed on Geneva cluster (CPU-only = cheap)
    """

    def setup(self):
        import requests
        self.session = requests.Session()
        self.url = "https://embedding-service-xyz.run.app"

    def __call__(self, image: bytes) -> list[float]:
        resp = self.session.post(
            f"{self.url}/embed",
            data=image,
            headers={"Content-Type": "application/octet-stream"},
            timeout=60,
        )
        resp.raise_for_status()
        return resp.json()["embedding"]


@geneva.udf(version="0.1")
class CloudRunCaptionUDF:
    """Geneva UDF that calls Cloud Run for captioning."""

    def setup(self):
        import requests
        self.session = requests.Session()
        self.url = "https://caption-service-xyz.run.app"

    def __call__(self, image: bytes) -> str:
        resp = self.session.post(
            f"{self.url}/caption",
            data=image,
            headers={"Content-Type": "application/octet-stream"},
            timeout=60,
        )
        resp.raise_for_status()
        return resp.json()["caption"]


def run_cloudrun_plus_geneva():
    """
    Cloud Run + Geneva: best of both worlds.

    - Geneva cluster: CPU-only, cheap
    - Cloud Run: GPU inference, pay-per-use
    - Data lifecycle: fully managed by Geneva
    """
    conn = geneva.connect("gs://my-bucket/geneva-workspace")
    table = conn.open_table("images")

    table.add_columns({
        "embedding": CloudRunEmbeddingUDF,
        "caption": CloudRunCaptionUDF,
    })

    # Geneva handles checkpointing + incremental
    # Cloud Run handles GPU scaling
    table.backfill("embedding", batch_size=100, commit_granularity=10)
    table.backfill("caption", batch_size=50, commit_granularity=10)
```
