# Modal vs Geneva: Image Embedding & Captioning Pipeline

This file compares Modal (serverless GPU platform) with Geneva
for building production image embedding pipelines.

## Key Insight

Modal excels at serverless GPU compute - define a function, call it, get results.
Geneva is a feature engineering platform that manages the data lifecycle.

**The fundamental difference:**
- **Modal**: "Run this function on a GPU" (compute-centric)
- **Geneva**: "Maintain this feature column over time" (data-centric)

## Summary

| Approach | Lines of Code |
|----------|---------------|
| Modal | ~180 lines for production ops (no built-in data management) |
| Geneva | ~60 lines total, data management included automatically |

---

## Modal Approach

### Modal Excels At

- Serverless GPU compute (no cluster management)
- Python-native function decorators
- Automatic container building and caching
- Pay-per-second GPU billing
- Cold start optimization

### You'll Need to Add

- Data loading/writing (Modal doesn't manage data)
- Checkpointing for fault tolerance
- Incremental processing logic
- Progress tracking for large datasets
- Orchestration for multi-step pipelines

---

### UDF Definitions

```python
import modal

# Create Modal app
app = modal.App("image-embeddings")

# Define the container image with dependencies
image = modal.Image.debian_slim(python_version="3.11").pip_install(
    "torch>=2.1.0",
    "open-clip-torch>=2.24.0",
    "transformers>=4.36.0",
    "Pillow>=10.0.0",
    "pyarrow>=14.0.0",
)


@app.cls(gpu="A10G", image=image, container_idle_timeout=300)
class EmbeddingModel:
    """
    Modal class for CLIP embeddings.

    Similar pattern to Geneva UDFs:
    - Class-based for model caching
    - Decorator for GPU allocation
    - Method for processing
    """

    @modal.enter()
    def load_model(self):
        """Called once when container starts - like Geneva's setup()."""
        import open_clip
        import torch

        self.model, _, self.preprocess = open_clip.create_model_and_transforms(
            'ViT-B-32', pretrained='laion2b_s34b_b79k'
        )
        self.model = self.model.cuda().eval()
        self.device = "cuda"

    @modal.method()
    def compute_embedding(self, image_bytes: bytes) -> list[float]:
        """Process a single image."""
        from PIL import Image
        import torch
        import io

        img = Image.open(io.BytesIO(image_bytes)).convert('RGB')
        tensor = self.preprocess(img).unsqueeze(0).cuda()
        with torch.no_grad():
            embedding = self.model.encode_image(tensor)
        return embedding.cpu().numpy().flatten().tolist()

    @modal.method()
    def compute_batch(self, image_bytes_list: list[bytes]) -> list[list[float]]:
        """Process a batch of images for efficiency."""
        from PIL import Image
        import torch
        import io

        results = []
        tensors = []

        for img_bytes in image_bytes_list:
            if img_bytes is None:
                tensors.append(None)
                continue
            img = Image.open(io.BytesIO(img_bytes)).convert('RGB')
            tensors.append(self.preprocess(img))

        valid_indices = [i for i, t in enumerate(tensors) if t is not None]
        if valid_indices:
            batch = torch.stack([tensors[i] for i in valid_indices]).cuda()
            with torch.no_grad():
                embeddings = self.model.encode_image(batch).cpu().numpy()

            embed_idx = 0
            for i in range(len(tensors)):
                if i in valid_indices:
                    results.append(embeddings[embed_idx].tolist())
                    embed_idx += 1
                else:
                    results.append(None)
        else:
            results = [None] * len(image_bytes_list)

        return results


@app.cls(gpu="A10G", image=image, container_idle_timeout=300)
class CaptionModel:
    """Modal class for BLIP captioning."""

    @modal.enter()
    def load_model(self):
        from transformers import BlipProcessor, BlipForConditionalGeneration

        self.processor = BlipProcessor.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        )
        self.model = BlipForConditionalGeneration.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        ).cuda().eval()

    @modal.method()
    def generate_caption(self, image_bytes: bytes) -> str:
        from PIL import Image
        import torch
        import io

        img = Image.open(io.BytesIO(image_bytes)).convert('RGB')
        inputs = self.processor(img, return_tensors="pt")
        inputs = {k: v.cuda() for k, v in inputs.items()}
        with torch.no_grad():
            output = self.model.generate(**inputs, max_new_tokens=50)
        return self.processor.decode(output[0], skip_special_tokens=True)
```

### Basic Pipeline (no production ops)

```python
@app.function(image=image)
def run_modal_pipeline_basic(input_path: str, output_path: str):
    """
    Basic Modal pipeline - simple and clean for one-shot jobs.

    NOTE: Modal doesn't manage data - you bring your own storage.
    """
    import pyarrow.parquet as pq
    import pyarrow as pa

    # Read data (you manage this)
    table = pq.read_table(input_path)

    # Get model instances
    embedding_model = EmbeddingModel()
    caption_model = CaptionModel()

    # Process all images
    images = table['image'].to_pylist()

    # Parallel processing with Modal's map
    embeddings = list(embedding_model.compute_embedding.map(images))
    captions = list(caption_model.generate_caption.map(images))

    # Build output table (you manage schema)
    output = pa.table({
        'id': table['id'],
        'image': table['image'],
        'embedding': pa.array(embeddings),
        'caption': pa.array(captions),
    })

    # Write results (you manage this)
    pq.write_table(output, output_path)

    print("Pipeline complete!")
```

### Production Pipeline (with custom ops)

For production, you need to add:
1. **Checkpointing** - Resume from failures
2. **Incremental processing** - Only process new data
3. **Progress tracking** - Know where you are in large datasets
4. **Error handling** - Don't lose work on failures
5. **Data management** - Track versions, schema changes

Modal provides compute. Everything else is your responsibility.

```python
import json
from pathlib import Path
from datetime import datetime


class ModalCheckpointManager:
    """
    Custom checkpoint manager for Modal pipelines.

    Modal doesn't include checkpointing - you build this yourself.
    """

    def __init__(self, checkpoint_path: str):
        self.checkpoint_path = Path(checkpoint_path)
        self.checkpoint_path.parent.mkdir(parents=True, exist_ok=True)
        self.state = self._load_state()

    def _load_state(self) -> dict:
        if self.checkpoint_path.exists():
            return json.loads(self.checkpoint_path.read_text())
        return {
            "processed_ids": [],
            "last_checkpoint": None,
            "total_processed": 0,
            "errors": [],
        }

    def _save_state(self):
        self.state["last_checkpoint"] = datetime.now().isoformat()
        self.checkpoint_path.write_text(json.dumps(self.state, indent=2))

    def mark_processed(self, item_id: str):
        self.state["processed_ids"].append(item_id)
        self.state["total_processed"] += 1
        if self.state["total_processed"] % 100 == 0:
            self._save_state()

    def is_processed(self, item_id: str) -> bool:
        return item_id in set(self.state["processed_ids"])

    def mark_error(self, item_id: str, error: str):
        self.state["errors"].append({"id": item_id, "error": error})
        self._save_state()


class ModalBatchProcessor:
    """
    Batch processor with checkpointing for Modal.

    Handles:
    - Tracking progress through large datasets
    - Checkpointing for resume
    - Error handling

    You build this yourself - Modal doesn't include it.
    """

    def __init__(
        self,
        input_path: str,
        output_path: str,
        checkpoint_path: str,
        batch_size: int = 100,
    ):
        self.input_path = input_path
        self.output_path = output_path
        self.batch_size = batch_size
        self.checkpoint = ModalCheckpointManager(checkpoint_path)

    def run(self):
        import pyarrow.parquet as pq
        import pyarrow as pa

        # Read input data
        table = pq.read_table(self.input_path)

        # Filter to unprocessed items
        pending = []
        for i in range(len(table)):
            item_id = str(table['id'][i].as_py())
            if not self.checkpoint.is_processed(item_id):
                pending.append({
                    'id': item_id,
                    'image': table['image'][i].as_py(),
                    'index': i,
                })

        print(f"Processing {len(pending)} items ({len(table) - len(pending)} already done)")

        if not pending:
            return

        # Get Modal model instances
        embedding_model = EmbeddingModel()
        caption_model = CaptionModel()

        results = []

        # Process in batches
        for batch_start in range(0, len(pending), self.batch_size):
            batch = pending[batch_start:batch_start + self.batch_size]
            batch_images = [item['image'] for item in batch]

            try:
                # Call Modal functions
                embeddings = list(embedding_model.compute_batch.remote(batch_images))
                captions = list(caption_model.generate_caption.map(batch_images))

                # Collect results
                for i, item in enumerate(batch):
                    results.append({
                        'id': item['id'],
                        'embedding': embeddings[i],
                        'caption': captions[i],
                    })
                    self.checkpoint.mark_processed(item['id'])

                print(f"Progress: {len(results)}/{len(pending)}")

            except Exception as e:
                print(f"Batch failed: {e}")
                for item in batch:
                    self.checkpoint.mark_error(item['id'], str(e))
                raise

        # Write results
        self._write_results(results)
        print(f"Complete! Processed {len(results)} items")

    def _write_results(self, results: list):
        import pyarrow.parquet as pq
        import pyarrow as pa

        output = pa.table({
            'id': pa.array([r['id'] for r in results]),
            'embedding': pa.array([r['embedding'] for r in results]),
            'caption': pa.array([r['caption'] for r in results]),
        })

        pq.write_table(output, self.output_path)


@app.local_entrypoint()
def main():
    """Entry point for Modal CLI."""
    processor = ModalBatchProcessor(
        input_path="s3://my-bucket/images/input.parquet",
        output_path="s3://my-bucket/images/output.parquet",
        checkpoint_path="/tmp/modal_checkpoint.json",
        batch_size=100,
    )
    processor.run()
```

---

## Geneva Approach

Geneva provides:
- Same UDF simplicity as Modal
- Built-in data management (tables, versions)
- Built-in checkpointing at fragment AND batch level
- Automatic incremental processing
- Dependency isolation per UDF
- No custom orchestration code needed

### UDF Definitions

```python
import geneva


@geneva.udf(num_gpus=1, version="0.1")
class ComputeEmbeddingGeneva:
    """
    CLIP embedding UDF for Geneva.

    Very similar to Modal's class pattern:
    - setup() instead of @modal.enter()
    - Type annotation -> schema (no separate schema definition)
    - Geneva handles batching internally
    """

    def setup(self):
        import open_clip
        self.model, _, self.preprocess = open_clip.create_model_and_transforms(
            'ViT-B-32', pretrained='laion2b_s34b_b79k'
        )
        self.model = self.model.cuda().eval()

    def __call__(self, image: bytes) -> list[float]:
        from PIL import Image
        import torch
        import io

        img = Image.open(io.BytesIO(image)).convert('RGB')
        tensor = self.preprocess(img).unsqueeze(0).cuda()
        with torch.no_grad():
            embedding = self.model.encode_image(tensor)
        return embedding.cpu().numpy().flatten().tolist()


@geneva.udf(num_gpus=1, version="0.1")
class GenerateCaptionGeneva:
    """BLIP captioning UDF for Geneva."""

    def setup(self):
        from transformers import BlipProcessor, BlipForConditionalGeneration
        self.processor = BlipProcessor.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        )
        self.model = BlipForConditionalGeneration.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        ).cuda()

    def __call__(self, image: bytes) -> str:
        from PIL import Image
        import torch
        import io

        img = Image.open(io.BytesIO(image)).convert('RGB')
        inputs = self.processor(img, return_tensors="pt")
        inputs = {k: v.cuda() for k, v in inputs.items()}
        with torch.no_grad():
            output = self.model.generate(**inputs, max_new_tokens=50)
        return self.processor.decode(output[0], skip_special_tokens=True)
```

### Pipeline

```python
def run_geneva_pipeline():
    """
    Geneva pipeline - data management and compute unified.

    No custom checkpoint code needed.
    No incremental processing code needed.
    No separate data loading/writing code needed.
    """
    conn = geneva.connect("s3://my-bucket/geneva-workspace")
    table = conn.open_table("images")

    # Add computed columns
    table.add_columns({
        "embedding": ComputeEmbeddingGeneva,
        "caption": GenerateCaptionGeneva,
    })

    # Run with automatic checkpointing, incremental processing, versioning
    table.backfill("embedding", batch_size=100, commit_granularity=10)
    table.backfill("caption", batch_size=50, commit_granularity=10)

    print(f"Processed {table.count_rows()} images")
    print(f"Table version: {table.version}")


def run_geneva_incremental():
    """
    Incremental processing in Geneva.

    Add new data -> backfill -> only new rows processed.
    No custom code needed.
    """
    conn = geneva.connect("s3://my-bucket/geneva-workspace")
    table = conn.open_table("images")

    # Add new images
    new_images = load_new_images()  # Your data loading
    table.add(new_images)

    # Backfill only processes rows where embedding IS NULL
    table.backfill("embedding", batch_size=100)
    table.backfill("caption", batch_size=50)
```

---

## Side-by-Side Comparison

### Feature Comparison

| Feature | Modal | Geneva |
|---------|-------|--------|
| Serverless GPUs | Core strength | Bring your cluster |
| Class-based UDFs | @modal.enter() | setup() |
| Dependency packaging | Image builder | Per-UDF automatic |
| Cold start optimization | Container caching | Worker pools |
| Pay-per-second billing | Yes | Cluster-based |
| **Data management** | Not included | Tables, versions |
| **Checkpointing** | Build yourself | Automatic |
| **Incremental processing** | Build yourself | Automatic |
| **Resume from failure** | Build yourself | Automatic |
| **Schema management** | Build yourself | From type hints |
| **Progress visibility** | Limited | Real-time commits |

### Code Comparison

| Component | Modal | Geneva |
|-----------|-------|--------|
| UDF definitions | ~55 lines | ~45 lines |
| Basic pipeline | ~25 lines | ~15 lines |
| Checkpoint system | ~50 lines (custom) | 0 (included) |
| Batch processor | ~70 lines (custom) | 0 (included) |
| Data loading/writing | ~15 lines | 0 (included) |
| **TOTAL (production)** | **~215 lines** | **~60 lines** |

---

## When to Use Each

### Modal is great for:
- Serverless GPU workloads (no cluster management)
- Burst compute needs (scale to 0 when idle)
- Event-driven processing (webhooks, APIs)
- Teams that want pay-per-second GPU billing
- One-off batch jobs without complex data management
- When you already have data management infrastructure

### Geneva is great for:
- Feature engineering with ongoing data updates
- Teams that need data versioning and time travel
- Production pipelines with fault tolerance requirements
- When incremental processing is critical
- Multi-team environments sharing feature tables
- When you want data + compute unified

---

## The Key Insight

**Modal asks**: "Where should this function run?"
**Geneva asks**: "How do I maintain this data over time?"

Modal is a compute platform - brilliant at running GPU functions at scale.
Geneva is a data platform - manages features as evolving datasets.

For a one-off embedding job, Modal is simpler.
For an embedding feature you'll maintain forever, Geneva handles the lifecycle.

**Think of it this way:**
- Modal: Run a script on powerful hardware
- Geneva: Manage a feature column in your data warehouse

**They can even work together:**
- Use Modal for ad-hoc experimentation
- Move to Geneva when the feature becomes production infrastructure

---

## Visual Architecture Comparison

### Modal Architecture (you build the boxes in gray)

**Your Code** (you build these):

| Data Loading | Checkpoint Store | Incremental Processor | Progress Tracking |
|--------------|------------------|----------------------|-------------------|
| (custom) | (custom) | (custom) | (custom) |

↓

**Modal**

| Serverless GPUs | Container Images | GPU Scheduling | .map() Parallel |
|-----------------|------------------|----------------|-----------------|

↓

**Modal Cloud** *(managed infrastructure)*

### Geneva Architecture (everything included)

**Your Code**

| UDFs (ML logic) | Pipeline (backfill) |
|-----------------|---------------------|

↓

**Geneva**

| Data Management | Checkpoint Store | Incremental Processor | Progress Tracking | Version Control |
|-----------------|------------------|----------------------|-------------------|-----------------|

| UDF System | GPU Scheduling | Distributed Execution | Dependency Isolation |
|------------|----------------|----------------------|---------------------|

↓

**Ray Cluster + Lance Storage**

### Complementary Usage

For teams using both:

| Stage | Focus | Tool |
|-------|-------|------|
| **Experimentation** | "Try this model" | Modal |
| **Prototyping** | "Works! Scale it" | Modal/Geneva |
| **Production** | "Maintain forever" | Geneva |
