# Cosmos-Xenna vs Geneva: Distributed AI Pipeline Platforms

## Overview

Cosmos-Xenna and Geneva are both Python libraries built on Ray for distributed AI workloads. However, they solve fundamentally different problems:

- **Cosmos-Xenna** = Distributed inference pipeline orchestrator
- **Geneva** = Feature engineering platform with lifecycle management

Both use Ray, both handle GPU scheduling, both support multi-stage pipelines. The key difference is **what happens to your data after the pipeline runs**.

---

## Quick Comparison

| Aspect | Cosmos-Xenna | Geneva |
|--------|--------------|--------|
| **Philosophy** | "Run multi-stage inference at scale" | "Maintain features over time" |
| **Primary use case** | Video/image processing pipelines | Production feature engineering |
| **Execution model** | Streaming stages with autoscaling | Ray Data with fragment checkpointing |
| **Data persistence** | You manage output storage | Built-in Lance tables with versioning |
| **Checkpointing** | None (stateless stages) | Fragment + batch level |
| **Incremental processing** | Not included (rerun pipeline) | Automatic (only new/changed rows) |
| **GPU scheduling** | Autoscaled worker pools | Explicit `num_gpus=1` per UDF |
| **SPMD support** | Native (tensor parallelism) | Not built-in |
| **Dependency mgmt** | Ray runtime_env | Per-UDF pyproject.toml isolation |
| **License** | Apache 2.0 | Proprietary |
| **Maintainer** | NVIDIA (Cosmos team) | LanceDB |

---

## Category Distinction

This comparison crosses categories:

| Category | Systems | Geneva's position |
|----------|---------|-------------------|
| **Inference Orchestration** | Cosmos-Xenna, Ray Serve, vLLM | Different problem space |
| **Feature Engineering** | Geneva, Pixeltable | Geneva's primary category |
| **Distributed Processing** | Ray Data, Daft, Spark | Lower-level building blocks |

**Cosmos-Xenna is NOT a direct competitor to Geneva.** They serve different stages of the ML lifecycle:

```
Training Data → [GENEVA] → Features → Model Training → [COSMOS-XENNA] → Inference Results
```

However, there's overlap in **batch inference for feature computation**—you could use either to run CLIP embeddings across a dataset.

---

## Architecture Comparison

### Cosmos-Xenna Architecture

```
Your Code
├── Stage classes (process_data methods)
├── PipelineSpec (stage ordering, worker counts)
└── run_pipeline() call
        │
        ▼
┌─────────────────────────────────────────┐
│            Cosmos-Xenna                  │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│  │Orchestrat│ │ Autoscale│ │   SPMD   │ │
│  │  Loop    │ │  Engine  │ │Coordinat.│ │
│  └──────────┘ └──────────┘ └──────────┘ │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│  │Backpress.│ │  P2P     │ │  Actor   │ │
│  │ Control  │ │Downloads │ │  Pools   │ │
│  └──────────┘ └──────────┘ └──────────┘ │
└─────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────┐
│              Ray Cluster                 │
│  Actors per stage, GPU bin-packing       │
│  Your responsibility: output storage     │
└─────────────────────────────────────────┘
```

### Geneva Architecture

```
Your Code
├── UDF classes (@geneva.udf decorator)
├── Table operations (add_columns, backfill)
└── Materialized views
        │
        ▼
┌─────────────────────────────────────────┐
│                Geneva                    │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│  │ Fragment │ │  Batch   │ │Dependency│ │
│  │Checkpoint│ │Checkpoint│ │ Isolation│ │
│  └──────────┘ └──────────┘ └──────────┘ │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│  │   UDF    │ │   GPU    │ │  Table   │ │
│  │ Versions │ │Scheduling│ │ Versions │ │
│  └──────────┘ └──────────┘ └──────────┘ │
└─────────────────────────────────────────┘
        │
        ▼
┌─────────────────────────────────────────┐
│         Ray Data + Lance Storage         │
│  Distributed execution on Ray clusters   │
│  Columnar, versioned Lance format        │
└─────────────────────────────────────────┘
```

---

## Code Comparison: Image Captioning Pipeline

### Cosmos-Xenna Approach (~80 lines)

```python
import cosmos_xenna.pipelines.v1 as pipelines_v1
from dataclasses import dataclass


@dataclass
class VideoSample:
    url: str
    buffer: bytes | None = None
    caption: str | None = None


class DownloadStage(pipelines_v1.Stage):
    """Download video/images from S3."""

    @property
    def stage_batch_size(self) -> int:
        return 10

    @property
    def required_resources(self) -> pipelines_v1.Resources:
        return pipelines_v1.Resources(gpus=0, cpus=1.0)

    def setup(self, worker_metadata: pipelines_v1.WorkerMetadata) -> None:
        import boto3
        self._s3 = boto3.client("s3")

    def process_data(self, samples: list[VideoSample]) -> list[VideoSample]:
        for sample in samples:
            # Parse S3 URL and download
            bucket, key = parse_s3_url(sample.url)
            obj = self._s3.get_object(Bucket=bucket, Key=key)
            sample.buffer = obj["Body"].read()
        return samples


class CaptionStage(pipelines_v1.Stage):
    """Generate captions using VLM."""

    @property
    def stage_batch_size(self) -> int:
        return 5  # GPU memory constraint

    @property
    def required_resources(self) -> pipelines_v1.Resources:
        return pipelines_v1.Resources(gpus=1.0, cpus=1.0)

    def setup(self, worker_metadata: pipelines_v1.WorkerMetadata) -> None:
        from transformers import BlipProcessor, BlipForConditionalGeneration
        self._processor = BlipProcessor.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        )
        self._model = BlipForConditionalGeneration.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        ).cuda()

    def process_data(self, samples: list[VideoSample]) -> list[VideoSample]:
        from PIL import Image
        import io
        import torch

        for sample in samples:
            img = Image.open(io.BytesIO(sample.buffer)).convert("RGB")
            inputs = self._processor(img, return_tensors="pt").to("cuda")
            with torch.no_grad():
                output = self._model.generate(**inputs, max_new_tokens=50)
            sample.caption = self._processor.decode(output[0], skip_special_tokens=True)
        return samples


class WriterStage(pipelines_v1.Stage):
    """Write results to database/storage."""

    @property
    def stage_batch_size(self) -> int:
        return 50

    @property
    def required_resources(self) -> pipelines_v1.Resources:
        return pipelines_v1.Resources(gpus=0, cpus=1.0)

    def setup(self, worker_metadata: pipelines_v1.WorkerMetadata) -> None:
        self._db = connect_to_database()

    def process_data(self, samples: list[VideoSample]) -> list[VideoSample]:
        for sample in samples:
            self._db.insert(url=sample.url, caption=sample.caption)
        return samples


def run_xenna_pipeline(input_urls: list[str]):
    """Run the Cosmos-Xenna pipeline."""
    samples = [VideoSample(url=url) for url in input_urls]

    pipeline_spec = pipelines_v1.PipelineSpec(
        input_data=samples,
        stages=[
            pipelines_v1.StageSpec(DownloadStage(), num_workers_per_node=15),
            CaptionStage(),  # Xenna autoscales GPU workers
            pipelines_v1.StageSpec(WriterStage(), num_workers_per_node=15),
        ],
    )

    pipelines_v1.run_pipeline(pipeline_spec)
```

**What you build additionally:**
- Output storage management
- Incremental processing logic (which URLs are new?)
- Resume after failure (restart from scratch or build checkpointing)
- Deduplication (did we already process this URL?)

### Geneva Approach (~60 lines)

```python
import geneva


@geneva.udf(num_gpus=1, version="0.1")
class GenerateCaption:
    """BLIP captioning UDF for Geneva."""

    def setup(self):
        from transformers import BlipProcessor, BlipForConditionalGeneration
        self.processor = BlipProcessor.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        )
        self.model = BlipForConditionalGeneration.from_pretrained(
            "Salesforce/blip-image-captioning-base"
        ).cuda()

    def __call__(self, image: bytes) -> str:
        from PIL import Image
        import torch
        import io

        img = Image.open(io.BytesIO(image)).convert("RGB")
        inputs = self.processor(img, return_tensors="pt").to("cuda")
        with torch.no_grad():
            output = self.model.generate(**inputs, max_new_tokens=50)
        return self.processor.decode(output[0], skip_special_tokens=True)


def run_geneva_pipeline():
    """Geneva pipeline with automatic checkpointing and incremental processing."""
    conn = geneva.connect("s3://my-bucket/workspace")
    table = conn.open_table("images")

    # Add caption column (UDF definition)
    table.add_columns({"caption": GenerateCaption})

    # Run with fragment + batch checkpointing
    table.backfill("caption", batch_size=50, commit_granularity=10)

    print(f"Processed {table.count_rows()} images")
    print(f"Table version: {table.version}")


def run_geneva_incremental():
    """Add new data - only new rows get processed."""
    conn = geneva.connect("s3://my-bucket/workspace")
    table = conn.open_table("images")

    # Add new images
    new_images = load_new_images_from_somewhere()
    table.add(new_images)

    # Only processes rows where caption IS NULL
    table.backfill("caption")
```

**What Geneva handles automatically:**
- Output storage (Lance tables with versioning)
- Incremental processing (only new/changed rows)
- Resume after failure (exact batch within fragment)
- Deduplication (checkpoint tracking)
- UDF version tracking (recompute if model changes)

---

## Key Differences Deep Dive

### 1. Data Model

| Aspect | Cosmos-Xenna | Geneva |
|--------|--------------|--------|
| **Input** | List of dataclass objects | Lance table rows |
| **Output** | You define (DB, S3, etc.) | Same Lance table (new columns) |
| **Persistence** | Not included | Built into platform |
| **Versioning** | Not included | Automatic (Lance format) |
| **Schema** | Dataclass fields | Type hints → Arrow schema |

**Cosmos-Xenna's bet:** You already have a data platform. Xenna runs the compute.

**Geneva's bet:** Feature engineering needs an integrated data + compute platform.

### 2. Checkpoint & Resume

| Aspect | Cosmos-Xenna | Geneva |
|--------|--------------|--------|
| **Checkpoint storage** | None built-in | Fragment + batch metadata |
| **Failure recovery** | Restart pipeline | Resume from exact batch |
| **Incremental runs** | Build yourself | Automatic |
| **Duplicate prevention** | Build yourself | Guaranteed |

**Scenario: Pipeline fails at 500K of 1M items**

| System | What happens |
|--------|--------------|
| **Cosmos-Xenna** | Restart from item 0. Build custom checkpointing or accept reprocessing. |
| **Geneva** | Resume from fragment 50, batch 15. No reprocessing of completed work. |

### 3. GPU Scheduling

| Aspect | Cosmos-Xenna | Geneva |
|--------|--------------|--------|
| **Allocation** | Autoscaled pools per stage | Explicit `num_gpus=1` per UDF |
| **Worker scaling** | Dynamic based on throughput | Static (Ray Data handles) |
| **SPMD (tensor parallel)** | Native support | Not built-in |
| **Backpressure** | Built-in queue limits | Ray Data streaming |

**Cosmos-Xenna advantage:** Autoscaling and SPMD for very large models (200B+ params).

**Geneva advantage:** Simpler mental model, explicit control.

### 4. Multi-Stage vs Single-Column

| Aspect | Cosmos-Xenna | Geneva |
|--------|--------------|--------|
| **Pipeline model** | Multi-stage DAG (A → B → C) | Per-column UDFs |
| **Stage dependencies** | Explicit ordering | Implicit (source columns) |
| **Materialization** | Between stages (optional) | Per-column commits |

**Cosmos-Xenna** is optimized for: Download → Decode → Model A → Model B → Write

**Geneva** is optimized for: Table with columns A, B, C each computed independently

### 5. Dependency Management

| Aspect | Cosmos-Xenna | Geneva |
|--------|--------------|--------|
| **Approach** | Ray runtime_env (cluster-wide) | Per-UDF pyproject.toml |
| **Conflicts** | Your problem | Isolated environments |
| **Model downloads** | P2P artifact distribution | Standard model loading |

**Cosmos-Xenna advantage:** P2P downloading reduces S3 bandwidth for large models.

**Geneva advantage:** Per-UDF isolation prevents dependency hell.

---

## Feature Comparison Table

| Feature | Cosmos-Xenna | Geneva |
|---------|--------------|--------|
| Distributed execution | Ray actors | Ray Data |
| GPU scheduling | Autoscaled pools | Explicit per-UDF |
| SPMD/tensor parallel | Native | Not built-in |
| Streaming execution | Stage-to-stage | Ray Data streaming |
| Checkpointing | Not included | Fragment + batch |
| Incremental processing | Not included | Automatic |
| Resume from failure | Restart | Exact batch |
| Data versioning | Not included | Lance built-in |
| Output storage | You build | Lance tables |
| P2P downloads | Built-in | Not included |
| Backpressure control | Built-in | Ray Data |
| Serving mode | Built-in | Not included |
| Dependency isolation | runtime_env | Per-UDF |
| Monitoring | Detailed reports | Standard Ray |
| Large model support | Excellent (SPMD) | Standard GPU |
| **Lines for basic pipeline** | ~80 | ~60 |
| **Lines for production ops** | +100 (custom) | +0 (included) |

---

## When to Use Each

### Use Cosmos-Xenna When:

- **Running inference pipelines** - Multi-stage video/image processing
- **Very large models** - SPMD for 200B+ parameter models
- **Throughput optimization** - Need fine-grained autoscaling
- **Existing data platform** - Output goes to your own storage
- **One-shot batch jobs** - Process dataset, write results, done
- **Real-time serving** - Need source/sink queues for live inference

### Use Geneva When:

- **Building feature columns** - Embeddings, captions, classifications as table columns
- **Ongoing maintenance** - Features need incremental updates as data grows
- **Fault tolerance critical** - Can't afford to restart from scratch
- **Data versioning needed** - Time travel, audit trails
- **Multi-team environment** - Shared feature tables
- **Production lifecycle** - Features live beyond the initial pipeline run

### The Honest Assessment:

| Scenario | Better Choice |
|----------|---------------|
| "Process 10M videos once for training" | Cosmos-Xenna |
| "Maintain embeddings for growing dataset" | Geneva |
| "Run CLIP + BLIP + LLM in sequence" | Cosmos-Xenna |
| "Add embedding column to existing table" | Geneva |
| "Serve model inference with autoscaling" | Cosmos-Xenna |
| "Track which rows have been processed" | Geneva |
| "Need tensor parallelism for huge model" | Cosmos-Xenna |
| "Resume exactly where job failed" | Geneva |

---

## Can They Work Together?

Yes. A reasonable architecture:

```
┌─────────────────────────────────────────────────────────────┐
│                    Data Lifecycle                            │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  Raw Data  ──→  [GENEVA]  ──→  Feature Tables               │
│  (S3/GCS)       (compute       (Lance format,               │
│                  embeddings,    versioned,                   │
│                  metadata)      checkpointed)                │
│                                                              │
│                      │                                       │
│                      ▼                                       │
│              Training Pipeline                               │
│              (use features)                                  │
│                      │                                       │
│                      ▼                                       │
│              Trained Model                                   │
│                      │                                       │
│                      ▼                                       │
│  Input  ──→  [COSMOS-XENNA]  ──→  Inference Results         │
│  Stream      (multi-stage         (DB, downstream           │
│              inference,            systems)                  │
│              autoscaled)                                     │
│                                                              │
└─────────────────────────────────────────────────────────────┘
```

**Geneva** for feature engineering (data at rest, maintained over time).

**Cosmos-Xenna** for inference pipelines (data in motion, processed once).

---

## Anticipated Cosmos-Xenna Advocate Arguments

### "Xenna's autoscaling is more sophisticated"

**Their argument:** "Xenna dynamically adjusts worker counts based on measured throughput. Geneva uses static worker allocation."

**Counter:**
- True. Cosmos-Xenna has excellent autoscaling for throughput optimization.
- Geneva uses Ray Data's scheduling, which is less dynamic but well-understood.
- For feature engineering, consistent throughput matters less than correctness and resumability.
- Trade-off: Autoscaling complexity vs. simpler mental model.

### "We don't need checkpointing for inference"

**Their argument:** "Inference pipelines are stateless. If it fails, rerun it."

**Counter:**
- Valid for one-shot inference jobs.
- Invalid for feature engineering where:
  - New data arrives daily
  - Jobs take hours/days
  - Reprocessing costs real money
- Different use cases, different requirements.

### "SPMD support is essential for modern models"

**Their argument:** "200B parameter models need tensor parallelism. Geneva can't do this."

**Counter:**
- True. Cosmos-Xenna has superior large model support.
- Most feature engineering uses smaller models (CLIP, BLIP, etc.) that fit on single GPU.
- If you need SPMD, Cosmos-Xenna (or vLLM directly) is the right choice.
- Geneva could add SPMD support, but it's not the primary use case.

### "P2P downloads save massive bandwidth costs"

**Their argument:** "Each model download once cluster-wide. Huge savings at scale."

**Counter:**
- Excellent feature for large model deployments.
- Less relevant for feature engineering where models are typically cached per-worker.
- Geneva could adopt P2P downloading, but standard Ray runtime_env caching works for most UDFs.

---

## Summary

| Dimension | Cosmos-Xenna | Geneva |
|-----------|--------------|--------|
| **Best for** | Inference pipelines | Feature engineering |
| **Data model** | Streaming stages | Table columns |
| **Output** | You manage | Built-in (Lance) |
| **Checkpointing** | Not included | Fragment + batch |
| **Incremental** | Not included | Automatic |
| **GPU scheduling** | Autoscaled | Explicit |
| **Large models** | SPMD native | Standard |
| **Production ops** | You build | Included |

**Bottom line:** Cosmos-Xenna and Geneva solve different problems in the ML lifecycle. Cosmos-Xenna excels at running multi-stage inference pipelines with sophisticated autoscaling and large model support. Geneva excels at maintaining feature columns over time with checkpointing, incremental processing, and data versioning. They're complementary tools, not competitors.

---

## Sources

- [Cosmos-Xenna GitHub](https://github.com/nvidia-cosmos/cosmos-xenna)
- [Ray Data Documentation](https://docs.ray.io/en/latest/data/data.html)
- [NVIDIA Cosmos Project](https://www.nvidia.com/en-us/ai/)
