---
title: Runtimes Overview
description: Supported inference runtimes in sparkrun.
---

sparkrun supports multiple inference runtimes through a plugin system. Each runtime knows how to launch a specific inference engine, configure networking, and manage containers.

## Runtime comparison

| Runtime | Engine | Multi-Node | Use Case |
|---|---|---|---|
| `vllm-distributed` | vLLM | Yes (native) | Default vLLM variant. Uses vLLM's built-in distributed backend. |
| `vllm-ray` | vLLM | Yes (Ray) | vLLM with Ray head/worker orchestration. |
| `sglang` | SGLang | Yes (native) | High-throughput inference, structured generation, experimental GGUF |
| `llama-cpp` | llama.cpp | Experimental | GGUF quantized models, lightweight deployment |
| `eugr-vllm` | vLLM (eugr) | Yes (Ray) | Gold standard community vLLM — nightly builds, mods, battle-tested. Extends `vllm-ray`. |

### Automatic runtime detection

Explicitly setting `runtime` in recipes is recommended, but when it is omitted sparkrun can infer the runtime from the `command` field prefix:

| Command prefix | Detected runtime |
|---|---|
| `vllm serve ...` | `vllm` |
| `sglang serve ...` | `sglang` |
| `python[3] -m sglang.launch_server ...` | `sglang` |
| `llama-server ...` | `llama-cpp` |

If the command doesn't match any known pattern (or there is no `command`), the recipe falls through to `vllm` behavior. An explicit `runtime` field always takes precedence over command-based detection.

See the [recipe format docs](/recipes/format/#automatic-runtime-detection) for the full detection rules.

### The `vllm` alias

You can write `runtime: vllm` in a recipe (or let it be auto-detected from a `vllm serve` command) — sparkrun treats it as a virtual alias and resolves it to a concrete runtime:

1. If the recipe has eugr signals (`build_args`, `mods`, or `recipe_version: "1"`) → **`eugr-vllm`**
2. If the recipe hints at Ray usage (`distributed_executor_backend: ray` in defaults, or `--distributed-executor-backend ray` in the command template) → **`vllm-ray`**
3. Otherwise → **`vllm-distributed`** (the default)

Both `vllm-distributed` and `vllm-ray` use the same container images and support the same vLLM flags — they differ only in how multi-node clustering is orchestrated.

## How runtimes work

**Recipes** declare which runtime to use via the `runtime` field. The runtime handles:

- Container launch configuration (volumes, networking, GPU access)
- InfiniBand/RDMA detection and NCCL environment variables
- Multi-node clustering (native distributed for `vllm-distributed` and `sglang`, Ray for `vllm-ray` and `eugr-vllm`)
- Model pre-sync and cache path resolution
- Command generation when no explicit `command` template is provided

## Choosing a runtime

- **vLLM** is the default and most widely supported. Write `runtime: vllm` and sparkrun picks the right variant automatically (see [the `vllm` alias](#the-vllm-alias) above).
  - **vllm-distributed** (default) uses vLLM's built-in distributed backend — each node runs the full serve command with node-rank arguments (`--nnodes`, `--node-rank`, `--master-addr`).
  - **vllm-ray** uses Ray head/worker orchestration — a Ray cluster is formed first, then the serve command runs on the head node.
- **SGLang** offers competitive performance, good structured generation support, and experimental GGUF quantized model serving.
- **llama.cpp** is ideal for GGUF quantized models or when you want a lightweight alternative.
- **eugr-vllm** extends `vllm-ray` with eugr's container build system and mod support — nightly builds, mods, and a battle-tested container pipeline.

## Maintained container images

Scitrera maintains several specialized container images for DGX Spark. For more information on how these images are built and maintained, see the [cuda-containers](https://github.com/scitrera/cuda-containers) repository.

| Image | Description | Docker Hub |
|---|---|---|
| `dgx-spark-vllm` | Optimized vLLM for DGX Spark | [Link](https://hub.docker.com/r/scitrera/dgx-spark-vllm) |
| `dgx-spark-sglang` | Optimized SGLang for DGX Spark | [Link](https://hub.docker.com/r/scitrera/dgx-spark-sglang) |
| `dgx-spark-llama-cpp` | Optimized llama.cpp for DGX Spark | [Link](https://hub.docker.com/r/scitrera/dgx-spark-llama-cpp) |
| `dgx-spark-pytorch-runtime` | PyTorch runtime with Spark-optimized dependencies | [Link](https://hub.docker.com/r/scitrera/dgx-spark-pytorch-runtime) |
| `dgx-spark-pytorch-dev` | PyTorch development image for DGX Spark | [Link](https://hub.docker.com/r/scitrera/dgx-spark-pytorch-dev) |

We recommend checking Docker Hub for the latest available tags for each image.

Note: These are not the only images that work with sparkrun; sparkrun is designed for general use. The above images are maintained by Scitrera to ensure compatibility and performance on DGX Spark, but you can use any container that meets the requirements of your chosen runtime.
The images above are intended to be a middle ground. The NVIDIA NGC containers are often a bit far behind in the interest of stability, while bleeding-edge community images can sometimes be a bit too volatile for production use.
The above images are regularly updated to keep up with the latest versions of vLLM, SGLang, and llama.cpp while ensuring stability on DGX Spark.
