---
title: eugr-vllm
description: The gold standard for community-driven use of vLLM on DGX Spark.
---

The `eugr-vllm` runtime extends sparkrun's [`vllm-ray`](/runtimes/vllm/) variant with support for [eugr/spark-vllm-docker](https://github.com/eugr/spark-vllm-docker) container builds and mods. Eugr's repo is the gold standard for community-driven use of vLLM on the NVIDIA DGX Spark — frequently updated with patches and details to support the latest models. It provides a container build system to facilitate building the latest from the main branch of vLLM and a system of maintained mods to resolve issues related to running models on the Spark.

Since `eugr-vllm` extends `vllm-ray`, it inherits all Ray-based orchestration: multi-node clustering via Ray head/worker nodes, container distribution, InfiniBand detection, log following, and cluster stop all work identically to the `vllm-ray` runtime. The eugr-specific additions are:

- **Container builds**: When a recipe specifies `build_args`, sparkrun calls eugr's `build-and-copy.sh` to build the image locally before distribution.
- **Mods**: When a recipe specifies `mods`, sparkrun copies each mod directory into the container and runs its `run.sh` after container launch but before the serve command starts.

## When to use

Use `eugr-vllm` when you want:
- The community's most mature vLLM container pipeline
- Nightly (or sometimes 3x a day!) vLLM builds with the latest features
- Custom modifications (mods) to patch as needed for the latest models
- Local container builds with full control

For prebuilt images without mods or custom builds, use the standard `vllm` runtime instead (which resolves to [`vllm-distributed`](/runtimes/vllm/) by default).

## Example recipe

```yaml
model: my-org/custom-model
runtime: eugr-vllm
container: vllm-node

runtime_config:
  mods: [my-custom-mod]
  build_args: [--some-flag]

defaults:
  port: 8000
  host: 0.0.0.0
  tensor_parallel: 1
  gpu_memory_utilization: 0.8

command: |
  vllm serve {model} \
      --gpu-memory-utilization {gpu_memory_utilization} \
      -tp {tensor_parallel} \
      --host {host} --port {port}
```

## runtime_config

The `runtime_config` field passes eugr-specific configuration:

```yaml
runtime_config:
  mods: [my-custom-mod]
  build_args: [--some-flag]
```

- **`build_args`**: Arguments passed to eugr's `build-and-copy.sh` during the prepare phase. If present, sparkrun builds the container image locally before distributing it to hosts.
- **`mods`**: List of mod directory names (relative to the eugr repo root). Each mod must contain a `run.sh` script. Mods are applied to containers via `docker cp` and `docker exec` after launch but before the serve command.

Unknown top-level recipe keys are automatically swept into `runtime_config`, so you can also write:

```yaml
mods: [my-custom-mod]
build_args: [--some-flag]
```

## Recipe compatibility

The sparkrun recipe format is designed to be mostly compatible with eugr's format. sparkrun translates any variations automatically. Recipes with `recipe_version: "1"` or eugr-specific fields (`build_args`, `mods`) are auto-detected as `eugr-vllm`.
