ramalama-bench 1

NAME

ramalama-bench - benchmark specified AI Model

SYNOPSIS

ramalama bench [options] model [arg ...]

MODEL TRANSPORTS
Transports | Prefix | Web Site
URL based | https://, http://, file:// | https://web.site/ai.model, file://tmp/ai.model
HuggingFace | huggingface://, hf://, hf.co/ | huggingface.co
Ollama | ollama:// | ollama.com
OCI Container Registries | oci:// | opencontainers.org
 |  | Examples: quay.io, Docker Hub,Artifactory

RamaLama defaults to the Ollama registry transport. This default can be overridden in the ramalama.conf file or via the RAMALAMA_TRANSPORTS environment. export RAMALAMA_TRANSPORT=huggingface Changes RamaLama to use huggingface transport.

Modify individual model transports by specifying the huggingface://, oci://, ollama://, https://, http://, file:// prefix to the model.

URL support means if a model is on a web site or even on your local system, you can run it directly.

OPTIONS

  --help, -h

show this help message and exit

  --network=none

set the network mode for the container

  --ngl

number of gpu layers, 0 means CPU inferencing, 999 means use max layers (default: -1) The default -1, means use whatever is automatically deemed appropriate (0 or 999)

DESCRIPTION

Benchmark specified AI Model.

EXAMPLES

    ramalama bench granite-moe3

SEE ALSO

ramalama(1)

HISTORY

Jan 2025, Originally compiled by Eric Curtin ⟨ecurtin@redhat.com⟩