# Evalytic Image Models Leaderboard

Powered by Evalytic
Run on fal.ai · 2026-03-12 · 5 models · 3 prompts · Judge: Consensus (gemini-2.5-flash, gpt-5.2)
| Family:
|

### VLM Dimensions

= 100%
VLM Total 80%
Visual Quality 33%
Prompt Adherence 33%
Text Rendering ⓘ 33%

### Deterministic Metrics

= 100%
Deterministic Total 20%
CLIP Score 40%
Sharpness 30%
Aesthetic 30%
Speed ⓘ 0%

### ⚠ Weight Validation Error

Image model leaderboard ranked by overall score
# | Model | Family | $/img | Overall | VQ | PA | TR | CLIP | Sharp | Aesthetic | Score/$ | imgsys | Time | Seed | fal.ai
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
1 | Gemini 2.5 Flash Image | Google | $0.040 | 4.57 | 5.0 | 5.0 | 4.3 | 0.314 | 0.864 | 0.531 | 115 | 1156 | 39.1s | 42 | Try →
2 | Z-Image Turbo | Other | $0.005 | 4.17 | 4.0 | 4.7 | 4.0 | 0.346 | 0.901 | 0.501 | 834 | 1080 | 2.3s | 42 | Try →
3 | Ideogram V3 Turbo | Ideogram | $0.030 | 4.12 | 4.0 | 4.7 | 4.0 | 0.300 | 0.940 | 0.529 | 137 | 1050 | 41.7s | 42 | Try →
4 | FLUX.2 [dev] | Black Forest Labs | $0.012 | 4.08 | 4.0 | 4.7 | 3.7 | 0.351 | 0.870 | 0.523 | 340 | 1149 | 3.3s | 42 | Try →
5 | FLUX.1 [schnell] | Black Forest Labs | $0.003 | 3.85 | 3.3 | 4.7 | 3.7 | 0.305 | 0.928 | 0.503 | 1283 | 950 | 1.2s | 42 | Try →

Prompt Browser (3 prompts × 5 models)
“A glowing neon sign reading EVALYTIC in a dark rainy alley, reflections on wet cobblestones, cinematic lighting”
flux-schnell
flux-2-dev
z-image-turbo
ideogram-v3
nano-banana
“A black cat sitting on a stack of old books in a cozy library, warm golden light from a desk lamp, dust particles in the air”
flux-schnell
flux-2-dev
z-image-turbo
ideogram-v3
nano-banana
“A vintage coffee shop storefront with hand-lettered chalkboard menu, morning sunlight, 35mm film photography style”
flux-schnell
flux-2-dev
z-image-turbo
ideogram-v3
nano-banana
Methodology

#### Benchmark Setup

- 5 models evaluated on fal.ai infrastructure
- 3 curated prompts from PartiPrompts, DrawBench, OneIG-Bench + custom. Each prompt contributes equally to final scores (simple average).
- Judge: Consensus (gemini-2.5-flash, gpt-5.2)
- Seed: 5 models with seed=42. A fixed seed ensures the same prompt always produces the same image, making results reproducible and comparisons fair.
- Concurrency: 8 parallel requests
- Pipeline: text2img
- Date: 2026-03-12

#### Column Definitions

- Overall — Weighted combination of VLM judge scores and deterministic metrics. Formula: `1 + 4 × (vlm_w × (vlm_avg−1)/4 + det_w × det_avg)`. Default: 80% VLM + 20% deterministic. Adjustable via the weight panel above.
- VQ (Visual Quality) — VLM judge score (1–5). Evaluates overall image quality: sharpness, color fidelity, coherence, realism, and absence of visual artifacts.
- PA (Prompt Adherence) — VLM judge score (1–5). How faithfully the generated image matches the text prompt — objects, attributes, spatial relationships, and style.
- TR (Text Rendering) — VLM judge score (1–5). Accuracy of text rendered within the image (signs, labels, logos). Only scored on prompts that contain text elements (~12% of prompts). Shows “—” for models/prompts without text.
- CLIP — Deterministic metric (0–1). CLIP ViT-L/14 cosine similarity between prompt text and generated image. Higher = better semantic alignment with prompt.
- Sharp (Sharpness) — Deterministic metric (0–1). Variance of Laplacian applied to the image. Higher = sharper, more detailed image. Low values may indicate blur or softness.
- Aesthetic — Deterministic metric (0–1). LAION aesthetic predictor — an MLP trained on human aesthetic ratings (AVA dataset) using CLIP embeddings. Higher = more aesthetically pleasing.
- Score/$ — Value efficiency: Overall score divided by cost per image. Higher = better quality per dollar spent.
- imgsys — ELO rating from imgsys.org community voting. Independent reference point for cross-validation.
- Time — Average image generation time in seconds.
- Seed — Fixed random seed for reproducibility. Models that support seed get deterministic output. Models without seed support are run 3× per prompt with median score taken.
- $/img — Cost per image generation in USD on fal.ai.

Link copied!