Evalytic Image Models Leaderboard

Run on fal.ai · 2026-03-12 · 5 models · 3 prompts · Judge: Consensus (gemini-2.5-flash, gpt-5.2)

| Family: |

VLM Dimensions

= 100%

VLM Total 80%

Visual Quality 33%

Prompt Adherence 33%

Text Rendering ⓘ 33%

Deterministic Metrics

= 100%

Deterministic Total 20%

CLIP Score 40%

Sharpness 30%

Aesthetic 30%

Speed ⓘ 0%

Image model leaderboard ranked by overall score
#	Model	Family	$/img	Overall	VQ	PA	TR	CLIP	Sharp	Aesthetic	Score/$	imgsys	Time	Seed	fal.ai
1	Gemini 2.5 Flash Image	Google	$0.040	4.57	5.0	5.0	4.3	0.314	0.864	0.531	115	1156	39.1s	42	Try →
2	Z-Image Turbo	Other	$0.005	4.17	4.0	4.7	4.0	0.346	0.901	0.501	834	1080	2.3s	42	Try →
3	Ideogram V3 Turbo	Ideogram	$0.030	4.12	4.0	4.7	4.0	0.300	0.940	0.529	137	1050	41.7s	42	Try →
4	FLUX.2 [dev]	Black Forest Labs	$0.012	4.08	4.0	4.7	3.7	0.351	0.870	0.523	340	1149	3.3s	42	Try →
5	FLUX.1 [schnell]	Black Forest Labs	$0.003	3.85	3.3	4.7	3.7	0.305	0.928	0.503	1283	950	1.2s	42	Try →

Prompt Browser (3 prompts × 5 models)

1 / 3

“A glowing neon sign reading EVALYTIC in a dark rainy alley, reflections on wet cobblestones, cinematic lighting”

flux-schnell

flux-2-dev

z-image-turbo

ideogram-v3

nano-banana

“A black cat sitting on a stack of old books in a cozy library, warm golden light from a desk lamp, dust particles in the air”

flux-schnell

flux-2-dev

z-image-turbo

ideogram-v3

nano-banana

“A vintage coffee shop storefront with hand-lettered chalkboard menu, morning sunlight, 35mm film photography style”

flux-schnell

flux-2-dev

z-image-turbo

ideogram-v3

nano-banana

Methodology

Benchmark Setup

5 models evaluated on fal.ai infrastructure
3 curated prompts from PartiPrompts, DrawBench, OneIG-Bench + custom. Each prompt contributes equally to final scores (simple average).
Judge: Consensus (gemini-2.5-flash, gpt-5.2)
Seed: 5 models with seed=42. A fixed seed ensures the same prompt always produces the same image, making results reproducible and comparisons fair.
Concurrency: 8 parallel requests
Pipeline: text2img
Date: 2026-03-12

Column Definitions

Overall — Weighted combination of VLM judge scores and deterministic metrics. Formula: 1 + 4 × (vlm_w × (vlm_avg−1)/4 + det_w × det_avg). Default: 80% VLM + 20% deterministic. Adjustable via the weight panel above.
VQ (Visual Quality) — VLM judge score (1–5). Evaluates overall image quality: sharpness, color fidelity, coherence, realism, and absence of visual artifacts.
PA (Prompt Adherence) — VLM judge score (1–5). How faithfully the generated image matches the text prompt — objects, attributes, spatial relationships, and style.
TR (Text Rendering) — VLM judge score (1–5). Accuracy of text rendered within the image (signs, labels, logos). Only scored on prompts that contain text elements (~12% of prompts). Shows “—” for models/prompts without text.
CLIP — Deterministic metric (0–1). CLIP ViT-L/14 cosine similarity between prompt text and generated image. Higher = better semantic alignment with prompt.
Sharp (Sharpness) — Deterministic metric (0–1). Variance of Laplacian applied to the image. Higher = sharper, more detailed image. Low values may indicate blur or softness.
Aesthetic — Deterministic metric (0–1). LAION aesthetic predictor — an MLP trained on human aesthetic ratings (AVA dataset) using CLIP embeddings. Higher = more aesthetically pleasing.
Score/$ — Value efficiency: Overall score divided by cost per image. Higher = better quality per dollar spent.
imgsys — ELO rating from imgsys.org community voting. Independent reference point for cross-validation.
Time — Average image generation time in seconds.
Seed — Fixed random seed for reproducibility. Models that support seed get deterministic output. Models without seed support are run 3× per prompt with median score taken.
$/img — Cost per image generation in USD on fal.ai.

Link copied!

Evalytic Image Models Leaderboard

VLM Dimensions

Deterministic Metrics

⚠ Weight Validation Error

Benchmark Setup

Column Definitions