Metadata-Version: 2.4
Name: flashinfer-bench
Version: 0.1.2
Summary: FlashInfer-Bench - AI for AI Infrastructure for Accelerating AI Deployment
Author: FlashInfer team
License: Apache License 2.0
Project-URL: Homepage, https://bench.flashinfer.ai
Project-URL: Documentation, https://bench.flashinfer.ai/docs
Project-URL: Repository, https://github.com/flashinfer-ai/flashinfer-bench
Project-URL: Issues, https://github.com/flashinfer-ai/flashinfer-bench/issues
Keywords: deep-learning,inference,benchmark,flashinfer,gpu
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <4.0,>=3.10
Description-Content-Type: text/markdown
License-File: LICENCE
License-File: NOTICE
Requires-Dist: torch>=2.8.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: safetensors>=0.5.0
Requires-Dist: flashinfer-python>=0.3.1
Requires-Dist: apache-tvm-ffi>=0.1.2
Requires-Dist: docstring-parser>=0.16
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: black>=22.0.0; extra == "dev"
Requires-Dist: isort>=5.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: setuptools; extra == "dev"
Provides-Extra: cuda12
Requires-Dist: nvidia-cublas-cu12>=12.3; extra == "cuda12"
Requires-Dist: nvidia-cudnn-cu12>=9.0; extra == "cuda12"
Dynamic: license-file

<div align="center" id="top">

<picture>
  <source media="(prefers-color-scheme: light)" srcset="docs/logo/fib-white-bg.png">
  <source media="(prefers-color-scheme: dark)" srcset="docs/logo/fib-black-bg.png">
  <img src="docs/logo/fib-white-bg.png" alt="FlashInfer-Bench logo" width="400">
</picture>


[![Documentation](https://img.shields.io/badge/docs-latest-green)](https://bench.flashinfer.ai/docs/)
[![License](https://img.shields.io/badge/license-apache_2-blue)](https://github.com/flashinfer-ai/flashinfer-bench/blob/main/LICENCE)
[![PyPI](https://img.shields.io/pypi/v/flashinfer-bench)](https://pypi.org/project/flashinfer-bench/)

**Building the Virtuous Cycle for AI-driven LLM Systems**

[Get Started](#get-started) | [Documentation](https://bench.flashinfer.ai/docs/) | [Blogpost](https://flashinfer.ai/2025/10/21/flashinfer-bench.html)
| [Slack (#flashinfer-bench)](https://join.slack.com/t/flashinfer/shared_invite/zt-379wct3hc-D5jR~1ZKQcU00WHsXhgvtA) </div>

**FlashInfer-Bench** is a benchmark suite and production workflow designed to build a virtuous cycle of self-improving AI systems.

It is part of a broader initiative to build the *virtuous cycle of AI improving AI systems* — enabling AI agents and engineers to collaboratively optimize the very kernels that power large language models.

## Installation

Install FlashInfer-Bench with pip:

```bash
pip install flashinfer-bench
```

Import FlashInfer-Bench:

```python
import flashinfer_bench as fib

print(fib.__version__)
```

## Get Started

This [guide](https://bench.flashinfer.ai/docs/start/quickstart) shows you how to use FlashInfer-Bench python module with the FlashInfer-Trace dataset.

## FlashInfer Trace Dataset

We provide an official dataset called **FlashInfer-Trace** with kernels and workloads in real-world AI system deployment environments. FlashInfer-Bench can use this dataset to measure and compare the performance of kernels. It follows the [FlashInfer Trace Schema](https://bench.flashinfer.ai/docs/flashinfer-trace).

The official dataset is on HuggingFace: https://huggingface.co/datasets/flashinfer-ai/flashinfer-trace

## Collaborators

Our collaborators include:

<div align="center">

[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/nvidia.svg" height=50/>](https://github.com/NVIDIA/TensorRT-LLM)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/gpu_mode.png" height=50/>](https://github.com/gpu-mode)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/sglang.png" height=50/>](https://github.com/sgl-project/sglang)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/vllm.png" height=50/>](https://github.com/vllm-project/vllm)
&emsp;
[<img src="https://raw.githubusercontent.com/mlc-ai/XGrammar-web-assets/refs/heads/main/repo/bosch.svg" height=50/>](https://www.bosch.com/)

</div>
