# CZ Benchmarks

### What is cz-benchmarks?
cz-benchmarks is a package for standardized evaluation and comparison of machine learning models for biological applications (first, in the single-cell transcriptomics domain, with future plans to expand to additional domains). The package provides a toolkit for running containerized models, executing biologically-relevant tasks, and computing performance metrics. We see this tool as a step towards ensuring that large-scale ML Models can be harnessed to deliver genuine biological insights -- by building trust, accelerating development, and bridging the gap between ML and biology communities.

### Why benchmarking? Why now?
Last year, CZI hosted a workshop focused on benchmarking and evaluation of ML Models in biology, and the [insights gained](https://virtualcellmodels.cziscience.com/micro-pub/benchmarking-workshop) have reinforced our commitment to supporting the development of a robust benchmarking infrastructure, which we see as critical to achieving our Virtual Cell vision.

### 💬 Community Feedback & Contributions
We're working to get the alpha version of cz-benchmarks stable to build with the community. In the meantime, for issues you may identify, feel free to open an issue on GitHub or reach out to us at [virtualcellmodels@chanzuckerberg.com](mailto:virtualcellmodels@chanzuckerberg.com).

## Getting Started

To get started with `cz-benchmarks`, refer to the [Quick Start Guide](https://chanzuckerberg.github.io/cz-benchmarks/quick_start.html).


### 📚 Additional Resources

- [How To Guides](https://chanzuckerberg.github.io/cz-benchmarks/how_to_guides/index.html)
    - [Add a Custom Dataset](https://chanzuckerberg.github.io/cz-benchmarks/how_to_guides/add_custom_dataset.html)
- [Developer Guides](https://chanzuckerberg.github.io/cz-benchmarks/developer_guides/index.html)
- [API Reference](https://chanzuckerberg.github.io/cz-benchmarks/api_reference.html)
- [Assets](https://chanzuckerberg.github.io/cz-benchmarks/assets.html)

📖 **Documentation**: The full documentation is available at [cz-benchmarks](https://chanzuckerberg.github.io/cz-benchmarks/)