Metadata-Version: 2.1
Name: pplbench
Version: 0.0.2
Summary: Evaluation framework for probabilistic programming languages
Home-page: UNKNOWN
Author: Facebook, Inc.
License: MIT
Project-URL: Documentation, https://pplbench.org
Project-URL: Source, https://github.com/facebookresearch/pplbench
Keywords: Probabilistic Programming Language,Bayesian Inference,Statistical Modeling,MCMC,PyTorch
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Scientific/Engineering
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: jsonargparse (>=2.32.2)
Requires-Dist: jsonschema (>=3.2.0)
Requires-Dist: numpy (>=1.18.5)
Requires-Dist: scipy (>=1.5.0)
Requires-Dist: pandas (>=1.0.1)
Requires-Dist: matplotlib (>=3.1.3)
Requires-Dist: xarray (>=0.16.0)
Requires-Dist: arviz (>=0.9.0)
Provides-Extra: dev
Requires-Dist: pystan (>=2.19.1.1) ; extra == 'dev'
Requires-Dist: pymc3 (>=3.9.0) ; extra == 'dev'
Requires-Dist: pyro-ppl (>=0.4.1) ; extra == 'dev'
Requires-Dist: numpyro (>=0.3.0) ; extra == 'dev'
Requires-Dist: black (==20.8b1) ; extra == 'dev'
Requires-Dist: isort ; extra == 'dev'
Requires-Dist: flake8 ; extra == 'dev'
Requires-Dist: mypy ; extra == 'dev'
Provides-Extra: ppls
Requires-Dist: pystan (>=2.19.1.1) ; extra == 'ppls'
Requires-Dist: pymc3 (>=3.9.0) ; extra == 'ppls'
Requires-Dist: pyro-ppl (>=0.4.1) ; extra == 'ppls'
Requires-Dist: numpyro (>=0.3.0) ; extra == 'ppls'

[![](docs/assets/logo.png)]((https://facebookresearch.github.io/pplbench/))



[![Build status](https://github.com/facebookresearch/pplbench/workflows/build/badge.svg?branch=master)](https://github.com/facebookresearch/pplbench/actions?query=workflow%3Abuild)
[![PyPI version](https://badge.fury.io/py/pplbench.svg)](https://pypi.org/project/pplbench/)

# Getting Started with PPL Bench

* [What is PPL Bench?](#what-is-ppl-bench)
* [Installing](#installation)
* [Getting Started](#getting-started)



## What is PPL Bench?

PPL Bench is a new benchmark framework for evaluating probabilistic programming languages (PPLs).

## Installation

1. Enter a virtual (or conda) environment
2. Install PPL Bench core via pip:

```
pip install pplbench
```

3. Install PPLs that you wish to benchmark. For PPL-specific instructions, see [Installing PPLs](docs/working_with_ppls.md).
You could also run the following command to install all PPLs that are currently supported by PPL Bench (except for Jags):

```
pip install pplbench[ppls]
```

Alternatively, you could also install PPL Bench from source. Please refer to [Installing PPLs](docs/working_with_ppls.md)
for instructions.

## Getting Started

Let's dive right in with a benchmark run of Bayesian Logistic Regression. To run this, you'll need to install
PyStan (if you haven't already):

```
pip install pystan
```

Then, run PPL Bench with example config:

```
pplbench examples/example.json
```

This will create a benchmark run with two trials of Stan on the Bayesian Logistic Regression model. The results of the run are saved in the `outputs/` directory.

This is what the Predictive Log Likelihood (PLL) plot should look like:
![PLL plot of example run](website/static/img/example_pystan_pll.svg)
![PLL half plot of example run](website/static/img/example_pystan_pll_half.svg)


Please see the [examples/example.json](examples/example.json) file to understand the schema for specifying benchmark runs. The schema is documented in [pplbench/main.py](pplbench/main.py) and can be printed by running the help command:

```
pplbench -h
```

A number of models is available in the `pplbench/models` directory and the PPL implementations are available in the `pplbench/ppls` directory.

Please feel free to submit pull requests to modify an existing PPL implementation or to add a new PPL or model.


## Join the PPL Bench community

 For more information about PPL Bench, refer to

1. Blog post: [link](https://ai.facebook.com/blog/ppl-bench-creating-a-standard-for-benchmarking-probabilistic-programming-languages)
2. Paper: [link](https://arxiv.org/abs/2010.08886)
3. Website: [link](https://facebookresearch.github.io/pplbench/)

See the [CONTRIBUTING.md](CONTRIBUTING.md) file for how to help out.

## License

This source code is licensed under the MIT license found in the
LICENSE file in the root directory of this source tree.


