Metadata-Version: 2.1
Name: optuna
Version: 2.4.0
Summary: A hyperparameter optimization framework
Home-page: https://optuna.org/
Author: Takuya Akiba
Author-email: akiba@preferred.jp
License: UNKNOWN
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: alembic
Requires-Dist: cliff
Requires-Dist: cmaes (>=0.6.0)
Requires-Dist: colorlog
Requires-Dist: joblib
Requires-Dist: numpy
Requires-Dist: packaging (>=20.0)
Requires-Dist: scipy (!=1.4.0)
Requires-Dist: sqlalchemy (>=1.1.0)
Requires-Dist: tqdm
Provides-Extra: checking
Requires-Dist: black ; extra == 'checking'
Requires-Dist: hacking ; extra == 'checking'
Requires-Dist: isort ; extra == 'checking'
Requires-Dist: mypy ; extra == 'checking'
Requires-Dist: blackdoc ; extra == 'checking'
Provides-Extra: codecov
Requires-Dist: codecov ; extra == 'codecov'
Requires-Dist: pytest-cov ; extra == 'codecov'
Provides-Extra: doctest
Requires-Dist: cma ; extra == 'doctest'
Requires-Dist: matplotlib (>=3.0.0) ; extra == 'doctest'
Requires-Dist: pandas ; extra == 'doctest'
Requires-Dist: plotly (>=4.0.0) ; extra == 'doctest'
Requires-Dist: scikit-learn (<0.23.0,>=0.19.0) ; extra == 'doctest'
Requires-Dist: scikit-optimize ; extra == 'doctest'
Requires-Dist: mlflow ; extra == 'doctest'
Provides-Extra: document
Requires-Dist: sphinx ; extra == 'document'
Requires-Dist: sphinx-rtd-theme ; extra == 'document'
Requires-Dist: sphinx-copybutton ; extra == 'document'
Requires-Dist: sphinx-gallery ; extra == 'document'
Requires-Dist: sphinx-plotly-directive ; extra == 'document'
Requires-Dist: pillow ; extra == 'document'
Requires-Dist: matplotlib ; extra == 'document'
Requires-Dist: scikit-learn ; extra == 'document'
Requires-Dist: plotly (>=4.0.0) ; extra == 'document'
Requires-Dist: pandas ; extra == 'document'
Requires-Dist: lightgbm ; extra == 'document'
Provides-Extra: example
Requires-Dist: catboost ; extra == 'example'
Requires-Dist: chainer ; extra == 'example'
Requires-Dist: lightgbm ; extra == 'example'
Requires-Dist: mlflow ; extra == 'example'
Requires-Dist: mpi4py ; extra == 'example'
Requires-Dist: mxnet ; extra == 'example'
Requires-Dist: nbval ; extra == 'example'
Requires-Dist: scikit-image ; extra == 'example'
Requires-Dist: scikit-learn (<0.23.0,>=0.19.0) ; extra == 'example'
Requires-Dist: xgboost ; extra == 'example'
Requires-Dist: keras ; extra == 'example'
Requires-Dist: tensorflow (>=2.0.0) ; extra == 'example'
Requires-Dist: tensorflow-datasets ; extra == 'example'
Requires-Dist: pytorch-ignite ; extra == 'example'
Requires-Dist: pytorch-lightning (>=1.0.2) ; extra == 'example'
Requires-Dist: thop ; extra == 'example'
Requires-Dist: skorch ; extra == 'example'
Requires-Dist: stable-baselines3 (>=0.7.0) ; extra == 'example'
Requires-Dist: catalyst ; extra == 'example'
Requires-Dist: torchaudio (==0.7.2) ; extra == 'example'
Requires-Dist: allennlp (==1.2.0) ; extra == 'example'
Requires-Dist: dask[dataframe] ; extra == 'example'
Requires-Dist: dask-ml ; extra == 'example'
Requires-Dist: fastai ; extra == 'example'
Requires-Dist: jax ; extra == 'example'
Requires-Dist: optax ; extra == 'example'
Requires-Dist: dm-haiku ; extra == 'example'
Requires-Dist: botorch ; (python_version > "3.6") and extra == 'example'
Requires-Dist: torch (==1.7.1+cpu) ; (sys_platform != "darwin") and extra == 'example'
Requires-Dist: torchvision (==0.8.2+cpu) ; (sys_platform != "darwin") and extra == 'example'
Requires-Dist: torch (==1.7.1) ; (sys_platform == "darwin") and extra == 'example'
Requires-Dist: torchvision (==0.8.2) ; (sys_platform == "darwin") and extra == 'example'
Provides-Extra: experimental
Requires-Dist: redis ; extra == 'experimental'
Provides-Extra: integration
Requires-Dist: chainer (>=5.0.0) ; extra == 'integration'
Requires-Dist: cma ; extra == 'integration'
Requires-Dist: lightgbm ; extra == 'integration'
Requires-Dist: mlflow ; extra == 'integration'
Requires-Dist: mpi4py ; extra == 'integration'
Requires-Dist: mxnet ; extra == 'integration'
Requires-Dist: pandas ; extra == 'integration'
Requires-Dist: scikit-learn (<0.23.0,>=0.19.0) ; extra == 'integration'
Requires-Dist: scikit-optimize ; extra == 'integration'
Requires-Dist: xgboost ; extra == 'integration'
Requires-Dist: keras ; extra == 'integration'
Requires-Dist: tensorflow ; extra == 'integration'
Requires-Dist: tensorflow-datasets ; extra == 'integration'
Requires-Dist: pytorch-ignite ; extra == 'integration'
Requires-Dist: pytorch-lightning (>=1.0.2) ; extra == 'integration'
Requires-Dist: skorch ; extra == 'integration'
Requires-Dist: catalyst ; extra == 'integration'
Requires-Dist: torchaudio (==0.7.2) ; extra == 'integration'
Requires-Dist: fastai ; extra == 'integration'
Requires-Dist: allennlp (==1.2.0) ; (python_version < "3.8") and extra == 'integration'
Requires-Dist: botorch ; (python_version > "3.6") and extra == 'integration'
Requires-Dist: torch (==1.7.1+cpu) ; (sys_platform != "darwin") and extra == 'integration'
Requires-Dist: torchvision (==0.8.2+cpu) ; (sys_platform != "darwin") and extra == 'integration'
Requires-Dist: torch (==1.7.1) ; (sys_platform == "darwin") and extra == 'integration'
Requires-Dist: torchvision (==0.8.2) ; (sys_platform == "darwin") and extra == 'integration'
Provides-Extra: optional
Requires-Dist: bokeh (<2.0.0) ; extra == 'optional'
Requires-Dist: matplotlib (>=3.0.0) ; extra == 'optional'
Requires-Dist: pandas ; extra == 'optional'
Requires-Dist: plotly (>=4.0.0) ; extra == 'optional'
Requires-Dist: redis ; extra == 'optional'
Requires-Dist: scikit-learn (<0.23.0,>=0.19.0) ; extra == 'optional'
Provides-Extra: testing
Requires-Dist: bokeh (<2.0.0) ; extra == 'testing'
Requires-Dist: chainer (>=5.0.0) ; extra == 'testing'
Requires-Dist: cma ; extra == 'testing'
Requires-Dist: fakeredis ; extra == 'testing'
Requires-Dist: lightgbm ; extra == 'testing'
Requires-Dist: matplotlib (>=3.0.0) ; extra == 'testing'
Requires-Dist: mlflow ; extra == 'testing'
Requires-Dist: mpi4py ; extra == 'testing'
Requires-Dist: mxnet ; extra == 'testing'
Requires-Dist: pandas ; extra == 'testing'
Requires-Dist: plotly (>=4.0.0) ; extra == 'testing'
Requires-Dist: pytest ; extra == 'testing'
Requires-Dist: scikit-learn (<0.23.0,>=0.19.0) ; extra == 'testing'
Requires-Dist: scikit-optimize ; extra == 'testing'
Requires-Dist: xgboost ; extra == 'testing'
Requires-Dist: keras ; extra == 'testing'
Requires-Dist: tensorflow ; extra == 'testing'
Requires-Dist: tensorflow-datasets ; extra == 'testing'
Requires-Dist: pytorch-ignite ; extra == 'testing'
Requires-Dist: pytorch-lightning (>=1.0.2) ; extra == 'testing'
Requires-Dist: skorch ; extra == 'testing'
Requires-Dist: catalyst ; extra == 'testing'
Requires-Dist: torchaudio (==0.7.2) ; extra == 'testing'
Requires-Dist: allennlp (==1.2.0) ; extra == 'testing'
Requires-Dist: fastai ; extra == 'testing'
Requires-Dist: botorch ; (python_version > "3.6") and extra == 'testing'
Requires-Dist: torch (==1.7.1+cpu) ; (sys_platform != "darwin") and extra == 'testing'
Requires-Dist: torchvision (==0.8.2+cpu) ; (sys_platform != "darwin") and extra == 'testing'
Requires-Dist: torch (==1.7.1) ; (sys_platform == "darwin") and extra == 'testing'
Requires-Dist: torchvision (==0.8.2) ; (sys_platform == "darwin") and extra == 'testing'
Provides-Extra: tests
Requires-Dist: fakeredis ; extra == 'tests'
Requires-Dist: pytest ; extra == 'tests'

<div align="center"><img src="https://raw.githubusercontent.com/optuna/optuna/master/docs/image/optuna-logo.png" width="800"/></div>

# Optuna: A hyperparameter optimization framework

[![Python](https://img.shields.io/badge/python-3.6%20%7C%203.7%20%7C%203.8%20%7C%203.9-blue)](https://www.python.org)
[![pypi](https://img.shields.io/pypi/v/optuna.svg)](https://pypi.python.org/pypi/optuna)
[![conda](https://img.shields.io/conda/vn/conda-forge/optuna.svg)](https://anaconda.org/conda-forge/optuna)
[![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/optuna/optuna)
[![CircleCI](https://circleci.com/gh/optuna/optuna.svg?style=svg)](https://circleci.com/gh/optuna/optuna)
[![Read the Docs](https://readthedocs.org/projects/optuna/badge/?version=stable)](https://optuna.readthedocs.io/en/stable/)
[![Codecov](https://codecov.io/gh/optuna/optuna/branch/master/graph/badge.svg)](https://codecov.io/gh/optuna/optuna/branch/master)
[![Gitter chat](https://badges.gitter.im/optuna/gitter.svg)](https://gitter.im/optuna/optuna)

[**Website**](https://optuna.org/)
| [**Docs**](https://optuna.readthedocs.io/en/stable/)
| [**Install Guide**](https://optuna.readthedocs.io/en/stable/installation.html)
| [**Tutorial**](https://optuna.readthedocs.io/en/stable/tutorial/index.html)

*Optuna* is an automatic hyperparameter optimization software framework, particularly designed
for machine learning. It features an imperative, *define-by-run* style user API. Thanks to our
*define-by-run* API, the code written with Optuna enjoys high modularity, and the user of
Optuna can dynamically construct the search spaces for the hyperparameters.

## News

- **2020-12-02** Python 3.9 is now supported. Integration modules are still being worked on and is tracked by [#2034](https://github.com/optuna/optuna/issues/2034)
- **2020-09-17** `isort` has been incorporated to keep import statements consistent. Read more about it in [CONTRIBUTING.md](./CONTRIBUTING.md)
- **2020-08-07** We are welcoming [contributions](#contribution) and are working on streamlining the experience. Read more about it in the [blog](https://medium.com/optuna/optuna-wants-your-pull-request-ff619572302c)

## Key Features

Optuna has modern functionalities as follows:

- [Lightweight, versatile, and platform agnostic architecture](https://optuna.readthedocs.io/en/stable/tutorial/001_first.html)
  - Handle a wide variety of tasks with a simple installation that has few requirements.
- [Pythonic search spaces](https://optuna.readthedocs.io/en/stable/tutorial/002_configurations.html)
  - Define search spaces using familiar Python syntax including conditionals and loops.
- [Efficient optimization algorithms](https://optuna.readthedocs.io/en/stable/tutorial/007_pruning.html)
  - Adopt state-of-the-art algorithms for sampling hyper parameters and efficiently pruning unpromising trials.
- [Easy parallelization](https://optuna.readthedocs.io/en/stable/tutorial/004_distributed.html)
  - Scale studies to tens or hundreds or workers with little or no changes to the code.
- [Quick visualization](https://optuna.readthedocs.io/en/stable/reference/visualization/index.html)
  - Inspect optimization histories from a variety of plotting functions.


## Basic Concepts

We use the terms *study* and *trial* as follows:

- Study: optimization based on an objective function
- Trial: a single execution of the objective function

Please refer to sample code below. The goal of a *study* is to find out the optimal set of
hyperparameter values (e.g., `classifier` and `svm_c`) through multiple *trials* (e.g.,
`n_trials=100`). Optuna is a framework designed for the automation and the acceleration of the
optimization *studies*.

[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](http://colab.research.google.com/github/optuna/optuna/blob/master/examples/quickstart.ipynb)

```python
import ...

# Define an objective function to be minimized.
def objective(trial):

    # Invoke suggest methods of a Trial object to generate hyperparameters.
    regressor_name = trial.suggest_categorical('classifier', ['SVR', 'RandomForest'])
    if regressor_name == 'SVR':
        svr_c = trial.suggest_loguniform('svr_c', 1e-10, 1e10)
        regressor_obj = sklearn.svm.SVR(C=svr_c)
    else:
        rf_max_depth = trial.suggest_int('rf_max_depth', 2, 32)
        regressor_obj = sklearn.ensemble.RandomForestRegressor(max_depth=rf_max_depth)

    X, y = sklearn.datasets.load_boston(return_X_y=True)
    X_train, X_val, y_train, y_val = sklearn.model_selection.train_test_split(X, y, random_state=0)

    regressor_obj.fit(X_train, y_train)
    y_pred = regressor_obj.predict(X_val)

    error = sklearn.metrics.mean_squared_error(y_val, y_pred)

    return error  # An objective value linked with the Trial object.

study = optuna.create_study()  # Create a new study.
study.optimize(objective, n_trials=100)  # Invoke optimization of the objective function.
```


## Integrations

[Integrations modules](https://optuna.readthedocs.io/en/stable/tutorial/pruning.html), which allow pruning, or early stopping, of unpromising trials are available for the following libraries:

* [AllenNLP](./examples/allennlp)
* [Catalyst](./examples/catalyst_simple.py)
* [Catboost](./examples/catboost_simple.py)
* [Chainer](./examples/pruning/chainer_integration.py)
* [FastAI](./examples/fastai_simple.py)
* [Keras](./examples/pruning/keras_integration.py)
* [LightGBM](./examples/pruning/lightgbm_integration.py)
* [MXNet](./examples/pruning/mxnet_integration.py)
* [PyTorch](./examples/pytorch_simple.py)
* [PyTorch Ignite](./examples/pytorch_ignite_simple.py)
* [PyTorch Lightning](./examples/pytorch_lightning_simple.py)
* [TensorFlow](./examples/pruning/tensorflow_estimator_integration.py)
* [tf.keras](./examples/pruning/tfkeras_integration.py)
* [XGBoost](./examples/pruning/xgboost_integration.py)

## Installation

Optuna is available at [the Python Package Index](https://pypi.org/project/optuna/) and on [Anaconda Cloud](https://anaconda.org/conda-forge/optuna).

```bash
# PyPI
$ pip install optuna
```

```bash
# Anaconda Cloud
$ conda install -c conda-forge optuna
```

Optuna supports Python 3.6 or newer.

Also, we also provide Optuna docker images on [DockerHub](https://hub.docker.com/r/optuna/optuna).

## Communication

- [GitHub Issues] for bug reports, feature requests and questions.
- [Gitter] for interactive chat with developers.
- [Stack Overflow] for questions.

[GitHub issues]: https://github.com/optuna/optuna/issues
[Gitter]: https://gitter.im/optuna/optuna
[Stack Overflow]: https://stackoverflow.com/questions/tagged/optuna


## Contribution

Any contributions to Optuna are more than welcome!

If you are new to Optuna, please check the [good first issues](https://github.com/optuna/optuna/labels/good%20first%20issue). They are relatively simple, well-defined and are often good starting points for you to get familiar with the contribution workflow and other developers.

If you already have contributed to Optuna, we recommend the other [contribution-welcome issues](https://github.com/optuna/optuna/labels/contribution-welcome).

For general guidelines how to contribute to the project, take a look at [CONTRIBUTING.md](./CONTRIBUTING.md).


## Reference

Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019.
Optuna: A Next-generation Hyperparameter Optimization Framework. In KDD ([arXiv](https://arxiv.org/abs/1907.10902)).


