
<p align="center">
<img width="350" src="https://github.com/gorkamunoz/rl_opts/blob/master/nbs/figs/logo_midjourney_scaled.png?raw=true">
</p>
<h1 align="center">
RL-OptS
</h1>
<h4 align="center">
Reinforcement Learning of Optimal Search strategies
</h4>
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
<p align="center">
<a href="https://zenodo.org/badge/latestdoi/424986383"><img src="https://zenodo.org/badge/424986383.svg" alt="DOI"></a>
<a href="https://badge.fury.io/py/rl_opts"><img src="https://badge.fury.io/py/rl_opts.svg" alt="PyPI version"></a>
<a href="https://badge.fury.io/py/b"><img src="https://img.shields.io/badge/python-3.9-red" alt="Python version"></a>
</p>

This library builds the necessary tools needed to study, replicate and
develop reinforcement learning agents for target search problems, as
well as a benchmark baselines with which to compare. This library is
based in two different publications:

- [“Optimal foraging strategies can be
  learned”](https://arxiv.org/abs/2303.06050) by *G. Muñoz-Gil, A.
  López-Incera, L. J. Fiderer* and *H. J. Briegel*. Here we developed
  agents able to learn how to forage efficiently in environments with
  multiple targets.

- [“Learning to reset in target search
  problems”](https://arxiv.org/abs/2303.06050) by *G. Muñoz-Gil, H. J.
  Briegel* and *M. Caraglio*. Here we extended the agents to be able to
  reset to the origin, a feature that has revolutionize target search
  problems in the last years.

### Installation

You can access all these tools installing the python package `rl_opts`
via Pypi:

``` python
pip install rl-opts
```

You can also opt for cloning the [source
repository](https://github.com/gorkamunoz/rl_opts) and executing the
following on the parent folder you just cloned the repo:

``` python
pip install -e rl_opts
```

This will install both the library and the necessary packages.

### Tutorials

We have prepared a series of tutorials to guide you through the most
important functionalities of the package. You can find them in the
[Tutorials
folder](https://github.com/gorkamunoz/rl_opts/tree/master/nbs/tutorials)
of the Github repository or in the Tutorials tab of our
[webpage](https://gorkamunoz.github.io/rl_opts/), with notebooks that
will help you navigate the package as well as reproducing the results of
our paper via minimal examples. In particular, we have three tutorials:

- <a href="tutorials/tutorial_learning.ipynb" style="text-decoration:none">Reinforcement
  learning </a> : shows how to train a RL agent based on Projective
  Simulation agents to search targets in randomly distributed
  environments as the ones considered in our paper.
- <a href="tutorials/tutorial_reset.ipynb" style="text-decoration:none">Learning
  to reset in target search problems </a> : shows how to train a RL
  agent similar to the previous, but with the ability to reset to the
  origin, an action that is learned along its spatial dynamics.
- <a href="tutorials/tutorial_imitation.ipynb" style="text-decoration:none">Imitation
  learning </a> : shows how to train a RL agent to imitate the policy of
  an expert equipped with a pre-trained policy. The latter is based on
  the benchmark strategies common in the literature.
- <a href="tutorials/tutorial_benchmarks.ipynb" style="text-decoration:none">Benchmarks
  </a> : shows how launch various benchmark strategies with which to
  compare the trained RL agents.

### Cite

We kindly ask you to cite our paper if any of the previous material was
useful for your work:

``` latex
@article{munoz2024optimal,
  title={Optimal foraging strategies can be learned},
  author={Mu{\~n}oz-Gil, Gorka and L{\'o}pez-Incera, Andrea and Fiderer, Lukas J and Briegel, Hans J},
  journal={New Journal of Physics},
  volume={26},
  number={1},
  pages={013010},
  year={2024},
  publisher={IOP Publishing}
}
```
