Future Works
============

Mathematical Optimization
-------------------------

* Cartis, C., Massart, E., & Otemissov, A. (2022). Bound-constrained global optimization of functions with low effective dimensionality using multiple random embeddings. Mathematical Programming, 1-62.
* Cartis, C., Massart, E., & Otemissov, A. (2022). Global optimization using random embeddings. Mathematical Programming, 1-49.
* Rios, L. M., & Sahinidis, N. V. (2013). Derivative-free optimization: A review of algorithms and comparison of software implementations. Journal of Global Optimization, 56(3), 1247-1293.
* Collette, Y., Hansen, N., Pujol, G., Salazar Aponte, D. and Le Riche, R., 2013. Object‐oriented programming of optimizers–Examples in Scilab. Multidisciplinary Design Optimization in Computational Mechanics, pp.499-538.

SPSA
----

* Spall, J. C., Hill, S. D., & Stark, D. R. (2006). Theoretical framework for comparing several stochastic optimization approaches. Probabilistic and Randomized Methods for Design under Uncertainty, 99-117.
* Spall, J. C. (1999, December). Stochastic optimization and the simultaneous perturbation method. In Proceedings of Conference on Winter Simulation (pp. 101-109).

ES
--

* Maheswaranathan, N., Metz, L., Tucker, G., Choi, D., & Sohl-Dickstein, J. (2019, May). Guided evolutionary strategies: Augmenting random search with surrogate gradients. In International Conference on Machine Learning (pp. 4264-4273). PMLR.
* Choromanski, K., Rowland, M., Sindhwani, V., Turner, R., & Weller, A. (2018, July). Structured evolution with compact architectures for scalable policy optimization. In International Conference on Machine Learning (pp. 970-978). PMLR.
* Pošík, P., Huyer, W., & Pál, L. (2012). A comparison of global search algorithms for continuous black box optimization. Evolutionary Computation, 20(4), 509-541.
* Arnold, D. V., & Beyer, H. G. (2004). Performance analysis of evolutionary optimization with cumulative step length adaptation. IEEE Transactions on Automatic Control, 49(4), 617-622.
* Beyer, H. G., & Arnold, D. V. (2003). Qualms regarding the optimality of cumulative path length control in CSA/CMA-evolution strategies. Evolutionary Computation, 11(1), 19-28.

EDA
---

* Teytaud, F. and Teytaud, O., 2009, July. Why one must use reweighting in estimation of distribution algorithms. In Proceedings of ACM Annual Conference on Genetic and Evolutionary Computation (pp. 453-460).

PSO
---

* Camacho-Villalón, C. L., Dorigo, M., & Stützle, T. (2021). PSO-X: A component-based framework for the automatic design of particle swarm optimization algorithms. IEEE Transactions on Evolutionary Computation, 26(3), 402-416.
* Bonyadi, M. R., & Michalewicz, Z. (2015). Analysis of stability, local convergence, and transformation sensitivity of a variant of the particle swarm optimization algorithm. IEEE Transactions on Evolutionary Computation, 20(3), 370-385.
* Su, S., Zhang, Z., Liu, A. X., Cheng, X., Wang, Y., & Zhao, X. (2014). Energy-aware virtual network embedding. IEEE/ACM Transactions on Networking, 22(5), 1607-1620.
* De Oca, M. A. M., Stutzle, T., Birattari, M., & Dorigo, M. (2009). Frankenstein's PSO: A composite particle swarm optimization algorithm. IEEE Transactions on Evolutionary Computation, 13(5), 1120-1132.
* Mendes, R., Kennedy, J., & Neves, J. (2004). The fully informed particle swarm: simpler, maybe better. IEEE Transactions on Evolutionary Computation, 8(3), 204-210.
* Clerc, M., & Kennedy, J. (2002). The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 6(1), 58-73.

MA
--

* Lozano, M., Herrera, F., Krasnogor, N., & Molina, D. (2004). Real-coded memetic algorithms with crossover hill-climbing. Evolutionary Computation, 12(3), 273-302.
* Renders, J. M., & Flasse, S. P. (1996). Hybrid methods using genetic algorithms for global optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 26(2), 243-258.

GA
--

* Drugan, M. M., & Thierens, D. (2010). Geometrical recombination operators for real-coded evolutionary mcmcs. Evolutionary Computation, 18(2), 157-198.

NM
--
* Gao, F., & Han, L. (2012). Implementing the Nelder-Mead simplex algorithm with adaptive parameters. Computational Optimization and Applications, 51(1), 259-277.

RS
--

* Li, L. and Talwalkar, A., 2020. Random search and reproducibility for neural architecture search. In Uncertainty in Artificial Intelligence (pp. 367-377). PMLR.
* Sener, O., & Koltun, V., 2019. Learning to guide random search. In International Conference on Learning Representations.
* Chechkin, A. and Sokolov, I., 2018. Random search with resetting: A unified renewal approach. Physical Review Letters, 121(5), p.050601.
* Falcón-Cortés, A., Boyer, D., Giuggioli, L. and Majumdar, S.N., 2017. Localization transition induced by learning in random searches. Physical Review Letters, 119(14), p.140603.
* Chupeau, M., Bénichou, O. and Voituriez, R., 2015. Cover times of random searches. Nature Physics, 11(10), pp.844-847.
* Qi, Y., Mao, X., Lei, Y., Dai, Z. and Wang, C., 2014, May. The strength of random search on automated program repair. In Proceedings of International Conference on Software Engineering (pp. 254-265). IEEE.
* Hein, A.M. and McKinley, S.A., 2012. Sensing and decision-making in random search. Proceedings of the National Academy of Sciences, 109(30), pp.12070-12074.
* Tejedor, V., Voituriez, R. and Bénichou, O., 2012. Optimizing persistent random searches. Physical Review Letters, 108(8), p.088103.
* Zabinsky, Z.B., 2003. Stochastic adaptive search for global optimization. Springer Science & Business Media.
* Viswanathan, G.M., Buldyrev, S.V., Havlin, S., Da Luz, M.G.E., Raposo, E.P. and Stanley, H.E., 1999. Optimizing the success of random searches. Nature, 401(6756), pp.911-914.
* Yakowitz, S. and Lugosi, E., 1990. Random search in the presence of noise, with application to machine learning. SIAM Journal on Scientific and Statistical Computing, 11(4), pp.702-712.
* Devroye, L.P., 1978. Progressive global random search of continuous functions. Mathematical Programming, 15(1), pp.330-342.
* Schrack, G. and Choit, M., 1976. Optimized relative step size random searches. Mathematical Programming, 10(1), pp.230-244. [ https://github.com/Evolutionary-Intelligence/pypop/blob/main/pypop7/optimizers/rs/rs.py ]
* Schumer, M.A. and Steiglitz, K., 1968. Adaptive step size random search. IEEE Transactions on Automatic Control, 13(3), pp.270-276. [ https://github.com/Evolutionary-Intelligence/pypop/blob/main/pypop7/optimizers/rs/rs.py ]
* Matyas, J., 1965. Random optimization. Automation and Remote control, 26(2), pp.246-253. [ https://github.com/Evolutionary-Intelligence/pypop/blob/main/pypop7/optimizers/rs/rs.py ]
* Karnopp, D.C., 1963. Random search techniques for optimization problems. Automatica, 1(2-3), pp.111-121.
* Rastrigin, L.A., 1963. The convergence of the random search method in the extremal control of a many parameter system. Automaton & Remote Control, 24, pp.1337-1342. [ https://github.com/Evolutionary-Intelligence/pypop/blob/main/pypop7/optimizers/rs/rs.py ]
* Brooks, S.H., 1958. A discussion of random methods for seeking maxima. Operations Research, 6(2), pp.244-251. [ https://github.com/Evolutionary-Intelligence/pypop/blob/main/pypop7/optimizers/rs/rs.py ]

BBO/DFO/ZOO
-----------

* Antonakopoulos, K., Vu, D.Q., Cevher, V., Levy, K. and Mertikopoulos, P., 2022, June. UnderGrad: A universal black-box optimization method with almost dimension-free convergence rate guarantees. In International Conference on Machine Learning (pp. 772-795). PMLR.
* Flaxman, A. D., Kalai, A. T., & McMahan, H. B. (2005, January). Online convex optimization in the bandit setting: gradient descent without a gradient. In Proceedings of Annual ACM-SIAM symposium on Discrete Algorithms (pp. 385-394).
