Evolution Strategies for Constrained Optimization

Evolution strategies (ESs), especially versions of the so-called covariance matrix adaptation ES (CMA-ES), are arguably the currently best-performing general purpose direct search methods for unconstrained optimization of real-parameter black-box optimization problems as often encountered in simulation-based optimization and other fields of engineering optimization. However, up until now, the success of these direct search strategies is rather restricted to the unconstrained case. That is, the incorporation of equality and inequality constraints in the design of ESs is still in an infant state when compared to other classes of Evolutionary Algorithms such as Differential Evolution. It is the goal of this project to foster the development of ESs for constrained optimization based on a theoretically-grounded basis. This will be accomplished by a closely coupled research program that connects theoretically motivated algorithm design, analysis, and evaluation of the direct search strategies developed. Being based on the knowledge gained through this research, a deeper understanding of the working principles of such search strategies in constrained search spaces is expected. This will not only lead to better performing ESs, but also to general design principles for Evolutionary Algorithms.

This research project was aiming at the analysis and design of algorithms for optimization problems in real-valued search spaces as they occur in many applications in engineering, natural sciences, and economy. While there are various approaches to tackle such optimization problems, the use of so-called Evolution Strategies (ESs) has proven itself as an alternative especially well-suited for difficult non-linear problems. The ES algorithms are gleaned from nature mimicking the process of Darwinian evolution by improving initial candidate solutions gradually -- as in nature -- by applying small mutations, recombination of solutions and selection, thus, getting better and better solutions. However, up until now, the incorporation of restrictions, so-called constraints, as they are often prevalent in real-world applications, has only been done in an ad hoc and non-systematic manner. This project has significantly changed the situation for Evolution Strategies. Being based on a thorough theoretically grounded analysis, ESs have been developed that are able to handle equality as well as inequality constraints. Unlike most of other evolutionary algorithms, the methods developed are also able to realize an inner-point behavior. That is, the solutions generated during the evolutionary improvement process are always valid (i.e. feasible). This is especially desirable in simulation-based optimization problems where leaving the domain of admissible parameters often results in abnormal terminations of the simulation software. This is also an issue in financial applications where balance equations must be exactly fulfilled. Besides the development of ES algorithms that are simpler, on par with, and are often better performing then the state-of-the-art, also the border of the theory of these probabilistic algorithms has been pushed forward. This is important because understanding these algorithms, which are difficult to be analyzed, is a prerequisite for developing even better performing algorithms.