Optimization
To show how the Evolutionary package can be used, we minimize the Rosenbrock function, a classical test problem for numerical optimization. We'll assume that you've already installed the Evolutionary package using Julia's package manager.
Objective Function Definition
First, we load Evolutionary and define the Rosenbrock function:
using Evolutionary
f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2 # Rosenbrock
There are various ways to define your objective function:
- For single-objective optimization, the objective function has to have one parameter and return a scalar value, e.g.
soofun(x) = x[1] + x[2]
- For multi-objective optimization, the objective function has to return an vector of values, e.g.
moofun(x) = [ x[1], x[2]+1 ]
- If multi-objective function has one parameter, the resulting value array will be copied. To reduce, additional data copy, the function can be defined with two parameters to perform in-place change of the result, e.g.
function moofun(F, x)
F[1] = x
F[2] = x +1
F
end
Perform Optimization
Once we've defined this function, we can find the minimizer (the input that minimizes the objective) and the minimum (the value of the objective at the minimizer) using any of our favorite optimization algorithms. With a function defined, we just specify a form of an individual x
of the population for an evolutionary algorithm, and call optimize
with a starting individual x0
and a particular optimization algorithm, e.g. CMAES()
:
x0 = [0.0, 0.0];
Evolutionary.optimize(f, x0, CMAES())
Evolutionary.optimize
— Functionoptimize(f[, F], indiv, algo[, opts])
optimize(f[, F], constr, algo[, opts])
optimize(f[, F], constr, indiv, algo[, opts])
optimize(f[, F], constr, algo, poplt[, opts])
Perform optimization of the function f
using the algorithm algo
with the population, composed of the initial population poplt
, or individuals similar to the original individual indiv
, or generated from the constraints constr
, with the options opts
.
- For multi-objective optimization, the objective value
F
must be provided.
Configurable options
There are several options that simply take on some default values if the user doesn't provide any.
Algorithm options
There are different algorithms available in Evolutionary
, and they are all listed below. Notice that the constructors are written without input here, but they generally take keywords to tweak the way they work. See the pages describing each solver for more detail.
General options
In addition to the algorithm, you can alter the behavior of the optimization procedure by using the following Options
keyword arguments:
Evolutionary.Options
— TypeThere are following options available:
abstol::Float64
: the absolute tolerance used in the convergence testreltol::Float64
: the relative tolerance used in the convergence testsuccessive_f_tol::Integer
: the additional number of the iterations of the optimization algorithm after the convergence test is satisfied (default: 10)iterations::Integer
: the total number of the iterations of the optimization algorithm (default: 1000)show_trace::Bool
: enable the trace information display during the optimization (default: false).store_trace::Bool
: enable the trace information capturing during the optimization (default: false). The trace can be accessed by usingtrace
function after optimization is finished.show_every::Integer
: show everyn
s successive trace message (default: 1)time_limit::Float64
: the time limit for the optimization run in seconds. If the value set toNaN
then the limit is not set. (default: NaN)callback
: the callback function that is called after each iteration of the optimization algorithm. The function accepts as parameter a trace dictionary, and must return aBool
value which iftrue
terminates the optimization. (default: nothing)parallelization::Symbol
: allows parallelization of the population fitness evaluation if set to:thread
using multiple threads (default::serial
)rng::AbstractRNG
: a random number generator object that is used to control generation of random data during the evolutionary optimization (default:Random.default_rng()
)
We currently recommend the statically dispatched interface by using the Evolutionary.Options
constructor:
res = Evolutionary.optimize(x->-sum(x),
BitVector(zeros(30)),
GA(selection=uniformranking(5),
mutation=flip, crossover=SPX),
Evolutionary.Options(iterations=10))
Obtaining results
After we have our results in res
object, we can use the API for getting optimization results. This consists of a collection of functions. They are not exported, so they have to be prefixed by Evolutionary.
. Say we do the following optimization:
julia> res = Evolutionary.optimize(x->-sum(x), BitVector(zeros(3)), GA())
* Status: success
* Candidate solution
Minimizer: [false, false, false]
Minimum: 0
Iterations: 11
* Found with
Algorithm: GA[P=50,x=0.8,μ=0.1,ɛ=0]
* Convergence measures
|f(x) - f(x')| = 0.0 ≤ 1.0e-12
* Work counters
Seconds run: 0.0466 (vs limit Inf)
Iterations: 11
f(x) calls: 600
You can inspect the result by using a collection of the auxiliary functions, e.g. the minimizer
and minimum
of the objective functions, which can be found using
julia> Evolutionary.minimizer(res)
3-element BitVector:
0
0
0
julia> Evolutionary.minimum(res)
0
Complete list of functions
An OptimizationResults
interface for representing an optimization result.
Evolutionary.OptimizationResults
— TypeAbstract evolutionary optimization result type
Base.summary
— Methodsummary(result)
Shows the optimization algorithm that produced this result
.
Evolutionary.minimizer
— Methodminimizer(result)
A minimizer object from the optimization result
.
Base.minimum
— Methodminimum(result)
A minimum value from the optimization result
.
Evolutionary.iterations
— Methoditerations(result)
A number of iterations to reach the minimum.
Evolutionary.iteration_limit_reached
— Methoditeration_limit_reached(result)
Returns true
if the iteration limit was reached.
Evolutionary.trace
— Methodtrace(result)
Returns a trace of optimization states from the optimization result
.
NLSolversBase.f_calls
— Methodf_calls(result)
Returns a number of an objective function calls.
Evolutionary.abstol
— Methodabstol(result)
Returns an absolute tolerance value of the optimization result
.
Evolutionary.reltol
— Methodreltol(result)
Returns a relative tolerance value of the optimization result
.
An implementation of the result object for evolutionary optimizations.
Evolutionary.EvolutionaryOptimizationResults
— TypeEvolutionary optimization result type
Evolutionary.converged
— Methodconverged(result)
Returns true
if the optimization successfully converged to a minimum value.
Trace
When store_trace
and/or show_trace
options are set to true
in the Options
object, an optimization trace is either captured and/or shown on the screen. By default, only the current state minimum value is displayed in the trace. In order to extend trace record, you need to override trace!
function providing specialize function behavior on one of specific parameters.
Evolutionary.trace!
— Methodtrace!(record::Dict{String,Any}, objfun, state, population, method, options)
Update the trace record
. This function allows to supplement an additional information into the optimization algorithm trace by modifying a trace record
. It can be overridden by specifying particular parameter types.
Commonly, you would define a specializations of a state
, population
, or method
parameters of trace!
function, e.g.
function trace!(record::Dict{String,Any}, objfun, state, population, method::CMAES, options)
record["σ"] = state.σ
end
Parallelization
If the objective function is heavily CPU-bound, it's possible to utilize multiple processors/threads to speed up computations. To enable multi-threading evaluation of the objective function, set parallelization
option to :thread
in the Options
object.