Dynamic Differential Evolution Variants

Published Date: 02 Nov 2017

*G. Jeyakumar **C. Shunmuga Velayutham

Amrita School of Engineering

Amrita Vishwa Vidyapeetham, Coimbatore, Tamil Nadu, India

*[email protected]

*[email protected]

Abstract : In this paper we extend the dynamicity of Differential Evolution proposed for DE/rand/1/bin and DE/best/1/bin to five more variants DE/rand/2, DE/best/2, DE/current-to-rand/1, DE/current-to-best/1 and DE/rand-to-best/1. We present an empirical, comparative performance, analysis of fourteen variants of Differential Evolution (DE) and Dynamic Differential Evolution (DDE) algorithms (seven variants with two crossovers – binomial and exponential) to solve unconstrained global optimization problems. The aim of this paper is to identify competitive DE and DDE variants which perform well on different problems, and to compare the performance of DDE variants with DE variants. The performance of fourteen variants of DE and DDE are analyzed by implementing them on 14 test functions. The analysis (done based on mean objective function value, probability of convergence and success performance) shows the superiority of DDE variants and identifies the competitive DE and DDE variants.

Key Words: Differential Evolution; dynamic differential evolution; differential mutation strategies; successful runs; probability of convergence

1. Introduction

Differential Evolution (DE) is an Evolutionary Algorithm (EA), introduced by Storn and Price in 1995 [1] as a global optimizer for continuous problem [2]. It has been widely applied as an optimizer to solve many global optimization problems [3, 4]. It has proved its superiority in solving both widely used benchmark problems and real-world applications [5].

Like other EAs, DE also employs the three key operators viz., mutation, crossover and selection and it is a reliable algorithm for solving nonlinear and multimodal problems. DE has some unique properties that makes it to behave superior than other EAs. In DE, the way of generating offspring (trial vectors) and the selection operation to decide the survivor for the next generation are different from other EAs. The mutation operation used in DE is termed as differential mutation operation which works based on the distribution of the parent vectors in the current population , the mutated vector then undergoes a recombination operation with a target vector to generate a trial vector. A selection operation (one-to-one greedy selection) is carried out between the trail vector and the target vector [6].

DE has become a widely used technique for real-valued parameter optimization, due to its simplicity, faster convergence rate and robustness. On the other hand, in the process of updating the population, DE is static i.e. DE does not respond to any progress immediately. In a particular generation a same population is used till the generation is over, even if it produces a better candidate it is reserved for next generation. This makes DE to converge slowly. To improve this problem Dynamic Differential Evolution (DDE) has been proposed in [7, 8] for two DE variants viz., DE/rand/1 and DE/best/1. In DDE, the population of a generation is updated after every selection operation, thus DDE responds to any improvement immediately.

We found, in literature, only few investigations to show the efficiency of DDE variants. With various mutation and recombination strategies, DE has come up with different variants. Choosing the best variant to solve the problem at hand is not an easy task, regard to No Free Lunch Theorem [9] no single DE variant is found to be best for all the problem. Further investigation on DE variants to understand their performance on problems with different features is essential to efficiently choose the required variant to successfully (less time and cost) solve the problems at hand.

In this paper we extend the dynamicity to seven DE variants viz. DE/rand/2, DE/best/2, DE/current-to-rand/1, DE/current-to-best/1, DE/rand-to-best/1 including already extended DE/rand/1 and DE/best/1. An empirical performance analysis of fourteen DDE variants (7 DDE variants with binomial and exponential crossovers) on fourteen benchmark problems has been carried out in this paper along with comparative performance analysis against their classical counterparts.

The content of the paper in presented as follows. The algorithmic description of DE and DDE is presented in Section 2, Section 3 describes the related works. Section 4 presents the design of experiment. The simulation results are discussed in Section 5, Section 6 concludes the paper.

2. DE and DDE Algorithms

DE algorithm explores the problem domain by sampling at multiple, randomly chosen NP D-dimensional parameter vectors (population of initial points), so-called individuals, which encode the candidate solution (initial population) . An iterative process (generation) is started after the population is initialized [6]. At each generation, DE employs the differential mutation operations to produce a mutant vector , with respect to each individual, the so called target vector, in the current population. The mutant vector is created using the weighted difference of parent solutions in the current population [6]. A number of differential mutation strategies have been proposed in the literature that differs in the way the mutant vector is created. The seven commonly used mutation strategies [10] are listed in "table 1".

Table 1

Mutation Strategies

Nomenclature

Variant

rand/1

best/1

rand/2

best/2

current-to-rand/1

current-to-best/1

rand-to-best/1

After the differential mutation strategy, DE then uses a crossover operation in which the mutant vector mixes with target vector and generates a trial vector or offspring. The two frequently used crossover schemes are binomial (uniform) crossover and exponential crossover [11]. After the mutation and crossover operations, a one-to-one knockout competition between the target vector and its corresponding trial vector based on the objective function values decides the survivor, among the two, for the next generation.

The above 3 steps of differential mutation, crossover, followed by selection marks the end of one DE generation. These steps are repeated generation after generation until a stopping criterion is satisfied. The fig. 1, depicts the algorithmic description of general DE. From the seven methods of mutation and two methods of crossover fourteen different DE variants [6] viz., DE/rand/1/bin, DE/rand/1/exp, DE/best/1/bin, DE/best/1/exp, DE/rand/2/bin, DE/rand/2/exp, DE/best/2/bin, DE/best/2/exp, DE/current-to-rand/1/bin, DE/current-to-rand/1/exp, DE/current-to-best/1/bin, DE/current-to-best/1/exp, DE/rand-to-best/1/bin and DE/rand-to-best/1/exp are formed.

The DE algorithm [11] in fig. 1, shows that the newly generated trail vectors do not make any updation in the current population. The better trail vectors generated in the current generation are reserved for the next generation.The dynamic differential evolution, shown in fig. 2, employs a dynamic population updation mechanism, the better trail vectors are placed in the current population itself, to use in the current generation. DDE differs from DE only by its population updation mechanism, except this DDE uses the similar set of operations and control parameter as DE.

Population Initialization

X (0) ← { x1(0),…,xNP(0) }

g ←0

Compute { f(x1(g)),...,f(xNP(g)) }

while the stopping condition is false do

for i = 1 to NP do

yi ← generatemutant(X(g))

zi ← crossover(xi(g), yi)

if f(zi) < f(xi(g)) then

xi(g+1) ← zi

else

xi(g+1) ← xi(g)

end if

end for

g ← g+1

Compute {f(x1(g)),…,f(xNP(g))}

end while

Population Initialization

X(0) ← { x1(0),…,xNP(0) } g ←0

Compute { f(x1(g)),...,f(xNP(g)) }

while the stopping condition is false do

for i = 1 To NP do

yi ← generatemutant(X(g))

zi ← crossover(xi(g), yi)

if f(zi) < f(xi(g)) then

xi(g) ← zi

endif

if f(zi) < f(xbest(g)) then

best ← i

endif

end for

g ← g+1

Compute {f(x1(g)),…,f(xNP(g))}

end while

Figure 1. The Algorithmic Description of DE Figure 2. The Algorithmic Description of DDE

In DDE, at each generation the target vector is replaced by the trial vector, if the trial vector is better than the target vector. And the trail vector is labeled as the current optimal vector, if the trail vector is better than the current optimal vector. Thus, the new trail vectors are generated using the updated population, and hence DDE always responds to any progress immediately. In DDE, the creation of every NP trial vectors is considered as one generation.

The dynamic evolution mechanism of DDE is extended to the above said 14 DE variants, the newly formed fourteen DDE variants [11,12] are as follows DDE/rand/1/bin, DDE/rand/1/exp, DDE/best/1/bin, DDE/best/1/exp, DDE/rand/2/bin, DDE/rand/2/exp, DDE/best/2/bin, DDE/best/2/exp, DDE/current-to-rand/1/bin, DDE/current-to-rand/1/exp, DDE/current-to-best/1/bin, DDE/current-to-best/1/exp, DDE/rand-to-best/1/bin and DDE/rand-to-best/1/exp. In this paper, an empirical comparative performance analysis of fourteen DE and DDE variants has been carried out.

3. Related Works

The conceptual simplicity and potential of DE has attracted many researchers, who are working on its improvement, resulting in many variants of the algorithm [13, 14, 15 and 16]. However, in this paper, we restrict to classical DE variants only.

Menzura-Montes et. al. [17] has reported a performance comparison of eight DE variants on unconstrained optimization problems. Variants with arithmetic recombination also considered in their work. The variants which are identified as competitive variants in this study are rand/1/bin, best/1/bin, current-to-rand/1/bin and rand/2/dir. The variants which we used in our study viz., best/2/*, rand-to-best/1/* and rand/2/* were not used in this study.

Performance analysis of ten DE variants is reported by Babu and Munawar [18], on solving the optimal design problem of shell-and-tube heat exchangers. Their study involved testing an extensive 9680 combinations of key parameters of DE variants and concluded best/*/* strategies to be better than rand/*/* strategies with best/1/* strategies to be the best out of the ten DE variants considered. The variants viz., current-to-rand/1/* and current-to-best/1/* of our variants suite, is not considered in this study.

A new variant of DE called SaDE (Self Adaptive DE) is proposed by Qin, Huang and Suganthan [19], which involves a self-adapting mechanism (learning from experience) to decide the trial vector generation strategy and control parameters. Four variants viz. rand/1/bin, rand-to-best/2/bin, rand/2/bin and current-to-rand/1/bin were considered in the strategy candidate pool.

Dynamic Differential Evolution is proposed by Qing in [7], the study reports the performance analysis of DDE/best/1 variant on a function minimization problem with 8, 16, 24, 50 and 100 optimization parameters and on a benchmark electromagnetic inverse scattering problem. The superiority of DDE over the classical DE is shown significantly.

Comparison of DDE/rand/1/bin and DDE/best/1/bin with classical DE is studied by Qing in [8]. The study used 37 benchmark functions with dimension less than 10 and three application problems with 6, 8, 9, 16 and 24 dimensions. DDE/best/1/bin was identified as competitive variant.

Jeyakumar and Shunmuga Velayutham [12], compared the efficacy of DDE variants over their corresponding DE variants by implementing the dynamic version of 14 DE variants on six benchmarking functions, the variants are compared by their mean objective function value (MOV) and probability of convergence (Pc). However, this paper extending the above said work by implementing the variants on 14 benchmark functions and also measuring the success performance of the variants, which indeed has given more insights for the comparison.

4. Design of Experiment

This paper investigates the performance efficacy of fourteen DDE variants against their classical DE counterparts, identified and listed in Section 2, on a set of benchmark problems with high dimensionality and different features.

We have chosen 14 test functions f1 - f14 [10, 17, 20] of dimensionality 30, "table 2", among which f1 - f5 are unimodal separable, f6 is unimodal nonseparable, f7 - f9 are multimodal separable functions and functions f10 - f14 are multimodal nonseparable. All the chosen test functions are dimension wise scalable and have an optimum value of zero except f7. To have the common optimum value of zero to all functions, description of f7 is adjusted by adding the optimal value 12569.49 [10, 17].

The three crucial control parameters of the DE and DDE algorithms, irrespective of the fourteen variants, are population size (NP), scaling factor (F) and crossover rate (CR). The population size NP is set as 60 and the maximum number of generations as 3000 (based on an empirical analysis). The algorithm will stop when it reaches the stopping criteria (fixed by us as 1 x 10-12), or will proceed up to the maximum number of generation. The range for the scaling factor F [17, 21] is set as F [0.3, 0.9], the F value is generated newly for each generation. The value for K is set same as F.

Table 2

The Details of the Benchmark Problems

f1 – Sphere model

f2 – Schwefel’s Problem 2.22

f3 - Schwefel’s Problem 2.21

f4 – Step Function

f5 – Quartic Function with Noise

f6 – Schwefel’s Problem 1.2

f7 – Generalized

Schwefel’s Problem 2.26

f8 – Generalized

Restrigin’s Function

f9-Bohachevsky Functions

f10 - Generalized Rosenbrock's Function

f11 – Ackley’s Function

f12-Generalized Griewank's Function

f13 – Generalized Penalized functions

f14 – Generalized Penalized functions

The crossover rate (CR) was calculated based on bootstrap test for each variant-test function combination [6]. Eleven different values for the CR viz., {0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 and 1.0} were used. For the combination of each variant-function-CR value, 50 independent runs were conducted. We calculated the confidence interval for the mean objective function value by applying a bootstrap test, based on the obtained results. The CR value corresponding to the best confidence interval (i.e., 95%) [11] was chosen to be used in our experiments. The DE and DDE variants along with the CR values for each function are presented in "table 3".

Table 3

The Fourteen Variants of DE and DDE Along with the CR Value used

Sno

Variant

f1

f2

f3

f4

f5

f6

f7

f8

f9

f10

f11

f12

f13

f14

1

rand/1/bin

DE

0.9

0.2

0.5

0.2

0.8

0.9

0.5

0.1

0.1

0.9

0.9

0.1

0.1

0.1

DDE

0.9

0.2

0.2

0.2

0.8

0.9

0.2

0.1

0.1

0.9

0.8

0.1

0.1

0.1

2

rand/1/exp

DE

0.9

0.9

0.9

0.9

0.9

0.9

0.0

0.9

0.9

0.9

0.9

0.9

0.9

0.9

DDE

0.9

0.9

0.9

0.9

0.9

0.9

0.3

0.9

0.9

0.9

0.9

0.9

0.9

0.9

3

best/1/bin

DE

0.1

0.1

0.2

0.1

0.7

0.5

0.1

0.1

0.1

0.8

0.1

0.1

0.3

0.8

DDE

0.9

0.1

0.9

0.1

0.9

0.6

0.1

0.1

0.1

0.9

0.1

0.9

0.4

0.8

4

best/1/exp

DE

0.9

0.8

0.9

0.8

0.9

0.9

0.7

0.9

0.8

0.8

0.8

0.8

0.9

0.8

DDE

0.9

0.9

0.9

0.9

0.9

0.9

0.6

0.9

0.9

1.0

0.8

0.9

0.9

0.9

5

rand/2/bin

DE

0.3

0.1

0.2

0.2

0.9

0.9

0.2

0.1

0.1

0.9

0.1

0.1

0.1

0.1

DDE

0.3

0.1

0.2

0.2

0.9

0.9

0.3

0.1

0.1

0.9

0.1

0.1

0.1

0.1

6

rand/2/exp

DE

0.9

0.9

0.9

0.9

0.9

0.9

0.3

0.9

0.9

0.9

0.9

0.9

0.9

0.9

DDE

0.9

0.9

0.9

0.9

0.9

0.9

0.6

0.9

0.9

0.9

0.9

0.9

0.9

0.9

7

best/2/bin

DE

0.1

0.3

0.2

0.1

0.5

0.7

0.7

0.1

0.1

0.6

0.4

0.1

0.1

0.1

DDE

0.1

0.3

0.1

0.1

0.5

0.9

0.0

0.1

0.1

0.6

0.3

0.1

0.1

0.3

8

best/2/exp

DE

0.9

0.9

0.9

0.9

0.9

0.9

0.3

0.9

0.9

0.9

0.9

0.9

0.9

0.9

DDE

0.9

0.9

0.9

0.9

0.9

0.9

0.4

0.9

0.9

0.9

0.9

0.9

0.9

0.9

9

current-to-rand/1/bin

DE

0.5

0.1

0.2

0.1

0.2

0.9

0.4

0.1

0.1

0.1

0.1

0.1

0.2

0.3

DDE

0.3

0.1

0.2

0.1

0.3

1.0

0.5

0.1

0.1

0.1

0.1

0.5

0.1

0.3

10

current-to-rand/1/exp

DE

0.9

0.9

0.9

0.9

0.9

0.9

0.3

0.9

0.9

0.9

0.9

0.9

0.9

0.9

DDE

0.9

0.9

1.0

0.9

0.9

1.0

0.0

0.9

0.9

1.0

0.9

0.9

0.9

0.9

11

current-to-best/1/bin

DE

0.2

0.1

0.2

0.3

0.2

0.9

0.8

0.1

0.1

0.1

0.1

0.2

0.2

0.1

DDE

0.2

0.1

0.2

0.3

0.2

1.0

0.0

0.1

0.1

0.1

0.1

0.2

0.2

0.1

12

current-to-best/1/exp

DE

0.9

0.9

0.9

0.9

0.9

0.9

0.2

0.9

0.9

0.9

0.9

0.9

0.9

0.9

DDE

0.9

0.9

1.0

0.9

0.9

1.0

0.3

0.9

0.9

0.9

0.9

0.9

0.9

0.9

13

rand-to-best/1/bin

DE

0.1

0.1

0.4

0.4

0.8

0.9

0.8

0.1

0.1

0.8

0.9

0.1

0.1

0.2

DDE

0.1

0.1

0.3

0.3

0.8

0.9

0.1

0.1

0.1

0.8

0.8

0.1

0.1

0.2

14

rand-to-best/1/exp

DE

0.9

0.9

0.9

0.9

0.9

0.9

0.4

0.9

0.9

0.9

0.9

0.9

0.9

0.9

DDE

0.9

0.9

0.9

0.9

0.9

0.9

0.3

0.9

0.9

0.9

0.9

0.9

0.9

0.9

5. Results and Discussion

The mean objective function value (MOV) achieved by each DE and DDE variants on the benchmark functions are presented in "table 4". The probability of convergence (Pc) [22,6] is calculated as ,where nc is total number of successful runs made by each variant for all the functions and nt is total number of runs, in our experiment nt = 14 x 100 = 1400. Pc value is calculated for each variant-function combination. This measure identifies variants having higher convergence capability to global optimum [10]. The Pc value measured for DE and DDE variants are shown in "table 5" along with number of successful runs.

The variants current-to-rand/1/exp and current-to-best/1/exp, in case of both DE and DDE, have not solved any of the unimodal functions. Although similar performance is observed for both DE and DDE, for unimodal nonseparable function f6 DDE variants displayed relatively better performance than DE counterparts. The case is reverse for multimodal non separable function f10. In case of other multimodal functions both DE and DDE variants have similar poor performance. However their binomial counterparts */current-to-rand/1/bin and */current-to-best/1/bin displayed better overall performance as is evident from the higher probability of convergence. Like in exponential variants case, for function f6, DDE variants displayed superior performance. Interestingly in case of function f12 while DE/current-to-rand/1/bin had 96 successful runs its DDE counterpart had none.

Table 4

MOV achieved by DE and DDE Variants for the test Functions f1 to f14

Sno

Variant

f1

f2

f3

f4

f5

f6

f7

f8

f9

f10

f11

f12

f13

f14

1

rand/1/bin

DE

0

0

0

0.02

0

0.07

0.13

0

0

21.99

0.09

0

0

0

DDE

0

0

0

0

0

0

0.15

0

0

17.2

0

0

0

0

2

rand/1/exp

DE

0

0

3.76

0

0.02

0.31

0.1

47.93

0

25.48

0.09

0.05

0

0

DDE

0

0

3.32

0

0.02

0.13

0.11

47.67

0

41.34

0

0.05

0

0

3

best/1/bin

DE

457.25

0.14

1.96

437.25

0.09

13.27

0

4.33

12.93

585899.88

3.58

3.72

3.52

973097.03

DDE

193.49

0.23

1.81

0.81

0.06

0

0

3.63

12.98

32747.01

3.52

2.64

1.06

15.16

4

best/1/exp

DE

583.79

4.05

37.36

591.85

0.06

57.39

0.01

50.74

32.18

64543.84

6.09

5.91

6.5

154434.94

DDE

199.3

3.34

9.4

207.83

0.05

0

4.46

52.06

18.39

449714.63

6.5

5.11

12.46

72512.56

5

rand/2/bin

DE

0

0

0.06

0

0.01

1.64

0.22

0

0

19.01

0.09

0

0

0

DDE

0

0

0.05

0

0.01

1.17

0.21

0

0

16.17

0

0

0

0

6

rand/2/exp

DE

0

0.02

32.9

0

0.05

269.86

0.27

101.38

0.01

2741.32

0.01

0.21

0

0.01

DDE

0

0.02

31.34

0.01

0.04

233.07

0.23

102.78

0.01

1877.42

0

0.19

0

0.01

7

best/2/bin

DE

0

0

0

0.07

0

0

0.17

0.69

0.12

2.32

0.09

0

0

0

DDE

0

0

0

0

0

0

0.1

0.65

0.05

1.87

0

0

0.01

0

8

best/2/exp

DE

0

0

0.05

0.39

0.01

0

0.08

80.63

2.53

1.12

0.83

0.03

1.04

0

DDE

0

0

0.04

0.05

0.01

0

0.07

79.78

2.99

0.78

1.04

0.03

0.13

0

9

current-to-rand/1/bin

DE

0

0.02

3.68

0.03

0.04

3210.36

0.14

37.75

0

52.81

0.01

0

0.01

0

DDE

0

0.02

3.7

0

0.04

61.85

0.14

37.49

0

56.68

0.01

0.14

0

0

10

current-to-rand/1/exp

DE

24.29

44.22

57.52

43.07

0.27

3110.9

0.12

235.14

18.35

199243.32

13.83

1.21

13.74

24.11

DDE

23.18

45.12

52.1

40.41

0.25

62.17

0.1

232.59

18.1

254655.74

13.74

1.21

10.55

23.96

11

current-to-best/1/bin

DE

0

0.02

3.71

0

0.04

3444

0.19

37.04

0

56.91

0.01

0

0.01

0

DDE

0

0.02

3.77

0

0.04

61.54

0.11

38.26

0

56.01

0.01

0

0

0

12

current-to-best/1/exp

DE

24.37

45.04

56.67

41.95

0.26

2972.62

0.1

232.8

18.21

119685.68

13.69

1.21

13.68

23.04

DDE

24.59

44.98

51.63

43.96

0.27

63.7

0.12

232.94

18.5

271844.82

13.68

1.22

10.39

22.13

13

rand-to-best/1/bin

DE

0

0

0

0

0

0.07

0.22

0

0

17.37

0.09

0

0

0

DDE

0

0

0

0

0

0.01

0.13

0

0

15.92

0

0

0

0

14

rand-to-best/1/exp

DE

0

0

3.38

0

0.01

0.2

0.12

48.09

0

24.54

0.09

0.05

0

0

DDE

0

0

3.48

0

0.01

0.14

0.09

47.4

0

32.52

0

0.06

0

0

The */best/1/* variants had low probability of convergence when compared with */current-to-*/bin variants. As can be seen from "table 4", */best/1/* variants performed poorly on multimodal functions. In fact the mean objective function values obtained by DDE/best/1/* variants in most of the cases, is relatively lower than that of their DE counterparts. Except for functions f3 and f5, in all other cases DDE/best/1/* variants have relatively higher number of successful runs.

Table 5

Number of Successful Runs and Pc(%) for the DE and DDE Variants

Sno

Variant

f1

f2

f3

f4

f5

f6

f7

f8

f9

f10

f11

f12

f13

f14

nc

Pc %

1

rand/1/bin

DE

100

100

100

98

60

73

4

100

100

0

0

100

100

100

1035

73.93

DDE

100

100

100

100

63

85

1

100

100

2

100

100

100

100

1151

82.21

2

rand/1/exp

DE

100

100

0

100

0

4

7

0

100

0

0

68

100

100

679

48.5

DDE

100

100

0

100

0

6

6

0

100

0

100

69

100

100

781

55.79

3

best/1/bin

DE

3

40

79

0

22

86

88

3

0

0

0

1

0

0

322

23

DDE

9

58

0

99

0

99

80

3

3

0

4

0

2

7

364

26

4

best/1/exp

DE

0

0

0

0

0

58

85

0

0

0

0

0

0

0

143

10.21

DDE

21

0

0

11

0

100

78

0

0

0

0

0

10

5

225

16.07

5

rand/2/bin

DE

100

100

0

100

2

0

1

100

100

0

0

100

100

100

803

57.36

DDE

100

100

0

100

4

0

2

100

100

0

100

100

100

100

906

64.71

6

rand/2/exp

DE

61

0

0

100

0

0

2

0

26

0

64

3

100

50

406

29

DDE

85

0

0

99

0

0

2

0

33

0

83

6

100

61

469

33.5

7

best/2/bin

DE

100

100

100

95

75

100

1

47

89

38

0

100

99

100

1044

74.57

DDE

100

100

100

100

63

100

11

49

95

48

100

96

98

99

1159

82.79

8

best/2/exp

DE

100

100

1

69

2

100

17

0

7

29

0

44

69

69

607

43.36

DDE

100

100

2

95

2

100

14

0

1

58

22

44

77

70

685

48.93

9

current-to-rand/1/bin

DE

100

0

0

100

0

0

2

0

100

0

56

96

96

100

650

46.43

DDE

100

0

0

100

0

0

5

0

100

0

82

0

100

100

587

41.93

10

current-to-rand/1/exp

DE

0

0

0

0

0

0

3

0

0

0

0

0

0

0

3

0.21

DDE

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

11

current-to-best/1/bin

DE

100

0

0

100

0

0

3

0

100

0

56

96

91

100

646

46.14

DDE

100

0

0

100

0

0

6

0

100

0

74

91

93

100

664

47.43

12

current-to-best/1/exp

DE

0

0

0

0

0

0

5

0

0

0

0

0

0

0

5

0.36

DDE

0

0

0

0

0

0

7

0

0

0

0

0

0

0

7

0.5

13

rand-to-best/1/bin

DE

100

100

100

100

60

79

0

100

100

0

0

100

100

100

1039

74.21

DDE

100

100

100

100

55

84

6

100

100

0

100

100

100

100

1145

81.79

14

rand-to-best/1/exp

DE

100

0

0

100

0

10

6

0

100

0

0

69

100

100

585

41.79

DDE

100

100

0

100

0

6

5

0

100

0

100

59

100

100

770

55

The variants */best/2/bin and */best/2/exp (which differ from previous pair by the number of perturbing difference vectors) have displayed higher probability of convergence than the previously described variants. The variants solved all unimodal and most of the multimodal functions. The relative better performance of DDE/best/2/* variants can be seen by comparing their mean objective function values, number of successful runs and hence probability of convergence.

The variants */rand/1/* displayed similar capability for high probability of convergence as */best/2/* variants. The */rand/1/bin variants performed well against */rand/1/exp variants. Between DDE and DE, the former displayed relatively better performance than the latter. The variants */rand/2/* displayed competitive convergence probability. In fact, like observed before the variants with binomial crossover ie., */rand/2/bin performed well against */rand/2/exp variants.

The convergence probability of */rand-to-best/* variants match those of */rand/1/* and */best/2/* variants. The better performance of variants with binomial crossover over those with exponential crossover and DDE variants over DE variants can once again be observed.

In case of both DE and DDE the best and worst performance was given by the similar set of variants, for most of the benchmark functions, among which DDE variants outperforms their counterpart DE variants in terms of both MOV and Pc%. The obtained results also show that the exponential variants are outperformed by their corresponding binomial variants.

The fig. 3, to fig. 5, compares the overall performances of all DE and DDE variants by plotting empirical distribution of normalized success performance [6, 19, 23]. The success performance (SP) has been calculated as follows. (1)

If the variant reaches the global optimum before the maximum number of function evaluations, that run is regarded as successful run. The SP values of all DE and DDE variants on each benchmark function are calculated and are normalized by dividing them by the best SP on the respective function. The variants with small values of SP and large values of empirical distribution will reach the top of the graph earlier. The first variant which reaches the top is regarded as best variant. For the sake of clarity, the variants are plotted in three groups.

As can be seen from the first row (a-c) of fig. 3, DE/rand/1/bin, DE/best/2/bin and DE/rand-to-best/1/bin (closely followed by best/2/exp) have displayed overall superior performance. It is worth noting that none of the DE variants reached top of the graph because none of them solved all the benchmark functions even once. However, as is shown in the second row (d-f) of fig. 3, DDE/rand/1/bin and DDE/best/2/bin reached top as they solved all the benchmark problems at least once. It is worth noting that both variants reached 0.9 earlier than their DE counterparts. Except DDE/current-to-rand/1/bin, other DDE variants displayed in fig. 3 (e-f), either achieved similar performance as of their DE counterparts earlier or outperformed their DE counterparts.

(a) (b) (c)

(d) (e) (f)

Figure 3. Empirical distribution of normalized success performance of DE (a-c) and DDE (d-f) variants on all fourteen functions

It is worth noting that fig. 3 (a-c), do not display the variants DE/best/1/exp, DE/current-to-rand/1/exp and DE/current-to-best/1/exp due to their poor overall performance (so that the scale involved in the graph could not show them). However fig. 3 (d-f), could display DDE/best/1/exp and DDE/current-to-best/1/exp due to their better performance over their DE counterparts. Thus the superiority of DDE over DE is much more evident in case of worst performing variants.

Fig. 4 (a-c) and (d-f) displays respectively the performance of DE and DDE variants on unimodal functions. The variants */rand/1/bin , */best/2/bin, */rand-to-best/1/bin and */best/2/exp displayed superior performance by reaching top of the graph with DDE variants showing better performance against their DE counterparts. In case of */rand/1/exp, */best/1/bin, */rand/2/bin, */rand/2/exp, */rand-to-best/1/exp, */current-to-rand/1/bin and */current-to-best/1/bin variants once again DDE performed better, in most of the cases, than their DE counterparts. DDE/best/1/exp, due to its relative better performance, managed to show up in the graph unlike its DE counterpart. The variants */current-to-*/exp are not displayed due to their poor performance.

Fig. 5 (a-c) and (d-f) displays respectively the performance of DE and DDE variants on multimodal functions. While the variants */rand/1/bin, */rand-to-best/1/bin and */best/2/exp displayed good performance as usual, variants like */rand/2/*, */current-to-*/1/bin and */rand-to-best/1/exp have also displayed competitive performance. The performances of variants */rand/2/exp, */current-to-best/1/bin and */current-to-rand/1/bin on unimodal and multimodal test functions are contrary. While on the former the three variants displayed poor performance with respect to the other variants, on the latter they have displayed a very competitive performance. The variants DDE/best/1/exp and DDE/current-to-best/1/exp managed to appear in the graph unlike in DE.

(b) (c)

(d) (e) (f)

Figure 4. Empirical distribution of normalized success performance of DE (a-c) and DDE (d-f) variants on unimodal functions

(a) (b) (c)

(d) (e) (f)

Figure 5. Empirical distribution of normalized success performance of DE (a-c) and DDE (d-f) variants on multimodal functions

Three of the four variants identified above rely on the strategy of using the best solution found so far for perturbation. It is known that such strategy perform well while solving unimodal problems. Despite the fact that the variant with best solution(s) are often falls in local optimum while solving multimodal problems, the variants also have performed well on multimodal test functions. Interestingly, as can be seen from the graphs, the binomial variants have consistently outperformed exponential variants.

6. Conclusion

In this paper, we extended the dynamicity of Differential Evolution proposed for DE/rand/1/bin and DE/best/1/bin to five more variants DE/rand/2, DE/best/2, DE/current-to-rand/1, DE/current-to-best/1 and DE/rand-to-best/1. A comparative study on the performance efficacy of DDE variants against their DE variants is presented in this paper, on 14 test functions of dimension 30. The experiments identified */best/2/bin, */rand-to-best/1/bin, */rand/1/bin and (closely followed by) */best/2/exp as competitive variants. The worst performing variants were */current-to-best/1/exp, */current-to-rand/1/exp and */best/1/exp. The DDE variants were often found to have outperformed DE variants and consistently showed better performance. It was also observed that variants with binomial crossover showed a better performance, in most of the cases, over that of the variants with exponential crossover.