Table 3 Wilcoxon and Vargha-Delaney statistical values from all search algorithms, considering all metrics on each scenario
Algorithm | Datasets | LowPrefs scenario | HighPrefs scenario | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
IBEA | MOCell | NSGA-II | Random | IBEA | MOCell | NSGA-II | Random | ||||||||||
WC | Â 12 | WC | Â 12 | WC | Â 12 | WC | Â 12 | WC | Â 12 | WC | Â 12 | WC | Â 12 | WC | Â 12 | ||
Generational distance (GD) | |||||||||||||||||
MOCell | Dataset-1 | 3.0E −10 | 1.000 | – | – | – | – | – | – | 1.3E −04 | 0.827 | – | – | – | – | – | – |
Dataset-2 | 3.0E −10 | 1.000 | – | – | – | – | – | – | 3.0E −04 | 0.814 | – | – | – | – | – | – | |
Dataset-3 | 3.0E −10 | 1.000 | – | – | – | – | – | – | 3.0E −10 | 1.000 | – | – | – | – | – | – | |
NSGA-II | Dataset-1 | 3.0E −10 | 1.000 | 0.001 | 0.211 | – | – | – | – | 6.0E −04 | 0.197 | 3.4E −07 | 0.084 | – | – | – | – |
Dataset-2 | 4.1E −10 | 0.996 | 0.042 | 0.284 | – | – | – | – | 0.435 | 0.652 | 0.083 | 0.301 | – | – | – | – | |
Dataset-3 | 3.0E −10 | 1.000 | 5.0E −10 | 0.000 | – | – | – | – | 3.0E −10 | 1.000 | 1.2E −08 | 0.042 | – | – | – | – | |
Random | Dataset-1 | 3.0E −10 | 1.000 | 3.0E −10 | 0.000 | 3.0E −10 | 1.000 | – | – | 3.0E −10 | 1.000 | 3.0E −10 | 0.172 | 3.0E −10 | 0.000 | – | – |
Dataset-2 | 3.0E −10 | 1.000 | 3.0E −10 | 0.000 | 3.0E −10 | 1.000 | – | – | 3.0E −10 | 1.000 | 3.0E −10 | 0.185 | 3.0E −10 | 1.000 | – | – | |
Dataset-3 | 3.0E −10 | 1.000 | 3.0E −10 | 0.000 | 3.0E −10 | 1.000 | – | – | 3.0E −10 | 1.000 | 3.0E −10 | 0.000 | 3.0E −10 | 1.000 | – | – | |
SPEA-II | Dataset-1 | 3.0E −10 | 1.000 | 4.3E −07 | 0.087 | 0.191 | 0.323 | 3.0E −10 | 0.000 | 2.2E −07 | 0.078 | 5.1E −09 | 0.032 | 0.011 | 0.254 | 3.0E −10 | 0.000 |
Dataset-2 | 6.1E −10 | 0.992 | 1.1E −04 | 0.168 | 1.000 | 0.406 | 3.0E −10 | 0.000 | 0.112 | 0.308 | 3.8E −06 | 0.117 | 6.4E −04 | 0.198 | 3.0E −10 | 0.000 | |
Dataset-3 | 3.0E −10 | 1.000 | 5.5E −10 | 0.006 | 1.000 | 0.592 | 3.0E −10 | 0.000 | 3.7E −10 | 0.997 | 2.6E −09 | 0.024 | 1.000 | 0.400 | 3.0E −10 | 0.000 | |
Spread (SP) | |||||||||||||||||
MOCell | Dataset-1 | 3.0E −10 | 0.000 | – | – | – | – | – | – | 3.3E −10 | 0.000 | – | – | – | – | – | – |
Dataset-2 | 3.0E −10 | 0.000 | – | – | – | – | – | – | 3.0E −10 | 0.000 | – | – | – | – | – | – | |
Dataset-3 | 1.000 | 0.555 | – | – | – | – | – | – | 0.117 | 0.690 | – | – | – | – | – | – | |
NSGA-II | Dataset-1 | 3.0E −10 | 0.000 | 2.0E −08 | 0.951 | – | – | – | – | 3.3E −10 | 0.000 | 1.000 | 0.614 | – | – | – | – |
Dataset-2 | 3.0E −10 | 0.000 | 6.7E −10 | 0.991 | – | – | – | – | 3.3E −10 | 0.000 | 3.0E −10 | 1.000 | – | – | – | – | |
Dataset-3 | 1.000 | 0.515 | 1.000 | 0.462 | – | – | – | – | 0.063 | 0.705 | 1.000 | 0.497 | – | – | – | – | |
Random | Dataset-1 | 3.0E −10 | 0.000 | 1.000 | 1.000 | 9.1E −07 | 0.097 | – | – | 3.3E −10 | 0.000 | 0.610 | 1.000 | 0.010 | 0.252 | – | – |
Dataset-2 | 3.0E −10 | 0.000 | 9.8E −07 | 1.000 | 3.0E −10 | 0.000 | – | – | 3.0E −10 | 0.000 | 5.9E −03 | 1.000 | 6.1E −10 | 0.007 | – | – | |
Dataset-3 | 1.3E −06 | 0.897 | 2.8E −04 | 0.444 | 2.9E −05 | 0.852 | – | – | 6.0E −06 | 0.875 | 2.5E −03 | 0.310 | 6.0E −04 | 0.802 | – | – | |
SPEA-II | Dataset-1 | 3.0E −10 | 0.000 | 1.000 | 0.393 | 8.9E −09 | 0.038 | 1.000 | 0.426 | 3.3E −10 | 0.000 | c 3.3E −10 | 0.000 | 3.3E −10 | 0.000 | 3.3E −10 | 0.000 |
Dataset-2 | 3.0E −10 | 0.000 | 0.900 | 0.627 | 6.5E −08 | 0.063 | 8.5E −08 | 0.933 | 3.3E −10 | 0.000 | 3.3E −10 | 0.000 | 3.3E −10 | 0.000 | 2.0E −09 | 0.021 | |
Dataset-3 | 1.3E −06 | 0.102 | 4.3E −07 | 0.087 | 8.4E −07 | 0.096 | 9.0E −10 | 0.012 | 1.5E −08 | 0.045 | 4.2E −09 | 0.030 | 5.6E −09 | 0.033 | 8.2E −10 | 0.011 | |
Hypervolume (HV) | |||||||||||||||||
MOCell | Dataset-1 | 3.3E −10 | 0.998 | – | – | – | – | – | – | 3.3E −10 | 0.998 | – | – | – | – | – | – |
Dataset-2 | 3.0E −10 | 1.000 | – | – | – | – | – | – | 3.0E −10 | 1.000 | – | – | – | – | – | – | |
Dataset-3 | 3.0E −10 | 0.000 | – | – | – | – | – | – | 3.0E −10 | 0.000 | – | – | – | – | – | – | |
NSGA-II | Dataset-1 | 3.0E −10 | 1.000 | 1.000 | 0.581 | – | – | – | – | 3.3E −10 | 1.000 | 7.0E −06 | 0.873 | – | – | – | – |
Dataset-2 | 3.0E −10 | 1.000 | 0.012 | 0.743 | – | – | – | – | 3.0E −10 | 1.000 | 1.000 | 0.607 | – | – | – | – | |
Dataset-3 | 6.5E −08 | 0.063 | 3.0E −10 | 1.000 | – | – | – | – | 8.2E −10 | 0.011 | 8.2E −10 | 0.988 | – | – | – | – | |
Random | Dataset-1 | 3.0E −10 | 0.000 | 3.0E −10 | 0.001 | 3.0E −10 | 0.000 | – | – | 3.3E −10 | 0.000 | 3.3E −10 | 0.001 | 3.3E −10 | 0.000 | – | – |
Dataset-2 | 3.0E −10 | 0.000 | 3.0E −10 | 0.000 | 3.0E −10 | 0.000 | – | – | 3.0E −10 | 0.000 | 3.0E −10 | 0.000 | 3.0E −10 | 0.000 | – | – | |
Dataset-3 | 3.0E −10 | 0.000 | 3.0E −10 | 1.000 | 3.0E −10 | 0.000 | – | – | 3.0E −10 | 0.000 | 3.0E −10 | 1.000 | 3.0E −10 | 0.000 | – | – | |
SPEA-II | Dataset-1 | 3.2E −09 | 0.973 | 4.7E −05 | 0.155 | 5.2E −06 | 0.122 | 3.0E −10 | 1.000 | 3.3E −10 | 1.000 | 0.013 | 0.742 | 0.484 | 0.351 | 3.3E −10 | 1.000 |
Dataset-2 | 3.7E −10 | 0.997 | 0.076 | 0.701 | 1.000 | 0.436 | 3.0E −10 | 1.000 | 3.0E −10 | 1.000 | 1.000 | 0.520 | 1.000 | 0.430 | 3.0E −10 | 1.000 | |
Dataset-3 | 2.4E −07 | 0.080 | 4.5E −10 | 0.995 | 0.820 | 0.368 | 3.0E −10 | 1.000 | 3.7E −10 | 0.002 | 8.2E −10 | 0.980 | 1.000 | 0.430 | 3.0E −10 | 1.000 | |