AABOH 2022

Analysing Algorithmic Behaviour of Optimisation Heuristics

Organized by: Anna V. Kononova, Hao Wang, Michael Emmerich, Peter A.N. Bosman, Daniela Zaharie, Fabio Caraffini, and Johann Dreo

Workshop as part of the Genetic and Evolutionary Computation Conference (GECCO 2022)
July 9-13, 2022 (Saturday - Wednesday), Boston, Massachusetts

Workshop date: July 9 or July 10 (TBD), 2022


1. Introduction

The Analysing Algorithmic Behaviour of Optimisation Heuristics Workshop (AABOH), as part of the 2022 Genetic and Evolutionary Computation Conference (GECCO'22), invited the submission of original and unpublished research papers. You can download the AABOH Special Session Call for Papers in PDF format and plain text file.

Optimisation and Machine Learning tools are among the most used tools in the modern world with its omnipresent computing devices. Yet, the dynamics of these tools have not been analysed in detail. Such scarcity of knowledge on the inner workings of heuristic methods is largely attributed to the complexity of the underlying processes that cannot be subjected to a complete theoretical analysis. However, this is also partially due to a superficial experimental set-up and, therefore, a superficial interpretation of numerical results. Indeed, researchers and practitioners typically only look at the final result produced by these methods. Meanwhile, the vast amount of information collected over the run(s) is wasted. In the light of such considerations, it is now becoming more evident that such information can be useful and that some design principles should be defined that allow for online or offline analysis of the processes taking place in the population and their dynamics.

Hence, with this workshop, we call for both theoretical and empirical achievements identifying the desired features of optimisation and machine learning algorithms, quantifying the importance of such features, spotting the presence of intrinsic structural biases and other undesired algorithmic flaws, studying the transitions in algorithmic behaviour in terms of convergence, any-time behaviour, performances, robustness, etc., with the goal of gathering the most recent advances to fill the aforementioned knowledge gap and disseminate the current state-of-the-art within the research community.

1. Topics of Interest

We encourage submissions exploiting carefully designed experiments or data-heavy approaches that can come to help in analysing primary algorithmic behaviours and modelling internal dynamics causing them. As an indication, some (but not all) relevant topics of interests are reported in the list below:

All accepted papers of this workshop will be included in the Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'22) Companion Volume.

2. Program

The AABOH workshop will take place on Sunday, July 10, 08:30-12:40 (EDT) and will consist of 2 sessions

Session 1: Contributed papers
Welcome Talk08:30
Survivor Selection in a Crossoverless Evolutionary Algorithm08:35
     Nielis Brouwer, Danny Dijkzeul, Levi Koppenhol, Iris Pijning, Daan van den Berg
Exactly characterizable parameter setings in a crossoverless evolutionary algorithm08:50
     Levi Koppenhol, Nielis Brouwer, Danny Dijkzeul, Iris Pijning, Joeri Sleegers, Daan van den Berg
Examining Algorithm Behavior using Recurrence Quantification and Landscape Analyses09:05
     Mario Munoz Acosta
The Effect of Decoding Fairness on Particle Median Problem09:20
     Pavel Kromer, Vojtech Uher
Dynamic Computational Resource Allocation for CFD Simulations Based on Pareto Front Optimization09:35
     Gašper Petelin, Margarita Antoniou, Gregor Papa
Using Structural Bias to Analyse the Behaviour of Modular CMA-ES09:50
     Diederick Vermetten, Fabio Caraffini, Bas van Stein, Anna Kononova
Session 2: Theoretical and Empirical Analysis of Optimisation Heuristics
Opening Talk10:50
Invited Talk10:55
     Benjamin Doerr, École Polytechnique, Palaiseau, France
    Title: Don't Implement, Think!
It is clear that the vast majority of algorithm analyses is experimental. In this talk, I shall argue that mathematical analyses of algorithms have a few undeniable advantages. I shall then argue that theory vs. experiments is not an exclusive-or. Rather, you should use your mathematical skills to understand as much as possible via theoretical means (allowing any degree of imprecision or unproven assumptions that is necessary to survive). Then, and only then, is the time to design a meaningful experiment that does not just blindly collect a mass of ununderstandable data, but that confirms (or disproves) the theories obtained before. Of course you should implement, but backed up by as much theory as possible!
Invited Talk11:40
     Thomas Bartz-Beielstein, TH Koeln, Institute for Data Science, Engineering, and Analytics, Germany
    Title: Hyperparameter Tuning of Deep Neural Networks
A surrogate model based Hyperparameter Tuning (HPT) approach for Deep Learning (DL) is presented. We will demonstrate how the architecture- level parameters (hyperparameters) of Deep Neural Networks (DNNs) that were implemented in keras/tensorflow can be optimized. The implementation of the tuning procedure is 100% accessible from R, the software environment for statistical computing. The performances of six Machine Learning (ML) methods (k-Nearest-Neighbor (KNN), Elastic Net (EN), Decision Tree (DT), Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Support Vector Machine (SVM)) are compared to the results from the DNN. The R package SPOT is used as a "datascope" to analyze the results from the HPT runs from several perspectives: in addition to Classification and Regression Trees (CART), the analysis combines results from surface, sensitivity and parallel plots with a classical regression analysis. This study provides valuable insights in the tunability of several ML and DL methods, which is of great importance for the AI practitioner. This keynote presents results from the forthcoming book "Hyperparameter Tuning for Machine and Deep Learning with R", which is edited by Eva Bartz, Thomas Bartz-Beielstein, Martin Zaefferer, and Olaf Mersmann that will be published by Springer.
Panel Discussion12:25
Closing Remarks12:35

3. Important Dates

If you have questions or suggestions, please contact Dr. Kononova at This email address is being protected from spambots. You need JavaScript enabled to view it. and Dr. Wang at This email address is being protected from spambots. You need JavaScript enabled to view it..

4. Instructions for Authors

Our workshop invites the submission of papers of at most 8 pages (excluding references), which should present original work that meets the high-quality standards of GECCO. Accepted papers appear in the ACM digital library as part of the Companion Proceedings of GECCO.

Each paper submitted to our workshop will be rigorously evaluated in a double-blind review process. Review criteria include the significance of the work, technical soundness, novelty, clarity, writing quality, relevance and, if applicable, sufficiency of information to permit replication.

Each paper accepted needs to have at least one author registered by the author registration deadline. By submitting a paper you agree to register and present at the conference in case the paper is accepted.

Papers must be submitted via the online submission system: https://ssl.linklings.net/conferences/gecco/.

As a published ACM author, you and your co-authors are subject to all ACM Publications Policies (https://www.acm.org/publications/policies/toc), including ACM's new Publications Policy on Research Involving Human Participants and Subjects (https://www.acm.org/publications/policies/research-involving-human-participants-and-subjects).

Please refer to https://gecco-2022.sigevo.org/Paper-Submission-Instructions for more detailed instructions.

5. Organizers

6. Program Committee

7. Chair Biographies

Portrait of Anna V. Kononova

Dr. Anna V. Kononova is currently an Assistant Professor at the Leiden Institute of Advanced Computer Science. She received her MSc degree in Applied Mathematics from Yaroslavl State University (Russia) in 2004 and her PhD degree in Computer Science from University of Leeds in 2010. After 5 years of postdoctoral experience at Technical University Eindhoven, the Netherlands and Heriot-Watt University, UK, Anna has spent a number of years working as a mathematician in industry. Her current research interests include analysis of optimization algorithms.

Portrait of Hao Wang

Dr. Hao Wang obtained his PhD (cum laude, promotor: Prof. Thomas Bäck) from Leiden University in 2018. He is currently employed as an assistant professor of computer science in Leiden University. Previously, he has a research stay at Sorbonne University, France (supervised by Carola Doerr). He received the Best Paper Award at the PPSN 2016 conference and was a best paper award finalist at the IEEE SMC 2017 conference. His research interests are in the analysis and improvement of efficient global optimization for mixed-continuous search spaces, Evolution strategies, Bayesian optimization, and benchmarking.

Portrait of Michael Emmerich

Dr. Michael Emmerich received his Dr.rer.nat. degree from Dortmund University (Prof. H.-P. Schwefel, Prof. P. Buchholz Promoters) in 2005. He is currently an Associate Professor (UHD) with LIACS, Leiden University, and the head of the Multicriteria Optimization and Decision Analysis Research Group (moda.liacs.nl) and Scientific Coordinator of Center for Computational Life Science (CCLS), Leiden University. Moreover, he is a visiting research fellow at the Multiobjective Optimization research group at Jyv¨akyl¨a University, Finland. He is known for pioneering work on model-assisted and indicator-based multiobjective optimization (SMS-EMOA, Expected Hypervolume Improvement, Set-Oriented Newton Method) and on theory of subset selection problems, set-oriented integration/differentiation of qualtity indicators, and multimodal multiobjective optimization. He has edited four books and co-authored over 120 papers in multicriteria optimization algorithms and their application in drug discovery, logistics, complex networks, and sustainable building design.

Portrait of Peter A.N. Bosman

Prof. Dr. Peter A.N. Bosman is a senior researcher in the Life Sciences research group at the Centrum Wiskunde & Informatica (CWI) (Centre for Mathematics and Computer Science) located in Amsterdam, the Netherlands. Peter obtained both his MSc and PhD degrees on the design and application of estimation-of-distribution algorithms (EDAs). He has (co-)authored over 150 refereed publications on both algorithmic design aspects and real-world applications of evolutionary algorithms. At the GECCO conference, Peter has previously been track (co-)chair, late-breaking-papers chair, (co-)workshop organizer, (co-)local chair (2013) and general chair (2017).

Portrait of Daniela Zaharie

Prof. Dr. Daniela Zaharie is a Professor at the Department of Computer Science from the West University of Timișoara (Romania) with a PhD degree on a topic related to stochastic modelling of neural networks and an Habilitation thesis on the analysis of the behaviour of differential evolution algorithms. Her current research interests include analysis and applications of metaheuristic algorithms, interpretable machine learning models and data mining.

Portrait of Fabio Caraffini

Dr. Fabio Caraffini is currently an Associate Professor in Computer Science and Mathematics at De Montfort University (UK) and a Fellow of the Higher Education Academy (UK). He received the BSc in "Electronics Engineering" and the MSc in "Telecommunications Engineering" degrees from the University of Perugia (Italy) in 2008 and 2011 respectively. Fabio holds a PhD in Computer Science (De Montfort University, UK, 2014) and a PhD in Computational Mathematics (University of Jyväskylä, Finland, 2016). His research interests include theoretical and applied computational intelligence with a strong emphasis on metaheuristics for optimization.

Portrait of Johann Dreo

Dr. Johann Dreo works at the Pasteur Institute and CNRS, Computational Biology Departement, USR 3756, System Biology Group. His scientific interests are in optimization, search heuristics, artificial intelligence, machine learning, algorithm design and engineering, automated planning, and differential geometry. He has more than 13 years of expertise of applying randomized optimization heuristics in practice.