Towards better estimation of statistical significance when comparing evolutionary algorithms

Research output: Chapter in Book/Report/Conference proceedingConference Proceeding (Non-Journal item)

Abstract

The use of well-established statistical testing procedures to compare the performance of evolutionary algorithms often yields pessimistic results. This requires increasing the number of independent samples, and thus the computation time, in order to get results with the necessary precision.

We aim at improving this situation by developing statistical tests that are good in answering typical questions coming from benchmarking of evolutionary algorithms. Our first step, presented in this paper, is a procedure that determines whether the performance distributions of two given algorithms are identical for each of the benchmarks. Our experimental study shows that this procedure is able to spot very small differences in the performance of algorithms while requiring computational budgets which are by an order of magnitude smaller (e.g. 15x) compared to the existing approaches.
Original languageEnglish
Title of host publicationGECCO '19
Subtitle of host publicationProceedings of the Genetic and Evolutionary Computation Conference Companion
EditorsManuel López-Ibáñez
PublisherAssociation for Computing Machinery
Pages1782-1788
Number of pages7
ISBN (Print)978-1-4503-6748-6
DOIs
Publication statusPublished - 13 Jul 2019
Externally publishedYes
EventGECCO 2019: The Genetic and Evolutionary Computation Conference - Prague, Czech Republic
Duration: 13 Jul 201917 Jul 2019
https://gecco-2019.sigevo.org

Conference

ConferenceGECCO 2019: The Genetic and Evolutionary Computation Conference
Country/TerritoryCzech Republic
CityPrague
Period13 Jul 201917 Jul 2019
Internet address

Keywords

  • multiple comparisons
  • statistical significance

Fingerprint

Dive into the research topics of 'Towards better estimation of statistical significance when comparing evolutionary algorithms'. Together they form a unique fingerprint.

Cite this