• Home
• Bibliographies and Resources
• Competition Results
• Median and Best Results
• Dates and News
  
• People
• Challenge Description
  
• Challenge Rules
  
• Scoring System
• ASAP Default Hyper-heuristics
• Leaderboard
• HyFlex Description
  
• HyFlex Download
  
• HyFlex Tutorial
  
• Benchmarking
• Documentation
• Registration and Submission
  

Organising Committee

Point Scoring System

The Formula 1 point system, used before 2010, has the following structure. The top eight drivers score 10, 8, 6, 5, 4, 3, 2 and 1 point respectively. We plan to use this structure for the cross-domain challenge (note that we may need to consider alternative schemes according to the final number of competitors). For each race, the top 8 ranking algorithms will receive the points indicated above, and the remaining algorithms will receive no points. These points are added for all the events. Therefore, if for example, 5 problem domains are considered with 5 instances each, the maximum possible score is 250 points.

Handling Ties

Under this system, and following the Formula 1 rules, ties are handled as follows. If the best solutions found by two or more competing algorithms, on a given instance, have the same objective function value (with a precision of 10^-6), then the corresponding points to the relevant positions are added together and shared equally for all those algorithms who tie. By using this system for ties, it is guaranteed that each event yields the same total number of points. The winner of the cross-domain challenge is the algorithm with the most points. If the number of points is the same, priority is given to the algorithm with more wins (times ranking first among all competing algorithms). If that is the same it will be decided on the most second places and so on.

Assessing the Performance of your Hyper-Heuristic

A program (ScoreCalculator.java) is available to allow competitors to asses the performance of their algorithms. This program simulates a competition  between your algorithm and 8 ASAP default hyper-heuristics that were developed during the preparation and testing of the competition software. We provide this test program ONLY as a rough benchmark. Although the hyper-heuristics implemented are inspired by state-of-the-art approaches, we make no guarantees about their quality (or lack thereof). Success against these hyper-heuristics will not necessarily translate into success in the competition, as that depends on the quality of the other competitors' entries.

If you are willing to gauge the performance of your algorithm against those of other competitors, please consider sending your results to be included in the Leaderboard.