Metaheuristics and metaphors

Comments (0)

Transcript of Metaheuristics and metaphors

MetaheuristicsKenneth Sörensenkenneth.sorensen@ua.ac.beWhat is a metaheuristic?A metaheuristic is a high-level problem-independent

algorithmic framework that provides a set of guidelines or strategies to develop heuristic optimization algorithms.Kenneth Sörensen and Fred Glover, Metaheuristics, Encyclopedia of Operations Research, Springer, Berlin, 2012 (to appear)Fred GloverUniversity of Colorado & OptTekHundreds of publicationsh-index > 50Founding father of the field of metaheuristicsTabu search, scatter search, path relinking, etc.OptimizationDecision variablesObjective functionConstraintsThe term is also used to refer to a problem-specific

implementation of a heuristic optimization algorithm according to the guidelines expressed in such a framework.2^6 possible solutionsFirst, a simple heuristicOopsMaybe we need a metaheuristicTabu searchUse memory structures to guide the searchTabu list (short-term)Frequency memoryElite memoryAspirationEtcEvolutionary algorithmsuse evolution as a metaphor for optimizationA whole new vocabulary!Solution - chromosomeObjective function - fitnessPopulationGenotypical - phenotypical encodingCrossover, mutation, selection, ...EtcAnt colony optimizationmodel optimization as the behavior of antsModels of nature that are relied upon for inspiration [in creating new metaheuristics] are ubiquitous, and it is easy to conjure up examples whose metaphorical possibilities have not yet been tapped. To take an excursion in the lighter side of such possibilities (though not too far from the lanes currently traveled), we may observe that a beehive offers a notable example of a system that possesses problem solving abilities. Bees produce hives of exceptional quality and complexity, coordinate diverse tasks among different types of individuals, perform spatial navigation, and communicate via multiple media. (It is perhaps surprising in retrospect that the behavior of bees has not been selected as a basis for one of the “new” problem solving methods.)Fred Glover, 1997Marco DorigoMarco Dorigo, Optimization, learning and natural algorithms, Ph. D. Thesis, Politecnico di Milano, Italy, 1992The Reincarnation Algorithmsubmitted to Journal of Heuristics in 2011“RA utilizes the techniques of local and population based search to cover the wide search space in a limited time frame. The system basically works with two sets of human population. One is salient population (called gurus) with higher fitness values and another is common population (called commoners) with low fitness values. The entire population of humans is dispersed evenly into many small subsets of communities. Humans are bound to perform karma or deeds that can upgrade or degrade their souls depending on the types of karma (either good or bad).Population of commoners gets influenced by the population of their own community guru(s) and their closed ones to search locally for better fitness value. The best guru i.e. the guru with the best fitness value obtained so far for the entire population, influences gurus as well as commoners of all the communities. This algorithm also considers atheists in the same population who are not influenced by anyone but through their own self realization.The search for better fitness value is termed as karma. RA uses karma and soul to handle input and output/fitness values respectively. The cycle of rebirth continues for many generations where karma of previous generation is carried forward to future generations through souls. Karma as discrete search space is heuristically updated in every generation based on reincarnation analogy to search for optimum solution.”Intelligent Water Drops algorithm (IWD) is a swarm-based nature-inspired optimization algorithm, which has been inspired by natural rivers and how they find almost optimal paths to their destination. These near optimal or optimal paths follow from actions and reactions occurring among the water drops and the water drops with their riverbeds. In the IWD algorithm, several artificial water drops cooperate to change their environment in such a way that the optimal path is revealed as the one with the lowest soil on its links. The solutions are incrementally constructed by the IWD algorithm. Consequently, the IWD algorithm is generally a constructive population-based optimization algorithm.Intelligent water drops?Sometimes the metaphor makes no senseNo new metaphors, pleaseFocus on the problemDo not develop a method without a problemNo black-box implementations of existing frameworksStudy the problem in detailKnow the literature on the problem and on related problemsStudy the relationship between methods from the literature and the problemUse the best parts from existing methods for your problems (e.g., matheuristics)A metaphor can serve as inspirationNever as justification!Different types of optimization methodsExact methodsHeuristic methodsGuarantee to find optimal solutionSlow - often only toy instancesNot very flexibleNo guarantee to find optimal solutionFast! = Real problems too!Flexible"Algorithms are conceived in analytic purity in the high citadels of academic research, heuristics are midwifed by expediency in the dark corners of the practitioner's lair ... and are accorded lower status."Fred Glover, 1977Metaphor-based metaheuristicsGet inspiration from some (natural) processMake a random moveIf it makes the solution better: accept itIf it makes the solution worse: accept with a certain probability, depending on the temperatureDeconstructionAlgorithm analysis & tuningFrankenstein methodsHow not to develop a metaheuristic1. Take your favorite method2. Add a component3. While not working to your satisfaction, go to 2Develop any method you like, but determine(a) the contribution of each component(b) the optimal value of each parameterAnalyze your methodDeconstruct your methodMake sure each component mattersFind the best parameter settingsUse statistics!Try to find out why and how your methods workUse the best parts from different sub-areas of optimization (e.g., matheuristics)Things should be as simple as possible, but no simplerMetaheuristicsMetaphorsBad designGood designWhat have we learned today?1. It's all about optimization (and nothing else)2. Methods should be designed according to accepted principles3. Methods should be analyzed, tested, and comparedAn example: tabu searchEvolutionary algorithmsNew methods?Water network design optimizationA field in which improvements are possibleMethods tested on the WDND optimization problemGenetic algorithmReal-coded genetic algorithmMemetic algorithmShuffled frog leaping algorithmCross entropy searchScatter searchDifferential evolutionTabu searchSimulated annealingImmune algorithmParticle swarm harmony searchAll are tested on two or three instancesAll achieve "excellent" performanceNone are better than a simple heuristic that can be described in two paragraphs.Use design principles from the literatureThere are not so many fundamentally different methodsSome tipsUse the standard optimization vocabularyStay within the existing taxonomyNo points are awarded for originality, only for performanceThere are more than enough "novel methods"It's all about optimization!Interesting parallels can be found in the literature(e.g., LNS is the constructive variant of ILS)The metaphor exposedMetaphors have been useful in exploring the possibilities of the research fieldHOWEVERMost "new" methods are just a repackaging of old ideasThe vocabulary is changed every timeSometimes the relationship between the algorithm and the metaphor is shallow or non-existentSometimes the metaphor doesn't even make senseFly algorithmSimilar to particle swarm optimization, but

x% of the flies change velocity because of other fliesy% of the "flies" appear in random positionsWhere is the relationship between the algorithm and reality?The metaphor exposed