Extreme Computing

Main menu

Post navigation

NFL

How many problems can we solve? To answer this question the No Free Lunch Theorems is a good start .

To explain this idea I usually think to this example . A generalization of genetic algorithms can be the problem to find the maximum value of a fitness function , this function is to much large to inspect it in every point so we need to build a strategy to inspect only relevant parts of it. The startegy can be to follow a direction when we found an increasing value of the fitness function and make some jump randomly to solve the problem of local maximum . The idea of NFL is that for every strategy you can choose this can not be a good strategy for every problem , in this case can not be a good strategy for every fitness function . It is simple to understand becouse given a strategy is possible to construct adhoc fitness function such that for the specific strategy it become the worst possible to reach maximum in the fitness function.

This idea is extendible to every kind of general problem solver and what it means is that there is not a way to have an optimal problem solver , given a good problem solver for a set of problems there is a set of problems where the solver become worst.

It is interesting observing the previous example where there are genetic algorithms that given a fitness function are empirical optimal. How is this possible? Constructing a strategy for an optimal solver means to make hypothesis about characteristic of the possible fitness functions , for example the continuity of the function but the number of possible fitness functions violate these characteristics is incredible so the probability to have an optimal solver with such strategy should become very very small . Instead it work ! How is possible? I think this is possible becouse in the real world is not possible to have every fitness function but only a small subset .

For these reasons the NFL is a very good mathematical result but the world is biased , the space is not exponential and is possible to find a prior knowledge to construct not a mathematical-optimal strategy but a real-optimal strategy.