heuristic

A heuristic is a system that will usually provide a shortcut to a correct answer. Opposed to algorithms, which will always reach a correct answer, but often take longer.

To illustrate, an algorithm for solving an anagram could test every possible combination of letters (it could also ignore letter combinations that never occur in English). This could take a long time. A heuristic user would would cut out letter combinations that almost never occur, and might look for certain combinations first. They might also take contextual factors into account.

Humans nearly always use heuristics. You will solve the anagram YBO without sorting through all six possibilities. You can probably use a simple heuristic on th2yEnrveig to cut through over 30 million possibilities in a matter of seconds. On the other hand, you might be slowed down by one of your heuristics with IPMHYLOD, as there is an unusual letter combination in the word.

Most math is done by algorithms; for example, adding the digits of a number to see if they are divisible by nine is an algorithm, as it will always work.

In a computer science context, a heuristic is a method to reach a result that has not been proven to be correct, or to always give optimal results (otherwise it would be an algorithm). Heuristics are used because in many cases, the algorithms that do the same task are much slower (especially for NP complete problems), and getting an occasional wrong result, or a not quite optimal one is better than having to wait forever for any result at all. A common heuristic is the usage of pseudoprime numbers in lieu of true primes in many cryptograhic programs like PGP and SSH.

In the world of economics, which is the context in which I have studied philosophy of science, a very important heuristic is "Don't ask people what they want, watch what they do". To ask people what they want is seen as unscientific, as their replies will be structured by what is seen as socially respectable. The punishment for breaking negative heuristics is to be thrown out of the scientific community, socially ostracised, in order to protect the Scientific Research Programme from your pseudoscience. The reward for actively using positive heuristics, theorising and doing experiments, is to achieve greater social status and gain the respect of the scientific community.

Imre Lakatos’s contribution was to get away from the idea that the rules governing progress are inherently logical – heuristics are often ideologically determined, practical tools which separate different research programmes. The measure by which they can be judged is not their philosophical or scientific validity, but the success or failure of the scientific research programme as a whole.

The Pitfalls of Heuristics*
Heuristic methods are often employed because they may be seen to "work" without having been mathematically proven to meet a given set of requirements.

Great care must be given when employing a heuristic method. One common pitfall in implementing a heuristic method to meet a requirement comes when the engineer or designer fails to realize that the current data set doesn't necessarily represent future system states.

While the existing data can be pored over and an algorithm can be devised to successfully handle the current data, it is imperative to ensure that the heuristic method employed is capable of handling future data sets. This means that the engineer or designer must fully understand the rules that generate the data and develop the method to meet those requirements and not just address the current data sets.

A simple example of how heuristics can fail is to answer the question "What is the next number in this sequence: 1, 2, 4?". One heuristic algorithm might say that the next number is 8 because the numbers are doubling - leading to a sequence like 1, 2, 4, 8, 16, 32... Another, equally valid, heuristic would say that the next number is 7 because each number is being raised by one higher interval than the one before - leading to a series that looks like 1, 2, 4, 7, 11, 16...

Statistical analysis must be conducted when employing heuristics to ensure that enough data points are utilized to make the likelihood of incorrect results statistically insignificant.