Software is delicate. Change a single bit in the wrong place and the whole thing crashes, sometimes with great violence. A program that modifies itself while running is such a risky enterprise that virtually all computer science wisdom recommends against it. Code, meaning software, is infinitely fragile.

So imagine how dangerous it would be for an animal to have access to its own code! Suppose people could decide to rewire their brain circuitry, to make improvements (or for a spasm of pleasure) ... and suppose that the mere act of thinking about it made it so? The human race would be extinct in no time. (Who among us would have survived adolescence?) Even the wisest of the wise would hesitate before doing irrevocable mods. "Measure twice, cut once" would seem utter recklessness; one would test and evaluate proposed mental changes thousands of ways to make sure there were no hidden side effects. Even then, the transition to a new state would be terrifying to undertake.

But stranger still: we can, and we do, reprogram our minds daily. We modify our code when we learn, particularly when picking up new skills or methods of problem-solving. Thank goodness (or Darwin), we're denied access to the depths of our own operating system. We can't flip bits or connect and disconnect neurons at will, lest we trigger hyper-epileptic seizures and suffer instant death. But we're constantly adjusting high-level priorities and procedures in our mental subroutines. We choose to watch TV, or read a newspaper, or practice the piano, or study Sanskrit ... and our choices act like flowing water, cutting channels, leaving our minds with increased or decreased potentials for further changes.

Some decisions leave us where we are—or worse, box us in until few exits remain. When we choose well, new doors open.

Also, a few computer programs do try to modify themselves while they run. It is a rarely performed method, and usually when it is done, the problem which it is applied to is not a interesting or important one. The value that such an approach has to A.I. programming is clear from the above, but I don't know if there are any well-known A.I. programs that try it.