Tuesday, February 25, 2014

Artificial Intelligence and the Bible: Sensory Learning in Smyrna, Part II

Abstract
In Part I, I wrote that, according to my interpretation of the message to the church of Smyrna in the Book of Revelation, the brain uses two types types of sensors: rich and poor. I explained that it takes an onset sensor and an offset sensor to properly represent a single sensory phenomenon or stimulus at a given amplitude. In today's post, I interpret verses 10 and 11 of the message to Smyrna which describe how sensory learning works. But first, a word about the importance of sensory timing.

The Timing of Sensory Signals in the Brain

Why is it so important that there be two complementary sensors for a stimulus? The reason is that perception is primarily concerned with the evolution of events, i.e., with how things change relative to one another. Phenomena come and go at precise times.

Certain changes happen concurrently and these are called patterns. Patterns succeed each other to form precisely timed sequences. For example, sensors A and B in the diagram above will sometimes fire concurrently with other sensors. Knowing when this happens is valuable information. Sensory learning consists of capturing the temporal correlations in the sensory space and this allows the brain to gain an understanding of how changes occur in the environment. With a good temporal model of the world, an intelligent system can form predictions, plan future actions, adapt to changes and achieve various goals. This is what intelligence is about.

Message to the Church of Smyrna

Revelation 2:8-118 “And to the angel of the church in Smyrna write, ‘These things says the First and the Last, who was dead, and came to life:
9 “I know your works, tribulation, and poverty (but you are rich); and I know the blasphemy of those who say they are Jews and are not, but are a synagogue of Satan.
10 Do not fear any of those things which you are about to suffer. Indeed, the devil is about to throw some of you into prison, that you may be tested, and you will have tribulation ten days. Be faithful until death, and I will give you the crown of life.
11 “He who has an ear, let him hear what the Spirit says to the churches. He who overcomes shall not be hurt by the second death.”’

Commentary (continued)10 Do not fear any of those things which you are about to suffer. Indeed, the devil is about to throw some of you into prison, that you may be tested, and you will have tribulation ten days. Be faithful until death, and I will give you the crown of life.

Sensory learning is a relentless and uncompromising trial. The phrase "some of you" means that new connections for patterns are chosen randomly and then subjected to a brutal trial period during which they are tested 10 times. As I will explain in a future article, a day symbolizes a single neuronal or firing cycle, which is the duration of a single pulse (about 10 milliseconds in the brain). In other words, there are 10 tests and each one lasts a single firing cycle. From this, we can logically deduce that every connection is tested for concurrency with other connections. Why? Because concurrency is the only thing that can be tested during the time of a single pulse.

One of the important things to note here is that connections either survive or they don't. The ones that fail are put to death, that is, they are disconnected. The "crown of life" means that disconnected synapses are reborn and tried again elsewhere. The "prison" metaphor symbolizes the fact that connections are not allowed to "earn a living" during the time of their trial. In other words, the connections cannot contribute to their churches (or patterns) until they pass all the 10 tests and are released from prison.

It goes without saying that the Biblical model sharply contradicts current neural network models that use fixed, pre-wired connections. Also, the Biblical model strongly suggests that synaptic learning is an either-or process: either a connection is made or it isn't. There are no in-betweens, i.e., there is no need for a range of connection weights to encode knowledge. This is why I maintain that deep learning will go the way of symbolic AI and that the high-tech industry is building a billion dollar AI castle in the air.

Finally, we must ask, why 10 test cycles? Why not 5 or 20? To answer this question, we must understand what exactly is being learned. The brain is looking for all possible patterns that occur often enough to be considered non-random. It does not care about their actual probabilities of occurrence because it uses a winner-takes-all mechanism whereby patterns and sequences in memory compete for activation: the ones with the greatest number of hits are the winners. A compromise must be reached between conducting too many tests, which would retard learning and miss low probability patterns, and not conducting enough tests, which would result in learning useless patterns. We can surmise that 10 is just an optimum number. On a side note, this would be a fairly easy hypothesis to falsify. The finding that sensory learning in the brain is based on a mechanism that counts to 10 would go a long way to corroborate this theory.

Note: I am still working on the Rebel Speech demo program and I hope to release it soon. I will also release the source code for the recognizer but not the learner. Rebel Speech incorporates all the principles I have described in this series on AI and the Bible.

11 “He who has an ear, let him hear what the Spirit says to the churches. He who overcomes shall not be hurt by the second death.”’

That first sentence in verse 11 is repeated in every message to the seven churches. It is a sign that the messages do not mean what they appear to mean on the surface. What is the meaning of the "second death" metaphor? I am not 100% sure at this point but it seems to mean that, once a connection is established through testing, it becomes permanent. In other words, unlike sequences which can be forgotten, patterns are retained forever. Note that I am still investigating this metaphor because it is mentioned elsewhere in the book of Revelation.

2 comments:

Thanks, Arcadia. Hawkins is light years ahead of all his critics, especially those in academia. In my opinion, Hawkins is the smartest mind in the AI business, bar none and he is closer to having a correct understanding of intelligence and the brain than anybody else. He was right to part ways with the math fanatic, Dileep George. He's right that intelligence is not about math. I use nothing more complicated in my neural network code than addition or multiplication.

However, I think Hawkins is crippled by his continued allegiance to a statistical approach to the problem. The allure of statistics is a powerful temptation but I am convinced it's one that leads to perdition. Nevertheless, I wish Hawkins the best because I like his passion for understanding the universe.