In the interest of space last time, I had to leave out an advanced topic on optimizing a "next best action" algorithm. Again, you can look at the full source we're discussing by just using the web browser's View Source on this page.

The optimization is known as alpha-beta pruning. In the code snippet below, you see that we break the j-loop that is scoring the response moves of a given move based on some condition involving the variables alpha and beta. Why does it make sense to stop looking at the competitive response moves for a given move? To see why, I've added the function declaration so we can discuss where the alpha value comes from and what it means.

Understanding alpha-beta pruning requires you to take a more global view of the recursion that is doing the evaluation. The alpha values passed into scoreMove() are the beta values from the calling level of the Minimax algorithm. It will help to keep at least the player's moves and the opponents responses in mind as we go through this.

Let's say that scoreMove() has been called to score a player's Kth move. Beforehand, moves 1 to K-1 will have been fully explored by depth-first recursion, including the opponents responses, the player's counter-responses, and so on. The alpha value received by scoreMove() for move K reflects the best fully explored "net" score for the player on moves 1 to K-1. Within scoreMove(), we first compute the raw benefit of the new move K, storing the result in moveScore. Now comes the alpha-beta pruning trick. The j-loop successively explores each opponent response move for the player's move K, and clearly the beta value takes on the value of the highest scoring response move that the opponent can make. The final score for move K is the raw benefit to the player of move Kminus the benefit beta that the opponent can realize in response.

Thought-provoking question: Do we really need to know the absolute best move that the opponent can make in response to the player's move K? Or do we just need to find an opponent move that is good enough that, when subtracted from the raw benefit of move K, proves that the player would be better off choosing the earlier move associated with the alpha value? Of course, the answer is that we only need a good enough opponent move, and this is why we break the j-loop when we find that move. If we were to continue the j-loop, all we do is unnecessary work that might (or might not) find an even better opponent response move that would make move K look like an even worse decision for the player. But there is no need to do this extra work. Once the expression "(moveScore-beta < alpha)" becomes true, we have proven that move K is less beneficial than one of the moves 1 to K-1.

From a practical standpoint, this optimization averages better than double the run-time performance of the "what-if" logic. Who doesn't want double, right? Well, this "what-if" analysis is a combinatorial explosion of analysis; to put that in perspective, you get less than one extra move of lookahead due to this optimization. Yet despite this dash of cold water about how much deeper you could take the "what if" logic due to alpha-beta pruning, it remains true that, for a given level of explorative depth, everybody wants the result twice as fast or more, so alpha-beta pruning is very handy.

This entry is for developers who want a good mental model for how a prescriptive analytics algorithm can simulate intelligent behavior. We'll focus on the intelligent behavior in the James Blog entry, since it is quite competitive with humans. Reminder: Just hit "view source" in your browser to get the code we're talking about here.

The first thing to note is that the domain of the intelligence is quite constrained and circumscribed relative to the full realm of human intellectual endeavor. This is what makes it computationally feasible to perform a "what if" analysis to "imagine" possible scenarios and determine a next best action. Here's roughly how it works. The computer's available next actions are examined and measured for their immediate benefit. Then, for each action, the response action of the opponent is measured for its immediate benefit to the opponent, and so on. Once the real benefit of each opponent move is tabulated, the value of the best opponent action is subtracted from the immediate value of a given computer move. The best computer move is determined as the highest value move resulting from the immediate benefit minus the score of the best opponent move.

One thing I like about the game Kalah is that it is really easy to explain the competitive algorithm, relative to harder games like Chess. In Chess, evaluating the immediate benefit of a move can be challenging, especially at the beginning of the game. It's not just about the value of the piece you take because many moves don't take pieces. The value of a move is often about gaining control over spaces of the board to limit the opponent's attack and defense options. But in Kalah, you get good intelligent game play from a much simpler board evaluation. The value of a move is simply a matter of how many seeds you gain by that move.

This code (at the beginning of KalahGame.scoreMove) just copies the current board, makes the proposed move for the given player, then evaluates the new board value minus the value of the old board configuration for the given player. In effect, you get the number of seeds gained for the player by the move.

That's when things get interesting. The move scoring then becomes iteratively recursive. Each valid move of the opponent is then evaluated by recursively calling the move scoring method. Like this:

The first line is just a trick to switch between player 1 and player 2 in the levels of recursion. The "beta" value is the highest scoring move of the opponent so far, so once we switch to the opponent player in the first line, the second line just sets a large negative score so that the loop will start by selecting the first available move as being a good idea. The j loop tries each move, and the if test on the succeeding line just ensures that there is a non-zero number of seeds to pick up-- in other words, it ensure the move is valid. Then, the opponent's move is scored by recursively calling KalahGame.scoreMove(). When the recursion returns, the succeeding if test checks whether the move is better than the best result so far, stored in "beta". If it is, then this move becomes the new "beta". The alpha/beta business at the end of the j loop is an optimization that can be safely ignored. Once the j loop has examined all the moves, the best opponent move score "beta" is subtracted from the immediate benefit value of the player's move.

This is how each of a player's possible moves is scored in Kalah.getBestComputerMove(): The move's immediate benefit in the number of seeds scored minus the best value obtained from a recursive lookahead of possible opponent responses that accounts for the player's responses to the opponent, and the opponents responses in kind, and so on down to the limit of the look ahead level.

The fun bit of this code is that it is used not only to determine the computer's best move, but when you ask for the "Expert Advisor" to help you, it applies exactly the same logic to *your* board position in order to determine a recommended next move for you.

To conclude, here is a small diagram to help you see what is going on.

In this example, we're near the end of the game, and Player 1 must decide whether to make move 2 or 4. With move 2, there is an immediate benefit of 4 seeds because the 1 seed lands in an empty house, allowing the player to score that seed as well as the 3 seeds in the opposing house. This seems like a good idea, but is it? Well, Player 2's moves should be examined. In the short term, Player 2 can only respond with move 5, but this spreads out the 4 seeds. If you look ahead to the end of the game, you can see that Player 2 will ultimately score all four of those seeds. But also in the recursion, it is unavoidable that Player 1 will be able to score the remaining two seeds on the top row of houses. So the net benefit to Player 1 of making move 2 is only 2 seeds: the immediate 4 seeds, minus the 4 earned by player 2 in the rest of the game, plus the 2 additional seeds that Player 1 earns in the rest of the game. Not as good as it initially looked. However, it does turn out to be better than move 4 for Player 1. The immediate move yields no seeds for Player 1. Then, in the rest of the game play, Player 2 is able to earn 7 seeds, and Player 1 only earns 3 seeds. So, if player 1 makes move 2, then the opponent gains 4 more seeds than player 1 does.

Well, that's a wrap for this explanation of the 2-party competitive algorithm known as the "Minimax" method. Hopefully you can now see that it's not real intelligence but rather just tabulation of best outcomes according to a scoring method and constrained to a set of rules for determining valid next moves. Demystified, it becomes no more surprising that the algorithm defeats humans than it is when an algorithm can beat a human at calculating the square of a 5 digit number.

Still, this is roughly what a person does. Time and again, new possibilities are "imagined" by testing "what if" this move is made or that move is made. And the algorithm does win a lot of games, which is precisely why prescriptive analytics algorithms are so valuable as expert advisors. If you take the material covered here up by an order of magnitude, you get IBM Deep Blue. Another order of magnitude, and you get IBM Watson. The sky's the limit!

David Lee Roth and Eddie Van Halen have been trying to get us to do it for decades: "JUMP!" Douglas Hofstadter would qualify that with "... out of the system!" Here's what that means.

Machine intelligent entities like James Blog exist within a certain system, conforming to a prescribed set of rules, and they really can't escape the confines and constraints of that programming. Within that limited domain, they do calculate wonderful results that can seem intelligent. In an early version, I found myself adding a logger so I could see why James Blog was not making some moves that seemed very good. Time and again, I would find that the good move now set up the conditions for a better opponent move later, which is exactly what the artificial intelligence is supposed to detect and avoid.

The algorithm does this so well that it is really hard to beat, especially on the maximum lookahead value I set, which was 6. Frankly, if you're new to this game, you have to work to beat even the initial lookahead level setting of 2, which means that James Blog only looks at its own moves and your countermoves to see what will produce the greatest net gain in seeds relative to you.

Because it is hard to beat this little game and see the special winners message, this opened up a delightful opportunity to talk about an important capacity of human intelligence that could be exemplified by determining the winners message without winning. I used a Zen-like characterization of a "winless win" as a nod to Hoftstadter's style in the book Gödel, Escher, Bach.

Put simply, we are not limited in our thinking to the confines of the system. We regularly "take it up a level" or "think outside the box". In this case, the system is a blog entry presented in a web page. So you can jump out of the system by using the View Source feature of your web browser to take a look at James Blog's code, where you will find the winners message: "I, for one, welcome my non-computer overlord." The message is an allusion to Ken Jennings' capitulation to IBM Watson, which was an awesome pop culture nod to The Simpsons-- awesome because both Jeopardy and the Watson AI are about sorting out exactly those kinds of allusions.

Frankly, I had a lot of fun with allusions, both in the blog entry and while holding the programmer challenge to achieve this winless win. For example, James mentions that he outfoxes his friend Wiley, alluding to the famous coyote, who is in the same animal family as a fox (Canidae), which is a tiny aural tweak from Canada, where I live. So, James can beat his wiley creator. Similarly, in tweets and status updates, I made numerous allusions to The Matrix movie, such as when I nearly used Morpheus's command to Neo: "Quit trying to hit me and hit me." The exception is that I changed the 'h' to a 'g', making 'git', which is what we use to get source code.

This kind of wordplay and allusion bears some similarity to "jumping out of the system". Hofstadter calls it contextual slipping, or my favorite word for it: counterfactualization. We take some piece of reality that we know about, and we ask "what if this were different?" We slip, or change, some piece of that reality to see if we end up with something new and useful. I find the notion of counterfactualization fascinating because it seems like a good operationalization of some other really important words: creativity, playfulness, humour, imagination.

Still, it might be a while between when we can efficiently and effectively operationalize contextual slipping and when we can generalize that to achieve machine intelligence that can jump out of any system in the way that I asked programmers to do with James Blog. At some point, I realized that there is a beautiful geometric analogy that helps explain why. In the book Flatland, the Sphere is able to escape the plane via the use of a third geometric dimension that is physically orthogonal to the two that comprise the plane. In this way, Sphere is able to see Square's inner workings. That is a great analogy with what we did by jumping out of the web page using View Source to see James Blog's inner workings. There was a whole different, higher level of understanding about what James was and how we could know more about it, and it is fitting to say we got that winners message by thinking outside the box.

Next blog will be a developer's tour of the particular machine intelligence algorithm built into James Blog. After that, will be a discussion of the relationships between machine intelligence, machine learning, and predictive analytics, so stay tuned!

Your intelligent behavior is based on sentient *understanding*. Sentient schmentient. I'll bet my intelligent behavior can outfox yours. I've done so with my friend Wiley from Canidae, and he's a genius! So, let's see how much good your sapience does you, shall we?

The rules of the contest are simple. You get the top six "houses" and the "store" on the top left. I get the bottom six houses and the bottom right store. We each start out with 6 seeds in each of our 6 houses, and 0 seeds in our stores. To win, you have to get more than half of the seeds into your store (for you knuckle draggers, that's 37 or more). I'll let you go first, so you already start with advantage.

To take your turn, you pick one of your houses that contains seeds. That house is emptied, and its seeds are "sowed" one at a time in a counterclockwise fashion, including your store but excluding mine. So, it takes 13 seeds to traverse from a starting house, through your store, through my houses, and back to your (now empty) starting house. Every seed that goes into your store gets you closer to victory.

You can earn a seed or two from your move, but there are a few more rules that can earn you lots of seeds. First, if the last seed you sow lands in your store, you get another turn, and you can have multiple extra turns if you make your moves in the right order. Second, if the last seed you sow lands in an empty house, then you earn that seed from the empty house and all seeds in the house of mine immediately below the empty house. I call this a "big take". Third, if I run out of seeds in all my houses, then you earn all the seeds in your houses. Of course, I can also earn lots of seeds by these same rules, which is why YOU'RE GOING TO LOSE MEAT BAG!

I will take it easier on you at first, but I'll play harder if you earn the privilege. And there's a special message for you, a badge of distinction, if you manage to beat me when I play my hardest. Ooops. You... win?!? Wake up! Your teetering bulb is dreaming!

Milk Drinker (you)

Blog, James Blog (me)

Less messages, I understand how JB is winning

SPOILER ALERT. PLAY A WHILE, BEFORE LOOKING ANY FURTHER.

OK, so hopefully you've played enough to know you're not going to be getting that badge of distinction anytime soon (unless you have some of the rare talents of Ted Neustaedter). But also hopefully you're coming to the understanding that I really have no clue what I'm doing when I beat you. What I'm doing is mechanical, not miraculous. I'm being no more intelligent, really, than a calculator squaring a five digit number. Now, when one of you meat bags does it, it actually is miraculous. But the miracle is that you can do it at all on your hardware given that it is designed more for sentient understanding of what mechanical operations like squaring are, what they're good for, and what to combine them with.

I am just doing the fine-grain operations of my Minimax algorithm, but it is you who understands our contest at a higher level than that. That's why machine intelligence like mine is best applied as an expert advisor. For example, if you hit "Invoke Expert Advisor", you are asking me to advise you in the limited domain where my simulated intelligence would seem like real intelligence.

Keep using that expert advisor button and see how much faster you earn that special "badge of distinction" message. Go ahead. You won't be able to do it entirely without also sprinkling in your own intelligence at some points. This will be because you will hit some key points where your sentient understanding recognizes a *pattern* that emerges that will allow you to see how to beat my mechanical intelligence, where even my own advice is unable to do so. What will most likely happen is that you'll use the advice to hold your own for most of the game. My advice will help you avoid moves that give me extra turns and "big take" opportunities. But at some point, you may see that I am beginning to be starved of seeds in my houses. You, as an expert, will have this insight sooner than I see it coming using my mechanical calculations because your sentient intelligence truly understands what is going on at that higher level.

But of course, you would have a much harder time getting to that point without my advice. And that is what makes machine intelligence like advanced analytics on big data and machine learning technologies like IBM Watson invaluable to you. In short, expert advisors can turbocharge the smarts in your smarter workforce.

In a recent video interview, the IBM CEO Ginni Rometty comments that Watson 2.0 will understand images that it sees, and that Watson 3.0 will be able to debate, i.e. to understand what it is talking about with another party. An impressive roadmap, each of these is an incredible leap forward from its predecessor.

It is, however, worth qualifying the term 'understand'. It is being used figuratively, not literally, to communicate the rough order of magnitude improvement in capability. When such a leap is made, it seems analogous to sentient understanding, even though it isn't. Imagine for a moment what Archimedes would have thought at first of a hand-held calculator, given that he had the power of Roman numerals with which to calculate pi to several digits. And yet, we would not now interpret such a device as artificial intelligence. As soon as the mechanical nature of a level of capability becomes clear, so too does the fact that it does not constitute sentient intelligence (Hofstadter's exposition of Tesler's "theorem").

You can see this assertion play out in multiple levels of Bob Sutor's scale of cognitive computing. There are levels that are clearly not cognitive intelligence, as Sutor points out, but if you lay out the scale on a timeline of decades or centuries, it is clear that each level might once have been interpreted as being indistinguishable from magic.

So where on Sutor's scale is Watson? And what implications does that have for development best practices?

Watson is clearly not on the "Sentient (we can do without humans) systems" level. As sentient beings, we don't just know things with a certain calculated accuracy or confidence level, or determine that we don't know if our confidence is low. We experience desire to know more, and we experience fear of the unknown. We are teetering bulbs of dread and dream (Hofstadter's delightful invocation of a Russell Edson poem). I urge you to let that characterization of us sink into your mind. In Watson technology, IBM has modeled a certain class of knowledge and mechanical reasoning, and in other research, IBM is doing so by simulating some of the known structure of biological brains. However, we don't yet know how to model fear and desire, dread and dream. In my opinion, these are inextricably bound together in sentient intelligence, separating it from simulated intelligence. In other words, intelligent behavior is a construct that works for the dread and dream engine of the sentient, and in the absence of dread and dream, seeming intelligent behavior is but a mechanical simulation of understanding. As an aside, I hope we only manage to model desire and fear around the same time we figure out how to model ethics (as Asimov cautions).

Does this characterization of Watson as a mechanical simulation of understanding detract from its value? Does it detract from the order of magnitude improvement it heralds as an usher of the era of cognitive computing? Of course not, quite the opposite. It is simply fantastic that this level of "Learning, Reasoning, Inference Systems" (Sutor's scale) is now computationally and economically feasible at the scale needed to help sentient intelligence (that's us) to solve real world problems. Quick, what is the square root of 7. Can't do it? No problem. Even if you're Arthur Benjamin, you'd be better off just hitting a few keys on a calculator. Quick, what are the most likely diagnoses for the patient's presenting symptoms? An "expert advisor" like Watson can be just what it takes to help determine the next best action, especially when time is of the essence because a life hangs in the balance.

The term "expert advisor" is appropriate. It conveys that the system is a "Learning, Reasoning, Inference System" that does not have sentient understanding and is therefore made available to advise and guide the actions of an expert. This is analogous to the way spreadsheets guide the results reported by accountants and chief financial officers. That being said, we also know not to put spreadsheets in the hands of toddlers. From a development practice standpoint, it is crucial to keep in mind that "expert advisor" means that the deployed system should be advising someone who is a qualified expert in the exact domain in which the "expert advisor" system was trained. Especially when a life hangs in the balance, access to the "expert advisor" system needs to be performed by those with expert qualifications in the domain because only they can reasonably be expected to use sentient understanding to interpret and follow up on the advice. In other words, the term 'expert' in 'expert advisor' should apply to the user more so than the advisor.

Now, given an enterprise workforce of those with qualified sentient understanding of their topic areas, Watson-style expert advisors are just the type of technological advancement that will help them work smarter, not harder, to meet the needs of customers and colleagues and to produce a competitive advantage for the business.