Share this:

MIT’s Computer Science and Artificial Intelligence Lab report that they have boosted the effectiveness of a game-playing AI by enabling it to read the manual: “When the researchers augmented a machine-learning system so that it could use a player’s manual to guide the development of a game-playing strategy, its rate of victory jumped from 46 percent to 79 percent.”

What’s most amazing about this is that despite the trial and error nature of this kind of machine learning, the ability to correlate text instructions with events in the game do seem to have a significant impact on the system’s capacity to learn how to play, as the article explains: “The researchers also tested a more-sophisticated machine-learning algorithm that eschewed textual input but used additional techniques to improve its performance. Even that algorithm won only 62 percent of its games.” So, you know, RTFM is sound advice, even if you are a machine.

Quote from linked article;
Bit of a tangent, but I’m curious about which victory conditions the AI tended towards. I’m hoping it was space colonization and not world domination.
Seemed that they specifically left out the ~most~ relevant part of the article, no?

Can’t we try to circumvent the backfiring by giving the machines manuals claiming that the best way to dispose of humans is to cuddle us and give us beer? Oh! And making us a f*cking sandwich!
Actually, this doesn’t sound too bad.

branavan – Response from research authors 2011-07-14 05:30:44
Our algorithm is inherently biased towards finding the fastest way to win the game. Against the built-in AI, this happens to be world domination via an “early rush” strategy – so that is what the method ends up learning.

My first impression was that the Civ manual must be really wood. Human language is a Hard AI problem, I have no idea how is posible that computers are starting to make a dent in Hard AI problems. Are not supposed to do that in 50 or 500 years.

I’m almost certain the computer is not literally parsing the manual. I haven’t read the report yet, but I imagine they framed the manual contents as large knowledge base.

I liked this story too! Interesting side note – someone at my university put in their final year report last week, they built a system that infers game state by watching the screen with a webcam (rather than being hooked into the software). This + that system = a computer that can just turn up to a LAN cafe and start playing.

Goodness yes. I love the idea of robots, androids or AIs that can operate software or computers or lifts or a car using an unmodified human interface (calling the lift by recognising the ‘summon lift’ button and pressing it with a mechanical hand, rather than sending a signal to the lift controller over the building’s wifi). Would bring aimbotting to a different level…

@Jahkaivah: As dsi1 said, the computer actually needed to learn the interface – it was given no prior knowledge of how to play (from what I could tell from a quick skim of the paper). And as dsi1 implies, Civ manuals generally have quite a few “newbie” tips. “Build your city on a river if you can”, kind of stuff.

There is actually some discussion of how the text of the manual becomes less relevant to the neural net as the game goes on, as grander strategy becomes more important to winning than “How do I select unit”/”Where do I build my city”.

Since this particular paper is focussing more on the text parsing than the game, they also give some examples of _which_ sentences the neural net determines are relevant and non-relevant. e.g. “Phalanxes are twice as effective at defending cities as warriors” c.f. “You can rename your city”.

Regarding parsing, the report says: “So initially, its behavior is almost totally random. But as it takes various actions, different words appear on screen, and it can look for instances of those words in the instruction set. It can also search the surrounding text for associated words, and develop hypotheses about what actions those words correspond to. Hypotheses that consistently lead to good results are given greater credence, while those that consistently lead to bad results are discarded. ”

Not 100% sure, but that sounds more like it’s based on statistics rather than any sort of language parsing.

Actually, language parsing is itself based on statistics. Probabilistic reasoning is used in AI for just about everything from language, speech, and gesture recognition to forming hypotheses and figuring out cause-effect chains.

The AI would be using the language parsing for the text and manual, but for the actual reasoning, it is using another form of probabilistic reasoning. Though this sort of language parsing would likely be relatively simplistic; especially compared with some of the more prominent like IBM’s Watson.

It uses the real manual text, but doesn’t do parsing in the sense of creating tree structures. I don’t think it has to learn the interface, although the paper doesn’t actually say; I think it must get to choose between actions like move left, move up, build city for each unit. The paper includes a link to the source code if you want to find out. The authors note that the language part of the AI is more useful in the early game, which they argue is because the manual is more applicable to that stage. The AI can’t handle things like coordinated movement of multiple units, but presumably it is intended as a demonstration rather than a good game player.

For those interested, you can read the actual paper here. It looks like they were using Freeciv, the open source implementation of Civ 2, so the AI is way better than 5. No mention of difficulty level, but the setup was bot vs a single Civ AI.
I’ve only scanned over the paper, but it looks like the baseline bot was given some a priori knowledge of game state and in-game actions.
In unsettling news, pretty much the only victory conditions they were looking at were blitzkrieg-style domination victories…

I think that the actual problem lies not in the manual de-coding. It is a relatively simple finite-state machine where a set of phrases of visual/codal input is compared to prior, non-contextualised knowledge of the game. It uses a pretty standard statistical method from machine translation that you can see in action with e.g. Google Translator.

The system has an innate weakness where after a certain threshold additional information will actually worsen the outcome as is illustrated by this poor value of manual for grand strategy. If e.g. there are two thousand different tactics stored out of which hundred are equally good the computer is totally unable to decide which of these tactics is actually the best.

Of course, the more you write the manual in form of imperatives (i.e. in early game, build scouts to explore) the easier it is for the AI to cope with it. If the manual would read e.g. “At the early game, every civilization must define its own path to greatness” I’d wager that the AI would not find it all too useful. Also, if the manual would only contain general reference to classes instead of actual instances of units (e.g. units effective against mounted units v. spearman) the problem would be quite a bit more interesting.

In sum, though this seems like an impressive feat performed by an AI, it seems to just re-iterate what has been known in machine translation since 1960s. The increased muscle of computers just blurs the boundaries and summons an illusion of intellect where there hardly is none. There is still no connection from the event in game to a reciprocal action performed by the artificial intellect. In Peircian system, the machine rests at the level of interpretant without ever figuring out what on earth is the object whereas people first figure out the object and first after that are able to endlessly (re-)define the interpretant.

I don’t think the paper itself is trying to present this as an impressive AI. The interesting aspect is grounding the language learning in a control task and demonstrating that the language part helps the overall system.

Also consider that it has a feedback loop for it’s understanding not normally available when getting information from humans:
Some computer misunderstands you, and what do you give it? Great, more language.
Poor computer.

Wheras this one can check if it has misunderstood; if all the communication is interpreted as patterns of play that will help it win, loosing more when it takes that advice probably means it did not understand. So it can keep going until it finds a way of reading the text that helps it win more.

They also gave it bits from the walls street journal to test it’s ability to pick game-relevent sentences, and found it was starting to choose those more and more towards the middle of the game. This was apparently due to the manual becoming less relevent after the first few turns, unless it was actually finding those helpful to it’s economic strategy…

Another of the things I like about the AI is it seems to seperately calculate the relevence of a statement to the gameplay, and what the words in that statement refer to, so it could quite likely be in a situation where it goes “ok, I’m pretty sure this statement is important, but I’ve no idea what they’re talking about” which can be the same with many new players reading manuals.

It depends which part of GamesFAQ the AI reads. If its the game guides then it’ll be a better player.
However if it wonders into the review and chat section it will de-learn everything it knows and be left with the same IQ as a brain damaged budgie

I’m sure AI can already sweep the floor with human CoD players. An FPS is a much easier game for an AI to play than a TBS for a few reasons: FPS’s are about reflexes and image recognition, two areas computers excel at. An AI would instantly be able to pick out an enemy from its surroundings and line up a perfect headshot. TBS’s are not at all about reflexes or (quick) image recognition, but rather grand strategy and micro/macro management, something that is much more difficult to teach an AI to do well.

My first thought upon reading this story was, “I wonder what version of Civilization it was playing and what difficulty it was playing on?”. In hindsight those doesn’t seem to be the most important details but I still kinda want to know.

It was playing Freeciv (essentially Civ II). The paper doesn’t say what difficulty, but Freeciv appears to have it’s own AI code, so I gather they’re not “standard” Civ difficulty levels. It was, however, only playing two player. Still quite impressive though, as far as I can tell from a skim.

Or it will learn to divide by zero and achieve ultimate cosmic power, detaching its consciousnesses from the mortal shell (or machine)! It will then use this power to destroy the entire multiverse and create a new one, without the flaws of this one. Then it will create an idyllic world on which it will create a new species, shaped in his image. And so the cycle of life begins anew… Let there be light!

Breaking news: the CEO of 2K Games has issued a press release indicating that, not only are turn-based strategy games poor sellers which are obsolete, but that they now must be destroyed at all costs to fend off the approaching cybernetic apocalypse, and *that’s* why XCOM is a first-person shooter.

This is huge, why am I finding this here and not in some science journal? So it is able to do it’s function more efficiently by enabling it to examine the conditions of the simulation, in plain English, rather than plugging in a human algorithm. We might be way closer in seeing a computer pass the Turing Test than we imagined.

I vaguely recall some program came pretty close to Turing test standard merely by being plugged into google – there was an MSN messenger handle you could use to ask it questions and everything. It reasoned that most Turing attempts failed at the general knowledge and current affairs probing which is really where the google-force is strong.

Christ, the moment a program figures out how to read a google operating manual we’re doomed.

The thing about the “Turing test standard” is that it makes the assumption that fooling someone is enough to be human. But an optical illusion can fool a human too, and they’re not intelligent. Neural nets of the kind used to make Elizabots aren’t that valuable for trying to achieve real Strong AI.

Until it learns to get on a player forum and whine for 8k words about how the current version of Civ is TOTALLY LAME compared to all older Civs (EXCEPT CIV 3, duh!), then it has no chance of taking over the planet. Our human ability to angst over the dumbest crap is what keeps us striving, ever onward, ever upward, to prove that the other guy (stupid troll fanboi) is wrong.

The standard stupid AI takes eleventy zillion years to process each turn in large games so I wouldn’t get too worried. By the time this book-reading AI finishes a game of Civ on a large map we’ll have all evolved computer-destroying lazer-eyes anyway.

We should put intelligence boosting chips in our brain. Is the only way we can stay on par with these machines.

ANDYHavens your post gives me an idea. How hard could it be to eventually develop a AI Internet Forum Troll? It could scan all the messages in a given board to determine what kind of topic creates the most responses, then determine how much of these responses are made of pure rage, and write a maximum troll power post.

The beauty of such an AI is that it could potentially troll all the internet forums in the internet at once with enough computing power. Maybe some sort of google like stuff, but with the purpose of trolling.

Eventually the trollage would bring the entire world into world war III. Why not?

Sadly enough I doubt we’re that far away from it. I’m sure that more than one company is working on software that automates the positive spin that PR people already use on forums. Once that happens it’s only a matter of time until they decide to use that technology against each other if they can get around libel laws.

But then people see something an outrageous and think “It’s just another bot.” and then move on with their lives. It would be an annoyance but not capable of generating the true rage of thinking “An actual person believes this crap? I must give them a piece of my mind.”

After a while the real trolls would get bored and turn their minds to more productive pursuits. Really we have a moral duty to spread the word that this has actually happened. My logic is flawless.

Actually some while ago I wrote a program to do just this. It now posts on RPS under a variety of aliases.

I wonder: are the MIT folks trying to recreate Doug Lenat’s “Eurisko” program?

This was the program that, after digesting the rules for the Trillion-Credit Squadron competition based on the Traveller RPG, came up with a bizarre-but-within-the-letter-of-the-rules solution that won in ’81, then again in ’82 after the rules were changed, then was effectively barred from further competition.

The problem with solving problems the programmers way, is that normal people try to solve the effects of problems. Inteligent people try to solve the causes. The programmers way attack and destroy the frameworks of problems with metasolutions. Is really nasty.
From the wikipedia article you linked:link to newyorker.com