This is progress beyond DeepMind’s former Atari 2600 game demo, and I don’t want to pooh-pooh the amount of work and brilliance that goes into making something like this ACTUALLY WORK !! But this doesn’t especially feel to me like some sort of breakthrough, just good solid work in deep reinforcement learning… In particular, it doesn’t convince me that deep RL as DeepMind has habitually practiced it is an adequate approach to AGI….

The key issue is that these video games are very constrained domains, so that a deep reinforcement learning system can handle them without being able to interpret data in a broader context. This kind of constrained domain can obviously be handled by methods that wouldn’t work for broader, messier, more contextuality-rich real-world domains….

So to me “learning independently in a very constrained world” is not entirely the same problem as “learning independently in a big rich messy world” — the former can be done by a system that ignores a lot of stuff any real-world AI has to pay a lot of attention to…. The universe of simple video games they’ve dealt with in these fascinating experiments, are in the end still a very small and special domain for which a software system can be heavily tuned in various ways (i.e. even if the tuning is independent of which specific game is being learned, it can still be particular to the domain of “games of this type”).

Personally I think DeepMind would need to add a lot of extra stuff to their AGI approach as I understand it (at least, a lot of stuff beyond anything they’ve discussed publicly, and beyond anything hinted at by this demo), to approach AGI seriously. I think the kinds of deep learning and deep reinforcement learning they’ve focused on are only a moderate-sized fraction of the story of what’s needed for human-level cognition. To me, this demo doesn’t show that they have embraced the complexity and multi-aspect nature of what’s needed for human-level AGI. (Maybe they HAVE, behind the scenes, but this video doesn’t show it….)

Another way to put my point is: To really hit a home run their system will need to be able to learn to deal with new DOMAINS, not just new games within a domain for which their system has already been tuned and tweaked and human-configured….

One thing this demo does show is that, since their acquisition by Google, DeepMind is pushing ahead with AGI-oriented R&D according to their own tastes and directions, rather than simply applying their brainpower to Google’s search and ads products and so forth. That’s very nice to see 😉

Alejandro says:

I think what we are seeing is mainly Volodymyr Mnih’s personal project continuing, not DeepMind’s or Google’s strategy into AI. There are very few differences between the architecture and algorithm used in this paper and the previous one that they published seven months before this paper got submitted (bigger, deeper network and delayed updating of Q evaluator, that’s it). Note also how in their recent DRAW paper (http://arxiv.org/abs/1502.04623) they could have used a reinforcement learning paradigm but they chose to go the supervised learning route instead, and they haven’t used reinforcement learning for anything that didn’t involve Mnih as far as I can tell.

Regarding it not dealing with different domains, it of course depends on what you mean by a domain. Even within one single game, there are parts that are different enough that I would call them different domains. On the other extreme, if you want domains to mean something completely general, you’ve got the No Free Lunch theorem telling you you can’t do that (“That’s not a new domain, you are still playing in a Kolmogorov-complexity defined universe”). In between you’ve got humans which can deal with some types of domain adaptations. I understand that you are saying that the commonality between the different environments it does learn in are too big. Your phrasing seems to indicate that you see a much harder line in what constitutes a domain than I can.

Thank you for any other wonderful article. The place else may just anybody get that kind
of information in such an ideal manner of writing? I’ve a presentation subsequent week,
and I’m on the search for such information.

Thanks for any other excellent article. The place else
could anyone get that type of information in such a perfect manner of writing?
I have a presentation next week, and I’m on the search for such info.

Thanks , I’ve just been searching for info about this subject for a long time and yours is the best I have came upon till now.
But, what about the bottom line? Are you certain in regards to the source?

We absolutely love your blog and find the majority of
your post’s to be just what I’m looking
for. Does one offer guest writers to write content to suit your needs?
I wouldn’t mind publishing a post or elaborating on some of the subjects you write concerning here.
Again, awesome weblog!

What i don’t realize is in fact how you’re no longer really much more neatly-preferred than you may be now.

You’re very intelligent. You realize thus significantly in terms of this topic, produced me in my view believe it from numerous various angles.
Its like women and men aren’t involved unless it’s something to accomplish with Girl gaga!
Your own stuffs excellent. All the time maintain it
up!

Hey I am so thrilled I found your website, I really found you by error, while I
was looking on Bing for something else, Anyhow I am here
now and would just like to say kudos for a remarkable post and a all round entertaining blog (I also love the theme/design), I don’t have time to
read through it all at the minute but I have bookmarked it and
also added your RSS feeds, so when I have time I will be
back to read a great deal more, Please do keep up the excellent job.