In order for this site to work properly, it uses cookies and javascript.
You can find more information here.

In order for this site to work properly, it uses cookies and javascript.
This site also tracks visits anonymously using cookies. Click 'agree' to confirm you are happy with that.
You can find more information in this site's policy.

Using Reinforcement Learning To Learn To Play Tic-Tac-ToeAbout a year ago I set myself the goal of writing an algorithm that could learn to play tic-tac-toe. I didn't want to tell the algorithm what the rules of the game are, nor did I want it to try and use some kind of calculation to look ahead at possible ...

The best opening move in a game of tic-tac-toeAs part of a machine learning project, I had to understand tic-tac-toe better, and so I have written an algorithm which a) finds all the possible unique games and b) gathers statistical information about those games. Based on Wikipedia's tic-tac-toe ...

No, not a report about the death of dinosaurs many millions of years ago, nor a report about how in fact many dinosaurs didn't die out, but rather evolved into birds...

Rather, this blog entry is about dinos which is a Facebook Application written by maxant. Basically, you get a virtual dinosaur which you need to feed. It can also fight or flirt with the dinosaurs of your friends, which directly affects its health. The idea is to keep your dino alive as long as possible.

There were several goals in developing it. First of all it was an experiment to see how the Facebook API worked. Second, it was to see how much faster a Facebook application would grow, compared to a normal website, because Facebook gives you "free website advertising" in terms of being able to invite friends to play your game/application. As such it is a type of viral marketing. The aim is to give it enough momentum that its growth will become exponential. At a minimum, growth should be linear shouldn't it? Well, right from the start, Google Analytics were used to track site traffic. Below are some graphs showing this traffic.

The general trend is slow but certain death, much like most original dinosaurs, all those years ago. The point of this blog is not to show that maxant is good at making crap games. Generally speaking Facebook application installations are falling fast, as the novelty wears off. The result is less new people playing the game and as long term players start to get bored, they just uninstall it. It's the same principal as pandas dying out because they don't mate and bear children frequently enough.

Now what's interesting about all that? Well there is a site called adonomics.com which tries to give valuations to Facebook applications (they also broker deals for application sales and do VC). Back in its hey day, dinos was valued at thousands of dollars. If someone had bought it (if indeed it had been for sale!) they would have hoped to earn revenue through advertising, to at least get a return on their investment, but hopefully to rake in much more! Originally and for only a short time, dinos had Google Ads on its page. After a short time, a much more lucrative advertiser came along namely cubics.com. During its lifetime, dinos has managed to earn roughly 50 US cents per thousand page views. It uses ethical advertising, so nothing where the user is forced or tricked into clicking on ads or where they are required to complete an ads questionnaire in order to help them in their game. And perhaps as a result, dinos hardly manages to pay its rent (that is, server costs). If development time, server costs and advertising revenue are summed, it bluntly never made a profit. Lucky for the individual who might have bought it based on the adonomics.com valuation, that it was never for sale!

If you look at bigger applications, who also use ethical advertising campaigns, their valuations are also well out. This is not because adonomics.com is hyping the applications values, it is because their calculations are probably based on the valuation of Facebook (an albeit high one).

You don't even have to go so far as looking at Facebook applications. Just take the market valuation of Facebook and divide the number of active users by it. It turns out (based roughly on figures I remember from last October) that an average Facebook user would need to generate hundreds if not thousands of dollars of revenue for people advertising on Facebook. Since I don't ever buy anything based on Facebook ads, and none of my friends do either, there must be some people spending thousands or tens of thousands of their hard earned cash on products being advertised on Facebook. Or maybe it is just seriously over valued. Or more likely, people/companies who advertise on Facebook are really really getting ripped off. Who knows?

But it does prove that making real money on the net is still really hard if you don't sell porn or host a gambling site.

In the last few weeks I have heard the term OSGi come up more and more, and one blog posting I read suggested that it was the hot topic of 2008. So I started to research a little. I am currently working heavily with the Eclipse Rich Client Platform building applications which use services deployed to IBM WebSphere. Both these platforms are built up on OSGi (the standard) and both use Eclipse Equinox (an implementation of the standard). So it must be important right?

Well you don't have to read too much before you start to get the feeling that you have been there and done that before. One aim of OSGi is to provide a micro kernel for deploying and managing services. Well, from a high level, JMX (MBeans) already does that. Not enough? Well there used to be a project called Apache Avalon Phoenix, which was a mirco kernal and although that project died and was resurrected as Loom from Codehaus (which incidentally has a very interesting history of Apache and Phoenix), it is still the basis of some big projects like the Apache James mail server.

Other micro kernals? How about JBoss? There is a good blog article dicussing how JBoss has been based on a micro kernel for some time now. The idea is nothing new and in fact in their case, OSGi does not really go far enough that they could be solely based on it.

Actually, doesn't the Java EE EJB specification let you deploy "services"? It even helps with transactions, security, robustness and scalability of your services.

So I start to get the feeling that as someone who spends a lot of time creating services, that OSGi might not really help me directly. Furthermore, I think about how I use Java. I know what it does, I know its API, but how it does it is of little concern. Ocassionally I will debug into a Java library, but not very often. In this same manner, do I really care about OSGi, or is it simply something that an application server or client framework will use as a means to an end, namely making services available, manageable and discoverable?

I then read on The Register that Sun has JSR 277 and OSGi doesn't like it because its competition. Well, I'm sure Phoenix didn't like its competition. And I'm sure OSGi has killed a few other similar specifications.

Before I get too excited about OSGi, I will also think about the opening line of its mission statement "Our mission is to create a market for universal middleware." That sounds a lot like Sun saying that the Java EE standard is great because you do not get locked into vendors. Erm... so within minutes of the Java EE specifications coming into existence, vendors all around the globe were adding value-added features to their application servers to give them market differentiation and unique selling points - things they need in order to survive normal day to day business. Just because a vendor is now OSGi compliant, does not mean their components will really deploy to other OSGi platforms does it? Not at least without a little bit of work.

A friend who works in Formula 1 recently told me a story of how his team added a little wing to their car after doing minimal wind tunnel testing to ensure it had no negative effects. The aim was not to give them an advantage. The aim was to get other teams to spend a lot of money and time in their wind tunnels trying to work out what the hell this little wing was doing on that car! Once something like OSGi has enough wind in its sails, and one big vendor starts using it, the others are almost compelled to use it in order to keep up with its competitors.

That all said, OSGi still looks useful, and I will not ignore it. I am a consultant who taylors my skills to my clients needs. If I need OSGi skills, I will aquire them. I recently had to dig a little deeper into OSGi in order to solve a problem I had deploying a web application. I found it interesting and I now understand it. But that doesn't mean I liked it. The problem was that my web application could not load up my plugin to display the plugin help online (see my blog about it). But come on, what happened to a good old ClassNotFoundException?! Instead I got no error at all. The thing simply didn't work. All thanks to the framework being based on OSGi, which provides the framework with hot deployable services, which in this case is a feature not even being used... Great. But at least I now have some OSGi skills.

=========

UPDATE!

A year on and I ask myself what I now thing, having used more and more OSGi on my current project. I have seen an implementation of Declarative Services which was more complex than it needed to be. And I have lost a lot of time fiddling with configuration of OSGi Modules (especially when it comes to import/export packages and version numbers). I think a fair statement right now is that the most useful feature of OSGi is that it allows you to deploy multiple versions of the same library and your application can sort out which one it loads depending upon which version it was built against. In our current application where we deploy two sub-applications to a workbench, this is quite powerful (albeit theoretical because until now the release schedules of the two applications have matched and so they were always built against the same library versions). So basically, I'm still not a great fan, although I do see it has power in certain situations. In the mean time I shall not convert all my applications to use OSGi, unless I happen to use RCP or something that is dependent upon it, in which case I will use not much more than just the versioning which it provides.

Previously in this blog, the Eclipse Help / Infocenter was discussed and details on how to set up the help for an RCP application were given. But often you want to have your help for an application online as well as part of the product, for example if the customer does not have the latest version of the product installed. Indeed the IBM and Eclipse web sites have what they call an Infocenter - an online version of their help system. In reality, when the Eclipse Help System is running, it runs as an embedded web server within your application. So in theory, it should be possible to deploy that as a standard web application.

Quoting the Eclipse Help, "The help system can run in three modes: workbench (normal), infocenter, and standalone." Normal is when it is part of your application. Infocenter is when it runs as a seperate process acting as a web server. Standalone is when it is used outside of an RCP application.

In fact, from Eclipse 3.4 upwards, it can also be deployed as a standard Java EE web application. The following blog shows how to deploy that help to a standard Java EE web server, namely Tomcat 5.5.

Unfortunately, at the time of writing, there is no simple way to get this running. Neither is there any good tutorial showing how to overcome the pitfalls of the descriptions provided in the Eclipse 3.4 Help (search for WAR and you will get the details). Those details are repeated here, as starting steps (I hope this does not breach copyright, but the aim of this tutorial is to give a complete guide to getting help to work as a web application):

Deploying the infocenter as a Web Archive

Using Eclipse 3.4 or later it is possible to configure the help plugins to be deployed as a web archive (war file) which will act as a fully functioning infocenter. The instructions below assume a Tomcat server has been installed, but with minor modifications these steps should work for any full featured server.

In your Eclipse 3.4 installation locate the plugin org.eclipse.help.webapp..jar and copy it to a temporary directory, unzip the copy into that folder.

In the webapp plugin locate the web-archive directory and underneath that there will be two directories titled "help" and "org.eclipse.help.infocenter-feature".

Export org.eclipse.help.infocenter-feature as a deployable feature and set the destination to be web-archive/help/WEB-INF/eclipse in the area where org.eclipse.help.webapp.<version>.jar was unzipped.

Download org.eclipse.equinox.http.servletbridge_<version>.jar and org.eclipse.equinox.servletbridge_<version>.jar from the equinox download site. Select a version of Equinox that matches the version of Eclipse you are running to take you to the downloads page.

Note A: The documents state "Add some documentation plugins to the webapps/help/WEB-INF/eclipse/plugins directory. " This has provided some confusion among the postings that I found on the internet. Basically there are two choices. The first is to take your plugin which contains the help folder, and put it into the folder named above. It can be put there as a deflated / flat folder (that is, unjarred), or as the standard exported plugin jar which the export wizard creates for you. The second choice is to create you help contents within a second plugin, seperate to your application, and to place that exported plugin into the above named folder. The advantage of the second approach is that you do not have to deploy all the dependencies which your plugin has, which are not related to help. I however, went for the first option, simply because it then only need to do one export and each time I export my plugin in order to create a new release / upgrade to the application, I simply take that exported plugin and pop it onto my webserver.

Note B: Once you have completed Note A and all the other steps in the citation above, you can continue with the last steps of the documentation: " At this stage you can create a war file from the help directory or you can copy the directory and its contents to the webapps folder of your Tomcat installation. "

If your'e lucky, it all works and your online help looks great! If however, you are more like me, then you just get a blank help screen. OK, not too bad, the solution to this is that instead of the URL being http://yourdomain.com/help, it needs to have the index page appended: http://yourdomain.com/help/index.jsp.

In the web.xml of the help webapp, there was no welcome page descriptor. I added one, but that didn't help! I think it is because the web app parses the requested URL and then works with a virtual file system. But why Tomcat doesn't step in before hand if it sees no requested page, I do not understand. Anyway...

The next bit didn't fully work for me either. The infocenter loaded nicely, but the contents was empty. It only took a few hours of fiddeling, and here is what I discovered:

The web application loads the Eclipse workbench behind the scenes. This is based upon the Eclipse Equinox implementation of OSGi, which has a neat little command line where you can find out what services are deployed in the workbench.

Within the web.xml of the help webapp, there is a servlet start parameter which is commented out. Some postings that I found online even say that it MUST be commented out. Well not in this instance. Ensure the following is not commented out in your web descriptor:

Restart Tomcat, from a command line, without starting the process in the background (that is on say linux, do not add the & command after . catalina.sh run.

Once it has started and gives you the commandline message saying Server startup in x seconds, hit return. As if by magic, an OSGi prompt appears. On this prompt you can type help (or any unrecognised command) to get help. The ss command will list installed services with their state. Doing so gives the following output (click image for a larger picture), where BookStore_1.0.0 is my plugin containing the help contents.

My plugin (ID 6) is installed!! Great. But actually it needs to be active before it is of any use to me. So the next command I type is start 6, which should start my plugin, since it was listed with ID 6 (click image for a larger picture).

In the above picture, I have highlighted the problem. My plugin is dependent upon org.eclipse.tomcat and that is a missing resource. Well you might think that it is strange that the webapp is running in Tomcat and Tomcat is missing, but it is actually the Tomcat plugin that is missing in the Eclipse workbench environment, namely under the WEB-INF/eclipse/plugins directory. Add this plugin from your Eclipse plugins directory (or indeed your RCP application export plugins directory), as well as any other plugins upon which your plugin is dependent, restart Tomcat and everything should work.

If it still does not work, try stopping Tomcat, deleting the <tomcat-install-directory>/work directory, and restarting Tomcat. Tomcat has a working directory where it caches webapps and in this case it is not very good at cleaning up the Eclipse workbench environment (either during hot deploys or during restarts).

If that still does not work, iteratively use the OSGi command line to until your plugin has a state of <<LAZY>>. This state is fine, since it will get started when you call up the URL. In fact the lazy status may never change, but it is sufficient to indicate that your plugin is now loaded and ready to run in the help system.

Before deploying this webapp to a production environment, don't forget to comment out the OSGi command line parameter in the web.xml file.

Eclipse offers plugin authors the ability to add Eclipse Help to their plugins. Opening that help will give a nice window with searchable help, something like this (click on the image to see it in full size):

The following blog entry shows how to integrate help into an RCP application (or indeed a plugin or feature).

The first thing you need to do, is to extend your plugin.xml to tell your plugin that you want to have help. In the source view of your plugin.xml include the following extension points:

To go with these entries, you now need to add the help folder to your plugin project:

The html folder under the help folder contains simple HTML documents for each page of help that you want to write. The toc.xml file is the table of contents which defines what appears in the left pane of the help system when it is opened. The help_contexts.xml file contains mappings from "context names" to files which are relevant for that context. Contexts are used when opening context sensitive help, for example when pushing the F1 button.

An example of the toc.xml follows:

An example of the help_contexts.xml follows:

Now before the help system will work, we need to do a few more things, namely:

Add dependencies

Add a help menu

Add context hooks

To add dependencies, you can open the .product file for your application (to export your plugin as an RCP application, you need to add a .product file). Open its configuration tab and click on the "Add Required Plug-ins" button. This should then add a list of all plugins upon which your plugin is dependent.

If this did not work, the following is a list of all the plugins which the BookStore plugin is dependent upon (if you open the .product file in the text editor, you can edit XML directly):

To add a help menu, edit your ApplicationActionBarAdvisor class. First add a help action as a class member variable:

private IWorkbenchAction helpAction;

Next, in the makeActions method, add the following code to register the action:

The last thing we need to do before the help system will work as an extra window within our application (Workbench Mode), is to add the context sensitive hooks. On any graphical component where the F1 key is pressed, the Eclipse workbench will look for a parent that has context sensitive help setup. If it finds such a parent, it will activate the help.

To enable context help for an item, add the following bit of code (here, for adding context specific help to a view):

Note that the String has the prefix "BookStore.", but in the context XML above, the context IDs do not contain this prefix! The prefix is one of several things, I'm not really sure which! It could be the Application Name, the Plugin Name, or the TOC Label in the toc.xml file. It could even be the Topic which is somehow set for the help system. It's best to refer to the documentation for more details here.

Right, now that is all done, it is time to run your application and try out the F1 key and the new help menu.

Notice how the first page which the help shows is a standard Eclipse help page? Wouldn't it be great if it could be customised? Well, it can!

You simply need to add a line to your plugin_customization.ini file (this file can be added to applications in order to customize things like their default accelerator keys) like this:

org.eclipse.help.base/help_home=/BookStore/help/html/home.html

In the above line, the new home page for the help needs to be located under help/html/home.html within your project. The name "/BookStore" which appears above, before the actual path to the file is this magical application or plugin name, discussed above.

All of the above work was done in Eclipse 3.3.0, although it should also work for Eclipse 3.2 as well as Eclipse 3.4 and later.