Posts Tagged ‘google app engine’

Exciting! It looks like the first iteration of Unity Networking might be implemented in Unity 5.1. I’m quite enthused by this, as this is the key development I’m waiting on before investing heavily in building the next version of my multiplayer networking project. Well, not necessarily. Recently I acquired a license for the bolt engine, which looks like a reasonable alternative contender for this sort of functionality in Unity. So I may actually proceed with that for the time being, and potentially also the foreseeable future.

In other news, I was working quite hard to deploy a custom build of browserquest on google app engine a month or two back. I’ll spare you, the reader, the details, but basically I found the main obstacle to be documented in this stack overflow issue, which led to the author of same raising this ticket. So it doesn’t look like this sort of thing is possible at the moment – at least on google app engine. On amazon elastic compute cloud though, it should certainly be quite doable to implement. Maybe one might be able to write a lambda function for the nodejs server requests. Now that would be an interesting learning experience.

Recently in a comment on a post on dba.stackexchange I learned about a service called NDB, which presumably allows one to efficiently model many-to-many entity relationships in the google datastore. And apparently it is pluggable with django, too, via this:

To use NDB with the Django web framework, add 'google.appengine.ext.ndb.django_middleware.NdbDjangoMiddleware', to the MIDDLEWARE_CLASSES entry in your Django settings.py file.

In terms of getting started with this particular kit, there is a tutorial here which I may investigate at some point.

As an additional remark, there is a comment on this post on stack overflow indicating that I’ve been a bit remiss all along, that intermediate tables are the way to store keys / manage joins if using django nonrel. Nonetheless I think I’ll look into NDB for my future investigations, as either solution (NDB or django nonrel) requires reworking of models used in a django project, and I’d rather use one that offers automatic caching and batching, which apparently are useful things to have.

I came across the following project the other day: http://code.google.com/p/gae-django-cms/ . It works right out of the box, and seems fully equipped with admin and all sorts of other useful things. Additionally, it seems quite easy to integrate it with other (simple) django projects, like django-monetise. Consequently this is probably where I will park my investigations of GAE for now.

I’ve also been looking at DjangoBB for something similar, in the singular forum sense, from which I could probably then easily abstract to multiple forums. DjangoBB is rather more polished than fofou, and, of course, is written in the Django framework, which is probably a bit more solid than a free-for-all implementation as above. The packages I am using, apart from Django Bulletin Board, are:

The price to the latter it would seem is the difficulty in getting the thing to work – in fact, I have since discovered that the only way to make the program work properly is either to rewrite djangobb (hard) or django-nonrel (very hard!). Although it is certainly instructive to have a bit of a fumble with the djangotoolbox and django-nonrel code, there are inherent limitations with nosql that make a full solution to some of these problems more or less impossible. Since many of the pluggable components to django (such as djangobb) implicitly rely on relational queries this can make things very difficult for running things on a NoSQL database such as GAE.

There are then several approaches one can then take. One, is to use Google Cloud SQL – which is not horrendously expensive and I believe can be used for multiple apps for a single instantiation of the API. Another is to use alternative forum techs, such as fofou, and another which I discovered recently, gforum, though that comes with a user-advisory regarding its hunger for user information via the widget for user logon. But otherwise relatively promising. So I am currently looking at defanging that particular piece of code. For the adventurous/curious, the googlecode repository is here: http://code.google.com/p/gforum/source/checkout .

The third approach is to somehow solve the NoSQL many to many field problem and then incorporate that into django-nonrel. Apparently this is one of those ‘untouchable’ or overly-ambitious problems in the area of computer science. But apparently it is possible to solve the problem (although the answer is currently not open-sourced) as per this announcement here (quite recent too, September 2012): http://fatfractal.com/prod/joins-and-nosql/:

I’m an engineer and not usually given to making sweeping statements like, “we’ve solved the many-to-many relationships problem for NoSQL;” but in this case, I hope you’ll agree, it’s merited.

FatFractal are another engine like GAE. Apparently the engine solves the problem prior to loading to a NoSQL database which I presume is something like MongoDB, such as app engine uses. Regardless, simply knowing that the problem can be solved wins half the battle. Indeed, if FatFractal’s claim is true, the fact that the problem is not impossible means that presumably it is only a matter of time before it is independently discovered how to do so, and the knowledge becomes public domain – and thereby applicable to the current django-nonrel distribution / github project (currently at version 1.5 development, 1.4 stable), here: https://github.com/django-nonrel/django/tree/nonrel-1.5-beta .

But until this happens, for me, and other non-experts like myself, I think the best strategy is to use alternative technologies to django (as above) to work on specific applications, since the biggest advantage to django (apart from the built in admin) is the pluggability of components (like ruby-on-rails), and more or less everything breaks and therefore needs to be rewritten (if the data models allow such) for app engine. Consequently one might as well use web2py which is fully supported. Although I’d certainly like to learn a bit more about how SQL and noSQL model data ; if nothing else, this would be quite instructive.

As to the question – why NoSQL, if it makes joins so abominably difficult? The quick answer – speed. NoSQL is a stripped down version of SQL, and is therefore faster and more horizontally scalable (apparently) so more suited to use with applications / services that need colossal amounts of data (eg, location specific information, weather data). SQL is slower and does not scale as well, so is more suitable to applications where the data is somewhat more limited, but more highly interwoven and connected (eg, forums, blogs).

In terms of where I would like to take my Unity game, I’ve had another idea that I would like to try. That is, I would like to have an NPC with a variety of options in a chat window – and that these would transport a player to another scene in the game. The scene would probably be another SFS room, though not necessarily. Then certain modes I would like to implement:

“Rocks fall, everyone dies” a game in which the player can be the game-master, or where people can team up and be the hapless adventurers.

“Double dragon” a game in which the player is the dragon and has to assist an army in crushing another army (AI controlled), or where the player can play as a grunt in one army or another.

For RFED, in game-master mode (vs or against AI) the idea would be to play essentially a 3D game of tetris, wherein blocks would be placed on a 6 x 6 grid, with 10 levels of height. The AI or players (assisted by AI for empty slots) would then try to dodge the blocks placed by the GM. The GM would score according to 1. how many players he squishes, and 2. how many complete levels he creates with the blocks (and which thereby eliminate tetris style).

The players & AI meanwhile would be cooperating to activate certain squares in the grid; if a player is crushed the activation would go down by one. (If a player is crushed they get to respawn in a random location in the grid). An activation of 4 is necessary to achieve the desired subobjective, which is to effectively raise the height of the board by one, so that the GM has less space in which to deploy 3D tetris blocks.

The game ends when the level has been raised to 10, either by the players or by the GM failing to deploy blocks at height <= 10 (including player level raises). I would then define victory by assigning and tallying the points in some suitable fashion.

I then thought that the players + AI could have at the GM incorporated in dragon form in phase 2 of the fight, on a flattened version of the 10th level of the tetris grid. Each player / AI would have control of a “squad” of characters to fight the dragon. Anyway, just an idea!

In terms of “double dragon”, the idea would be more to have a situation in which there is a massive battle scene AI vs AI and the player’s job is merely to inflict havoc as a dragon. (I kind of like the idea of using dragon avatars for some reason).

Anyway, these are certainly a couple of goals to aim for in the mid to long term with this project.

————————————

For something of a change of topic, I have been thinking a bit about deploying a forum registration service on GAE. I’ve decided in particular that an appropriate use of DjangoBB might be the way to go, with the django-registration package to register users – I would then monetise the service using google adwords. If I became more ambitious along the way, I might introduce a django-admin moderated bannerspace bidding tool; the idea would be for advertisers to bid for “shares” in bannerspace (ie, probability that their ad is displayed), together with specified tags (which would decrease the “share” of bannerspace by restricted the ad to certain sites arranged by tag keyword, but which would allowed the advertising to be more targeted and therefore more beneficial to the advertiser). Naturally if there was more demand for bannerspace if the bidding tool was set up correctly, so the price would reflect this accordingly.

In terms of the tech I would use for the forum service, I’d probably opt for Django on google app engine, using google cloud SQL. The cost for a small application seems a bit pricey (~ $1.50 / day for the lowest tier, comparable to what I was spending – $2 / day – on my first RightScale experiment using an Amazon EC2 virtualbox) but there is a free tier usable for one application (0.5GB of storage free), so I might experiment with that. This, together with the excellent backup options and user-friendliness of the service makes it worth investigating.

Alternatively I was thinking of investigating a service where people could SSH in in an intuitive fashion after setting up an account, and deploy static websites (with ads). The TokBox experiment I may defer until I am slightly more experienced with these matters.

Regardless, a few things to keep in mind while investigating google app engine as a business tool.

In other news, I have a strategy for extracting the mysql tables for manual backup from this wordpress blog. Essentially a copy of ~/data/wp-content via this method here. (I can check that this is the correct location by

dotcloud run -A <name of application> <name of database service>
ls

in a terminal window.) Evidently I will need to test that this works. My plan is to create a new temporary application and see if I can “restore” the backup data to that application. If I can, I won’t have to mess around too much with automation of the process (I suppose automation is especially useful if one is setting up something like this for a client, and you don’t necessarily want to trust them with command line access to the code on the server itself).

It is worth noting that Google app engine also now supports backup (although still experimental), see here. But it looks very workable!