Archives

All of my spare time has been taken up with the Google App Engine project I mentioned before. As with all tech these days, the platform is a moving target. With the initial release of GAE only supporting Python I dived straight in to learning Python and the Django framework. Then they release Java as the second language for GAE. Java is my day job so this would have saved a lot of time at the start. Learning Python was useful but spare time is a limited resource.

Then I spent quite a lot of time writing an authentication system for it as the one supplied, that utilises Google Accounts, did not associate a Google account with a specific ID, just an email address which might change – not particularly useful. The latest version of GAE fixes this.

Oh well. I’m happy with what I have but I’m still a fair way from going live.

But, as you can tell from the title, I am dipping into a couple other ideas I have. One is for a utility that might help the company I work for with their distributed development model. I wanted a highly productive server framework and a rich UI so decided on a Flex front end and Grails with BlazeDS for the server side. It seemed ideal. Especially as I could use Grails with the Flex plugin. I tried the quick start guide and it worked straight away. Within 10 minutes I had a flex client that could call a BlazeDS remote service. Creating the service was childs play, it was fantastic.

Then I hit a snag thats held me up for a couple of days. The quick start guide gets you to create the Flex app as MXML and AS files inside the Grails app folder structure. This is served to the client by the Flex webtier compiler which compiles on the fly. Great for development but no good for production. (Well it should be great for dev but my app didnt compile properly – no errors it just didn’t work properly and I haven’t worked out why yet) . The main problem was, whenever I compiled a swf file the remoting stopped working. I’m using Flashdevelop for the Flex IDE not FlexBuilder. The latter app may make the reason for this problem obvious but it wasn’t to me.

Anyway the solution for me was:

When you install the Flex plugin and create remoting services a services-config.xml file is created in the “web-app/WEB-INF/flex” folder of your Grails app. The client Flex app needs to know about this file in order to get the channel definitions. To do that you have to copy the XML file to your Flex project folder (I had a config folder alongside the src folder) and then tell the flex compiler about it with the -services argument. So I added this argument to the compile command “-services config\services-config.xml”. In the XML file there are a couple of tokens and the compiler did complain about the “context.root” token so I had to hard code this value into the XML file. But having done that and then re-compiled I copied the generated swf file and associated webpage into the Grails webapp folder, started the application, navigated to that page and remoting started working again 🙂

Having got it working, this whole process of creating and consuming AMF remote services is incredibly simple and fast. Convention over config rocks.

I’ve been looking into creating a community driven web site for a while now and when I found the Google App Engine (GAE) I decided it would make the ideal platform for my application.

I’m a great believer in the right tool for the right job so I’m not tied to a particular language or platform. Java is my day job but deployment and hosting purely in terms of cost is a barrier. With GAE there is no cost of entry, almost unlimited scaling ability and you only pay if the site becomes very popular. So this was an ideal opportunity to learn a little Python, the only language that is currently supported. I’d dabbled before but just couldn’t get past the “white space has meaning”. It just didn’t feel right.

I have to say that now I’ve spent some more time with the language I quite like it. As it is a scripting language and you get the Google App Engine SDK for local development, the whole code, compile, deploy, test cycle that comes with Java is thankfully avoided. I’ve decided to use Django with GAE as Google have provided a version of it that is compatible with the Google datastore and it is a great MVC web framework that takes away a lot of the work that would have to be done manually without it.

The only thing I have struggled with is the Google datastore. The version of Django that come with GAE has been modified to make the datastore look like a relational database. But it isn’t. It is just one big table that can store anything you want. This makes it fairly fast and very scalable. But.

The main problem you have is with JOINS, which are handy with relational data. I come from the world of normalised databases where repeated data is bad design. Whilst creating my app I stuck with the normalised database model. As the application has become more complex the datastore persistence framework has allowed me to progress without putting any further thought into the datastore. The framework allows you to define an entity with joins to other entities. However, in order to retrieve these linked entities the datastore will perform a completely seperate retrieve as soon as they are referenced.

To take a forum example: if I create a forum post my user will be stored on that post. The post will be stored against the topic etc. So you may have post.topic and post.user fields referring to other entities. In my controller I can retrieve all posts for a topic and pass them to my template view. Then in the template view I can display fields like post.text and post.user.name – herein lies the problem. If I have a topic with 50 posts and each post shows the users name then this will cause 50 seperate calls to retrieve each user individually as they are referenced. Whilst the datastore is fairly fast and very scalable it is expensive to call both in terms of time and processing power. Because of this Google also have quotas on the number of calls you make and start charging when they are exceeded. Most likely the free quotas they give you will be adequate but if your site takes off it could start getting expensive.

The mantra is: take the pain in the updates so that the reads are cheap. Because of this I have started de-normalising my database and it feels terrible. Continuing with the example above I would add the user name to the message table. That way I do not need to access the user table at all when displaying a topic. If a user changes his name then it all goes a bit pear shaped but the idea is, is that this does not occur too often. Take the hit with the update to ease the read.

For the ultimate in speed and scalability Google say that a single entity should match what is required in the view. I can’t envisage any application where this is even remotely possible so you have to take a pragmatic approach.

I have had to look at every join I have defined and at how I reference the joined data and then take a view on how often it is likely to change before deciding on whether I duplicate the data. It is a painful process but will pay off in the future if Im lucky 🙂

The next problem area is data migration with new software versions. I might see if I can make use of this somewhere: GAE BAR

Not convinced on the name of that one though 😉 It might be too slow for migrations on a live system so maybe this one: GAE Migrations

With a complete lack of personal time and the long awaited Linux version of AIR still awaited (come on Adobe, the Mono boys will get Silverlight running on Linux before you get a proper non Beta release out) I’ve parked development of it.

Also, having found on-line guides like this one: Sky TV Listings, I hardly use my own application. They’ve done a great job of producing almost everything I have all in JavaScript in a browser. Very nice!

I think I shall move on to mobile apps now. I fancy an Android phone but can’t really afford the G1. £40 a month for 18 months. Thats an expensive phone. An iPhone would be nice but equally expensive and I don’t like the fact that you have to pay Apple just to get your own application on your own phone. I’m hoping for a few more Google phones next year, then the price will come down a bit.

Haven’t posted in a while. Been very busy working long hours on site implementing a new software system. I’m getting code withdrawal symptoms 🙂
Haven’t had any time to do any coding at all on my own side projects.

About a month ago a Dell Inspiron 9400 laptop I had bought 53 weeks before, started gettng graphics problems, 1 week out of warrranty. The BIOS loading screen had many artifacts and Vista would not boot except in safe mode. The Nvidia 7900 GS Go graphics card had given up. I read around and found a few people with similar problems so thought I might have a case under UK consumer law as the laptop was not fit for purpose.

I then spent over a week corresponding via e-mail with Dell support. In all fairness they always responded the same day and were always very professional and polite. But that was the only good thing. They obviously use a decision tree for all support and cant vary from it. They requested I run all manner of tests which I did. All tests pointed to the graphics card. The card in this system is not soldered to the board and can be replaced. Dell told me that the system was out of warranty and that I could pay for the repair which would involve replacing the motherboard and memory.

Oddly I refused this option as, aside from their unfathomable diagnosis, I believed the machine was not fit for purpose. Dell then requested I perform further tests and insisted I would have to pay for the repairs. This went on for a week while I continued to insist they repair it free of charge. They finally told me to run the system restore facility which I flatly refused to do as I could guarantee it would have no effect and I asked to escalate the matter to someone with the authority to decide whether Dell would cover the cost of the repair.

The next day I received an e-mail saying Dell would, as a one off good will gesture repair the laptop free of charge! It was quickly collected and returned a week later all fixed. Result! I knew it was a heat problem so I installed I8KFANGUI and now run the fans at max all the time. It’s noisy but always keeps the GPU temp below 70.

This was a couple of weeks ago.

In my regular trawl around the tech sites I have just seen this exact issue really blowing up in Nvidia’s face:

Potentially 18 million affected units from the big manufacturers. Wow.
I know that my laptop could go again as the replacement part still has the manufacturing flaw but at least I managed to get it fixed outside the warranty period. What is Dell going to do? Replace all these machines? Their interim fix of a BIOS update to turn on the fans can’t be their final solution can it?

Another odd piece of data from the Radio Times website meant that a film director had HTML code in the datastring. This screwed up the XML which then failed to parse. I have now added some code to strip out problematic data like this.

When they released the new version of Ubuntu I fancied a clearout so I copied my whole home folder to a second disk and did a fresh install of Hardy.

USB keyboards aren’t handled very well. It’s a good job I wanted the installation to be in English because I had no way of changing it on the very first screen. It’s just as well it assumes English after a couple of minutes.

The install went well. Firefox 3 is very nice. Then a few problems started appearing:

Reboot doesn’t work, it just hangs when it’s finished shutting down

Sound is flakey. This is a surprise as it’s bread and butter stuff. Apparently they have chosen to use PulseAudio by default to control desktop sound. This is known to be dodgy in it’s current version. The result is that playing videos, or sounds sometimes runs incredibly slowly, appearing to hang. Where possible you can select to use alsa instead and it all works again! This meant I had to switch from Totem to MPlayer for videos.

USB drives do not get mounted. My card reader doesn’t work, and when I plug my camera in it does not get mounted. The workaround is to plug the device in and use dmesg to find the device name when it gets recognised (sdf in my case). Then you can use “sudo fdisk -l” to find the file type of the USB device (FaT32 in my case). Finally you can create a folder to mount the drive to and then manually mount it with a “sudo mount -t vfat /dev/sdf1 /media/camera/“. When you’ve finished you can run “sudo umount /dev/sdf1” to dismount it.

These are the only problems so far but they aren’t exactly unique to my configuration. Although I’ve found workarounds to every problem it’s a bit naff really. And this is an LTS version as well!

I have for some time been plagued with problems when web browsing, particularly on sites like YouTube. Firefox would simply die without warning and I would have to restart it. Fortunately, it allows the session to be resumed so I lived with it for a while. As with a lot of minor issues it started getting really annoying so I googled around for it. It seems that quite a few people are having problems with Flash and Ubuntu.

Navigating to the “about:plugins” page (literally typing that into the address bar without the quotes and hitting enter) I noticed that Firefox was reporting 2 Shockwave Flash Plugins. One libflashplayer.so at version 9.0 r115 which is the latest version and a second libflashplayer.so at an earlier version.

I searched my hard disc for files with this name and it came back with two of them. One in my ~/.mozilla/plugins folder (where ~ is my home folder) and one in /usr/lib/flashplugin-nonfree

I didn’t know which was which so deleted the version in my home folder and restarted Firefox. The plugins page then only showed the latest version of the Flash plugin.

Since then I have not have a single crash in Firefox. I’m still not sure how the older plugin version came to be on my system.

If you have Firefox 2 problems with Flash try this out. It seems a bit of a simple solution so there may be other factors involved but it’s worth a shot.

I haven’t rebooted into Vista for several months now. I might try installing SP1 if I get bored 🙂

I haven’t updated this site for ages due to free time issues but I have finally managed to update the application for the released version of AIR. I have also taken this opportunity to open source the whole lot (except for the auto updater code which isn’t mine).

You can find the latest version of the application (0.5) from the AirWaves page linked on the left and also in there is a new link to the whole FlexBuilder project. You may find it useful if you want to see how to do transparent windows or minimise to tray etc. Or you could fix a few of my bugs (the date time drop down isn’t working properly…) or enhance the functionality if you fancy it but please follow the GPL license restrictions (basically any derivative works have to be GPL’ed as well, with the source available).

I have noticed that any TV guide relying on XMLTV has been broken recently due to null characters in the Radio Times data, but as I’ve written my own data formatter it doesn’t suffer from this problem 🙂

Finally I have gotten around to upgrading AirWaves TV Guide to the latest version of the AIR runtime: Beta 3.

Unfortunately they have changed the application descriptor so that the version number is no longer an attribute of the top level node but rather an XML node in it’s own right. Basically this means that the auto updater will no longer work. So if you want the latest version you will have to download the AIR file and install it manually. Hopefully they wont break it again as it must be getting close to a release candidate version soon.

I have also tuned the classic EPG style programme view a little further. We are now down to about 3-4 seconds and way less memory consumption.

I had a quick look at the Android SDK the other day and immediately hit what appears to be a common problem.

ActivityManager: Error: Activity class blah does not exist.

All I did was create a new Android project and run it without any customisation code at all. I then spent an evening searching for a solution but nothing appeared to work. I tried re-creating the project but since I hadn’t modified the project in any way this was unlikely, I tried killing the debug process “{SDK_Location}/tools/adb kill-server”, deleting the userdata.img file. No change. I finally found a post suggesting to login to the adb shell and remove the deployed applications manually.

So I ran “./adb shell” in the SDK Tools folder and then cd’ed into the /data/app folder. I found in there all the old projects that I had created and run, i.e. several apk files. I deleted each one leaving the ApiDemos.apk which comes with Android. When I ran my project from Eclipse again, it started working 🙂

I am assuming the issue was caused as I had created a previous project for trying stuff out and subsequently deleted it and somehow adb got confused along the way.

It’s all fine now though so all I have to do is think of a killer app 😉 Anybody want a mobile TV Guide? Maybe I could sms back to MythTV to record shows???? That positioning LocationManager looks cool as well, but how’s it going to work on a security level? Will a user opt to have their location tracked?