Bruce's Blog

I’ve now been in Seattle for almost a full day. The flight was great, except that I couldn’t sleep properly. I’ve got a car (Toyota Corolla) and an apartment (200m from work :-/), but I’ve managed to bust the internet already and I have to wait until tomorrow to get it fixed. Luckily they have free WiFi in the guest lounge as well, so thats where I am right now

Interesting Points on Day one:

A fenced off area where you can look after your dog at LAX airport. Why would you take your dog to LAX? I especially like that they have put a hydrant in there, just in case your pooch is picky in what he pisses on.

I saw an Amazon Fresh (the place where I’ll be working) delivery truck while I was out getting a feel for the neighbourhood. Good to see that presence!

Bicycle polo! Played in the park in Capitol Hill.

And finally, I saw a guy shouting at himself “Don’t forget me, motherfucker!” over and over. At least there’s one thing that reminds me of Melbourne :)

With the formalities now over, I turn my attention to my favourite subject: Me. I jet off tomorrow for the United States, where I will be taking up a position with Amazon, working on Amazon Fresh. Melissa will join me in a few months. We’re very excited by the opportunity to live and work in a new place, and to work with one of the world’s top tech companies.

If you have any suggestions for things to do in Seattle or weekend trips that we can take, I’m all ears :)

I’ll miss Melbourne, and I’ll miss the friends I have made here. I don’t think the move is permanent. We are planning on returning to Australia in a few years with a bunch of awesome stories :) Viva las aventura!

I’m posting this little snippet up because I spent ages trying to work out how to do this, and thought that other googlers might benefit from this.

I’ve got an Android application that stores some of its information on the SDCard which has some commercial value, and we don’t want our competitors simply walking away with the information. The security doesn’t need to be too tight, so I’m happy to have a password based encryption scheme which has the password in the code of the application. If a competitor really wants to get at it, they can, but it will stop casual theft of the data.

To do this, I have written a simple little script in ruby to generate and then encrypt the data files. I dind’t want to write the encryption code in java too, so I needed something that would work across Java and Ruby. OpenSSL to the rescue! OpenSSL provides a simple little command line tool to encrypt files. So, to encrypt the files, I used this snippet:

I could have also used the built in openssl gems of ruby, but I couldn’t work out how to get it to save the salt/IV in the file as well as the encrypted data, so I just left it as a command line thing. Perhaps somebody can provide an improvement.

Android provides a pretty comprehensive encryption library through the BouncyCastle Project. The documentation isn’t very helpful though, and it took me a while to work out the correct combination of parameters to decrypt it on the Java side. Here’s the result:

Security.addProvider(newBouncyCastleProvider());byte[]encrypted=read(args[0]);// Whole encrypted file.Stringpassword="ThisIsMyPassword";Cipherc=Cipher.getInstance("AES/CBC/PKCS5Padding","BC");// Openssl puts SALTED__ then the 8 byte salt at the start of the file. We simply copy it out.byte[]salt=newbyte[8];System.arraycopy(encrypted,8,salt,0,8);SecretKeyFactoryfact=SecretKeyFactory.getInstance("PBEWITHMD5AND128BITAES-CBC-OPENSSL","BC");c.init(Cipher.DECRYPT_MODE,fact.generateSecret(newPBEKeySpec(password.toCharArray(),salt,100)));// Decrypt the rest of the byte array (after stripping off the salt)byte[]data=c.doFinal(encrypted,16,encrypted.length-16);

The Key line is the SecretKeyFactory.getInstance() call, which sets up the cipher to accept the openSSL data. Once we’ve got that, its trivial. Nothing to it really, once you know the magic string!

I’m not a very good graphics artist/web designer. That doesn’t stop my trying though, as you’ll see through this site, and a plethora of others. I’m getting better all the time, but I’m still not up to a decent grade. This article is about one of those “getting better” moments, and I thought I’d share.

One of the things I like my sites to do is to behave differently on different sized screens, mobiles in particular. Often this is wrapped into the concept known as Responsive Web Design, but really its all about making your sites work on teeny tiny screens. So far, I’ve been doing this using CSS Media Selectors, which delivers different CSS styles to the browser depending on helpful information provided such as screen size. Although some people say that CSS media selectors are fools gold, I’m more inclined to think that they are sufficient for the majority of cases.

The basic principle is that when you create your CSS, you include some additional hints to the browser as to when it should be applied. Below is the usually accepted method, in which you place the following HTML tags in the <head> of your document

The idea is that you have your standard CSS first, which applies to the desktop version. The mobile version, which is only matched when the browser passes the media test, will override any settings that need to change the site to work properly on mobiles. This is the most basic media selector. You can also look at the orientation (landscape/portrait), the media it uses (printer, screen) and so on. I use this on this site, and in much greater depth on the jewellist’s portfolio site.

The problem is, they’re inconsistent, and they don’t always work. Many devices lie about these properties to try and get the right experience for users. In the above example, we’ve specified that the style will match on devices that have a maximum width of 480px. An iPhone4 has a maximum resolution of much wider than 480 pixels, thanks to its lovely retina display. In order to give users a consistent experience however, apple has made the iPhone lie about its resolution. Some browsers (notably Windows Phone 7) don’t even respect the media selector attribute. This means that developers that want to use these queries have to understand the quirks of the devices rather than have meaningful tests.

In an even more dangerous turn though, there’s been a bug in the Android implementation of WebKit (its built in browser) which has meant that it sometimes decided it wouldn’t apply the style sheet until you refreshed the browser. This has infuriated me for some time now, but I have found a workaround, thanks to this lovely post.

To make it work correctly on Android, you have to add an additional query parameter, which also matches any screens based upon their window width, in addition to the device width.

This is a hack to be sure, but it does work. It also shows another challenge to using this method. Notice how I’ve changed the max-device-width parameter to 854px? Thats because we want it to work on landscape orientations too, ones that don’t lie about their resolution like the iPhone does. Some Android phones have lengths of 854px. This works great for current devices, but what about newer devices that have higher resolutions. The next generation of phones are likely to have 1280x720px screens, so these pixel based approaches will once again fall down (unless the phones lie).

All of this seems to push me in the direction of using User-Agent matching to deliver different content, rather than using CSS selectors. Most big sites seem to take this approach, having a www.blah.com site and a m.blah.com site. This also gives you the advantage of allowing users the option to switch over to the desktop version if they choose. It makes sense, but requires more server side logic, which kind of goes against my static website philosophy. I’ll need to look more into that…

As I am presenting to MobSIG on tuesday about Android Widget programming, I thought I should put together a slide pack. The session will be mostly coding, so there’s not too much to it, but here are the Slides anyway. I’ve decided to do the slide pack totally in HTML5. The skeleton for the slides was shamelessly stolen from HTML5 Rocks. I hope that in the future, I’ll be able to tweak the presentation a bit more to make it work really well, and fit in with the theme of the site.

Please note that as the slideshow is written in HTML5, it won’t work with older versions of browsers, in particular Internet Explorer. If you really want to see the presentation and you use IE, let me know.

The links that are important for the presentation are also included below:

As I mentioned the other day, I am moving on from Unico and becoming an independent consultant (a grandiose title I give myself… Really I’m a contractor). To coincide with this, I am also revamping the layout of my site, with additional information about the sort of work that I do, community engagement, and how to get in touch. The site should now render better on mobile devices as well.

Is it an improvement on what was there before? Maybe. Hopefully I’ll be tweaking it more in the coming weeks.

I have been asked to present a talk to the Melbourne Mob SIG, to be held at the Telstra Conference Centre on the 2nd of August. It will be a technical presentation, where I open up eclipse and show people how widgets work on the Android platform. This could be considered a basic topic, but it is one of the most requested topics on the Build Mobile site, so I thought it’d be a good topic.

If you have any other suggestions for presentations, or articles that I could write for BuildMobile, please let me know.

After four years working at Unico Computer Systems, I handed in my resignation this morning. I have been working in the enterprise space performing integration architecture and devleopment work. I have found however that my own interests have shifted more and more towards working with mobile devices, and this has become a real passion of mine.

As a result, I have now accepted a contract to work in the logistics industry working with mobile devices to make truckies lives easier. In the long term, I hope to grow this work into a business working across the mobility industry, working on the (rapidly dissapearing) boundary between enterprise systems and mobility solutions.

I’d like to thank Unico for its support over the last four years. They gave me a job when I moved to Melbourne, and I have learned a lot in the period that I have been here. Furthermore, I have made a lot of friends, whom I hope to be able to continue to work with in the future.

So the speculation was wrong. Its not terribly surprising. Requiring people to buy a new device to sync their media would have been an impost, and Apple have a brand new shiny data center which will do the job nicely. Its a shame however, I was hoping for something a bit more personal. Wishful thinking!

Opinions seem to be that whilst its a welcome addition and will integrate seamlessly, its hardly anything revolutionary: Just well executed.

Rumours are circulating before WWDC about how Apple will be supplying its iCloud service using a new version of its Time Capsule router come backup device. The idea is that the new version will contain a processor similar to that found in the iPhone and iPad and that it will run iOS and apps.

The ability to sync my apps and music using iCloud sounds great, but I’m beginning to get excited about the other opportunities that a device like this would bring. Would a device like this finally provide the impetus for people to have a personal server in their homes? People have long tried to get this sort of capability going, originally through their own PCs but more recently through PVRs, SmartMeter devices and appliances such as the PogoPlug. None have had the business model or the customer appeal to really gain a foothold with consumers. If this rumor turns out to be real, it smells like it has the possibility to work.

I have ranted recently about the possibilities of having collaborative apps (agents) running on “the cloud”. One of the advantages I postulate is the ability to take ownership of your data again. The idea is that each person have their own stable of applications/servers that can do things in the background but don’t have to run directly on the person’s phone. I think an app enabled Time Capsule provides exactly the right sort of platform to perform these tasks.

Hopefully, the way it works is that when you install an app, companion apps automatically get installed on the time capsule, or there is a way an app can direct companion apps to be installed. The app on the Time Capsule does all the repetitive, background and possibly battery draining tasks, so that the mobile devices don’t have to. If they can make the cooperative nature of the apps seamless, then this will be a game changing, “magical” move by Apple.

So does Google have an answer to this brave new world? Possibly. We know that Google are consolidating their Android and GoogleTV products to form a single code base, so the GoogleTV could provide these capabilities. Thats not supposed to happen till the end of the year, and the rumors claim that Apple have their solution is ready to go now. I think that Apple might really have a game changer on their hands here, and I’m excited by the prospect. What remains to be seen is how open they make it for 3rd party developers. If it is limited to the apple provided applications, then it won’t be anywhere near as exciting. Perhaps the next must-have application will be a TimeCapsule?

For those that are interested in the topic, I suggest looking at the video of Steve Jobs’ thoughts on the matter from 1977 in the rumor article from macrumors.com).

I have been playing around with the ideas of mobile processing agents lately, and to do this I need to execute code that has been uploaded to my processing environment, rather than the traditional “deployed” code. Java has this ability built in, via ClassLoaders. At any point, a java application can create classes from byte arrays. I wanted to do this on Google App Engine, but GAE does not support the traditional methods of loading classes would work. There are no “Files” or direct access to URLs in GAE. The only real data store available is the Data Store, which provides a HTTP based file upload and storage service through its own proprietary interface.

Creating a ClassLoader which took advantage of the Data Store wasn’t difficult however. I made use of the (currently experimental) File Store API to access the code once it has been uploaded to GAE’s datastore using the file upload facility. Once I’d gotten that sorted out, it was simply a matter of writing the classloader to read in the data.

To make things a bit more interesting, I’ve also added an AppLoader, which can set up a classloader for a contained application with extra JARs and a manifest, similar in structure to a WAR file. For example, if we uploaded a JAR file to GAE with the following structure

The AppLoader would construct a classloader with the contents of the classes folder, plus any JAR files that are in the lib dir. The agent-inf.yaml file specifies which class is the “main” class for the agent, which will be used for execution.

There is one limitation to the classloader that I’ve written. Sometimes java code refers to non-class files that are stored on the classpath, usually referred to as Resources. They are loaded by the Class.getResource() and Class.getResourceAsStream() methods. getResource() returns a URL which points to the resource asked for. The problem that I have is that GAE does not support URLs properly, and certainly doesn’t allow you to have your own URL handlers. As a result, getResource() does not work in my classLoader. This may break some libraries. getResourceAsStream() does work however.

One final warning about isolation and Classloaders. Giving people access to upload arbitary code to your app server is a dangerous undertaking. Make sure you understand who is uploading classes and make sure they can’t do anything they shouldn’t be doing. In my agent code, I want each of the agents to be completely isolated from each other. The challenge that I have is that each one, if it has access to the DataStore API, will be able to overwrite the others, which isn’t cool. I’m still thinking about ways that I can fix that.

In case anyone else is looking for a similar solution, I thought I’d post it here. The Code for this example is available at my GitHub Repo

Now, back to writing that mobile processing agent system…

Categories: google, appengine, java, code example, technical and not for you richard

Sitepoint.com has just launched a mini-site dedicated to the builidng of mobile applications in all their forms, named BuildMobile. Inexplicably, they have chosen to feature my little application NodeDroid as their first featured app, and even more explicably they have asked me to contribte some stories as well. The featured app is up now, and my article will be posted in a few days.

There’s nothing like having a wider audience (not to mention a deadline) to inspire you to write, so hopefully I’ll be a bit more regular with my postings there than I have been here :)

When I started writing NodeDroid, I did it to learn about writing mobile applications, and all of its associated technologies. One of those technologies was advertising. I added Admob advertising to the bottom of the application, and wated for the megabucks to roll in :). In the little over 3 months that its been available, it has netted a grand total of $13.88US. Thats enough to cover hosting costs, but thats about it.

A number of people have asked if I can support their provider. I have variously been asked to support Optus Cable, Exetel, Telstra, and a bunch of others. In order for me to do this properly, I’d need to have access to an account to be able to perform testing. People have quite understandably been reluctant to share this information with me, which means we are stuck with the providers that NodeDroid already has.

I’d like NodeDroid to support as many providers as posisble. I’d like it to be better than consume, the iPhone application. I don’t think I can do that on my own. As a result, I’ve decided to open source NodeDroid, in the hope that other geeks out there will write their own providers for their own provider, and then contribute the code back to the application.

I can’t ask people to work for free on something that I could make money off (even if it is a paltry amount), so I have also decided to remove all advertising from the application. I just uploaded a new version that has no admob.

The code for NodeDroid can now be found on GitHub at the NodeDroid Repository. If you are a developer, please feel free to check out the code, and get in contact with me. If you have a bug that you want to report, you can also report issues there too.

I’ve been playing with jekyll to create my website over the past few days. Primarily, I’m doing it to play with Ruby, but its also nice to have a new website :)

Jekyll produces a static site, but does this using templates and markup. Its remarkably easy to set up a site, and to give it the look and feel that you want. Any dynamic capability can be provided by external services (e.g. I use disqus for comments) and javascript.

One thing that I wanted to do was have a post tree in the sidebar. By Post Tree, I mean a tree showing all my previous posts, broken down by year and month. Have a look to the right here and you should see it. Jekyll doesn’t provide the ability to do it out of the box, but it is very easy to extend, so I thought I’d write a plugin to do it.

Here’s the code I wrote. The first file is the ruby plugin, which would be placed in _plugins/postsintree.rb . Its responsibility is to set up the data in a format that is easy for the template to output:

moduleJekyll# Extends Site to have an field that gives you a map of posts by year.classSitedefpostsbyyear# Create a tree of the posts by year and then monthtree={}self.posts.eachdo|post|year=post.date.yearmonth=post.date.monthiftree[year]==niltree[year]={"number"=>year,"count"=>0,"months"=>{}}endiftree[year]["months"][month]==niltree[year]["months"][month]={"number"=>month,"name"=>Date::ABBR_MONTHNAMES[month],"count"=>0,"posts"=>[]}endtree[year]["months"][month]["posts"]<<postend# Turn the tree into sorted arrays, so it is easier to interpret# in liquidyears=tree.values.sort{|x,y|y["number"]<=>x["number"]}# Calculate counts of posts and sort each of the months as wellyears.eachdo|year|year["months"]=year["months"].values.sort{|x,y|y["number"]<=>x["number"]}year["months"].eachdo|month|month["count"]=month["posts"].sizemonth["posts"]=month["posts"].sort{|x,y|y.date<=>x.date}endsum=0year["months"].each{|month|sum+=month["count"]}year["count"]=sumendreturnyearsend# Redefine site_payload to include our posts by year. This is ugly# but I don't know how else to do this without changing the jekyll code# itself. #rubynoobdefsite_payload{"site"=>self.config.merge({"time"=>self.time,"posts"=>self.posts.sort{|a,b|b<=>a},"pages"=>self.pages,"html_pages"=>self.pages.reject{|page|!page.html?},"categories"=>post_attr_hash('categories'),"tags"=>post_attr_hash('tags'),"postsbyyear"=>self.postsbyyear})}endendend

Now that we have the data in the right format, its just a matter of altering our page template to show the tree. This is facilitated by the following HTML:

Earlier this month, I posted a new website for my hobby company 8bitcloud.com. As part of this change, I have now decided to host my personal blog here as well. My Blog will henceforth redirect to http://8bitcloud.com/blog.html. All URLs will continue to retrieve the correct posts, but with the new layout. There shouldn’t be any disruption to service, but who knows. RSS feeds should automatically switch over as well. I apologise if it re-posts everything I’ve ever done…

So why have I done this? Blogger is a great platform, but I’ve been experimenting with jekyll as a website creation method, and thought I’d try it out for a bit. If I don’t like it, I think I’ll move to wordpress, which I’ve been mulling over for a while, so perhaps this is a simple way of testing it out. In the end, I did it so that I had something new to play with. Yeah, I know… I’m sad…

This site is primarily intended to allow me to play around with new stuff. As part of this, I’m playing with Jekyll, which is a simple, ruby based web generator which creates static sites programatically.

Some advantages:

Simpler hosting requirements. All you need is a web server like Apache

If you want dynamic capabiilties, they can be added in via javascript.

Will be able to survive a slashdotting (not that that is ever likely to happen to me)

Not hackable through attacks on the CMS product, because there isn’t one

Uses Markdown, which makes writing the content much easier. You can always fall back to HTML when you need it though

Want to cross-post? Thats easy. Simply symlink between your different blogs and re-publish.

This also gives me the opportunity to replace the old design for 8bitcloud with a new one. I really enjoy doing web site designs. What a pity I’m no good at it :)

I'm the author of a usage tracking utility on Android called NodeDroid. Originally it only supported Internode, but I am now expanding it to support other ISPs and telcos. The first one I want to try out is Optus. I've just uploaded the new version of the application, and I'm hoping some of you guys would like to try it out.

The application works by screen-scraping the optus web site, and presenting it in a better format, along with usage graphs and the normal sort of thing you would expect from a usage meter. In the future, I hope to add the ability to see bills, as well as provide warnings when your quota is being reached. That sort of thing.

Because there are a large number of plans, and they all have different rules, I would imagine that I will need to take into account a large number of factors. At the moment, I've only been able to work from my own plan, which is an Extreme Cap. It should work quite well for other Caps, but it will probably break on prepaid and other account types.

If you would like to participate, you must already have a log in to the optus account page. If my application can't read your usage, it would be useful for me to see what your usage page on the optus portal looks like, along with all the usage lines expanded. If you can provide a screenshot or source, that would be beneficial (remember to black out the phone numbers first though!!)

If you would like to participate, please fetch the application from the market by searching for NodeDroid, or through AppBrain Here

I will also be expanding the program to support other providers in the near future. Vodafone Mobile prepaid broadband will be easiest for me, as I have an account, as well as Telstra prepaid, but if you are extra keen on getting something, please send me a whim.

This weekend, I thought I'd extend my little Android usage tracking application to work on more ISPs than the one (Internode) that it already does. As my phone is (sadly) on Optus, I thought I'd write one for that.

Internode was easy to add, as they have a documented API for accessing usage counters which are ideal for computer consumption. Optus on the other hand only provide a web application interface to check usage, necessitating the use of a web scraper. A web scraper is an application that pretends to be a user on a web page, makes all the appropriate calls (and fudges any javascript calls that are necessary) to get the results it needs. It then parses the (often non-compliant) HTML results that come back to get data. I have no problem doing this, and have done so on several occasions before, but it is not easy work and can be quite fiddly. Parsing the HTML is often the most difficult part, as it is usually not well formed XML so you can't just use DOM to parse it.

In short order, I had a working prototype that used JTidy to clean up the HTML into something that I could use properly and then XPath to extract the elements of the document I needed. It works great, except that the document clean up and parsing into DOM takes a really long time on a resource constrained device such as a phone. It takes about 20 seconds to clean up and parse the document on my development emulator, which is too slow to produce a good mobile experience, especially if you have to parse multiple documents as I do.

So now I'm faced with a choice. I could write a man in the middle service that the phone sends the user's login details to which then performs the parsing on behalf of the user and sends the results on to the phone, but there are a number of drawbacks to this:

This means that the user is sending his login details to a 3rd party, which is a security no no.

It introduces a single point of failure into the equation. If my app gets popular the middle man service could get slammed. If Optus decides that they don't like what I'm doing, they could easily block it.

It means I need to host a service, which means additional expense.

I don't want to do this, so what I'm left with is more hacky solutions, using regular expressions to find what I want in the HTML documents retrieved from the provider. This will take me longer to code, will be more prone to failure, and is just generally nasty. I'm not happy. Devices these days are very powerful, and there should not be the need for intermediary servers to help with processing.

Of course this would all be much easier if the providers published web services interfaces to their data, rather than just web applications. This has been the mantra of SOA and internet connected businesses since the terms were coined. It doesn't even cost them that much more to do it, and would lead to better designed web applications, but thats a subject for another rant. Optus doesn't do this because there's no economic incentive for them to do so. They gain nothing directly from publishing a usable web interface, so they can't be bothered... bah!

To be fair to Optus, they aren't the only ones that don't get it. No ISPs and telcos provide any decent interfaces, other than Internode.

I purchased an Android phone a few weeks ago. Part of the reason that I got it was that I wanted to see what the differences were between Android and iPhone. This extends out to how to program them as well, so I had to write an application, just like I did for iPhone.

Last night, I released my little application. Its a usage meter for my ISP, Internode. I deliberately chose something quite simple so that I could cut my teeth on the platform, and I must say that I'm very impressed. I found it very easy to write the application for Android, especially because it uses the same tools and libraries that I am used to using for my day job. One other difference that I notice is that it is much more obvious what is going on inside an Android application. The documentation describes things clearly and gives you full visibility. Apple on the other hand like to keep their platform a little more mysterious. There are plenty of good documents on how to do things, but you still get the impression that there's something going on under the hood that you don't quite understand. Perhaps its just that I'm more familiar with the Java ecosystem.

If you'd like to have a look at it, check out its site at 8-bit cloud. I've also been playing with the web site, and I will be improving it and hopefully making it more fun.

PS: I was tempted to call the application NodePony after a recent meme that Internode has got going with its cute little plush toys. In the end, I decided that I shouldn't try to cash in on it. It would spoil the meme...