Wednesday, November 16, 2011

Back to blogging after a break from it as I was too busy the last few days... with some great news!

The Apache Software Foundation and the Apache HTTP Server Project have announced the release of Version 2.3.15-beta of the Apache HTTP Server (also known as "Apache" or "HTTPD"). This version is the 4th, and likely final, beta release before the general availability of the 2.4 release, with lots of interesting new features, described there.

The following will be particularly useful if you're using HTTPD server to run apps in the cloud:

So, with HTTPD 2.3.15 you get a more robust and faster HTTP server with load balancing for horizontal scaling in the cloud! I've been playing with the 2.3.* code for some time, and it has been working really well for me. It'll be great to finally get a 2.4 GA release with all these new cool features! Hopefully soon...

For more info, config examples and performance data, see this presentation from Jim Jagielski.

That'll build two useful command line tools: minify_html_bin, an HTML rewriter / minifier which also minifies inline CSS and Javascript, and jsmin_bin, a Javascript minifier which also works well for CSS.

I won't go into the Automake specifics now but these rules will run the Page Speed minifiers on your Javascript, CSS and HTML pages as part of your Make build. I'm using a simple naming convention for the minified files, index-min.html is the minified version of index.html.

John Mc Carthy passed away yesterday. After Steve Jobs and Dennis Ritchie earlier this month, October has been a sad month for the computing community...

Another good article from Paul Graham, 'The Roots of Lisp' helps explain John Mc Carthy's discoveries with Lisp.

If you don't know Lisp, you should learn it.

I had a brief look at it 15 years ago, but was too impatient and unexperimented to get the point then. I took more time to actually learn it on a summer vacation 2 years ago, then went on to learn Scheme (a Lisp dialect) as I was going through the Structure and Interpretation of Computer Programs classic Computer Science book from MIT.

Believe me. Lisp (or Scheme) will transform you into a different programmer. After that, even if you still have to program in C, C++, Python, or even Java, you're in a different world. You'll see programming through different eyes, and there will be no going back.

Think about a few fun facts... Pretty much everything you manipulate in Lisp is a list of values (Lisp stands for 'list processing'). Values can be numbers, strings, symbols or lists. Data and code are interchangeable and represented in the same way, as lists of values. You can write a Lisp interpreter in half a page of Lisp!

So, if you're working with 'modern' OO programming languages, scripting languages, or functional programming languages, if you're using XML dialects, JSON, DSLs, and you're still struggling to map between documents, data, objects, services, code and configuration... spend the time to learn Lisp. It'll open your eyes.

Thursday, October 20, 2011

Last time I showed how to use XMLHttpRequest and local storage to download and cache Javascript scripts. Here I'm going to show how to download and cache an image, and then inject it dynamically into your HTML page.

That technique can be particularly useful to optimize an image intensive app (a game or a photo gallery app for example) and allow it to work offline.

Configure your Web server to serve .b64 files as text/plain. If you're using the Apache httpd 2.x Web server -- and if you're not using it, you should :) -- add the following to your httpd.conf server configuration file:text/plain b64

The reason why we want to convert our images to base64-encoded is that it'll make it much easier to use them in the HTML5 app pages. We're getting to that part now.

Place the following Javascript script in your HTML page, under the <head> element for example:<script type="text/javascript">
window.appcache = {};

I'm not showing the error handling code to keep it short, but you get the picture. The get(url) function downloads a resource and caches it in local storage. The img(url) function gets an image resource and converts it to a data: URL. A data: URL (as defined in Internet RFC 2397) allows you to include resource data (here our base64-encoded image data) immediately in the URL itself.

Now, later in your HTML page, say you have an <img/> element like this:<img id="foo"/>

You can set the cached image into it dynamically, like this:<script type="text/javascript">
var foo = document.getElementById('foo');
foo.src = window.appcache.img('/foo.b64');
</script>

The image src property will recognize the data: URL, the image/png;base64 media type at the beginning of the URL, and read the base64-encoded content as a PNG image.

That's it. With the little trick I've described here you should now be able to:

control the download of your images;

cache them in local storage for speed and working offline;

inject them dynamically into your HTML page as needed.

Hope this helps. In the next few days I'll show how to handle CSS style sheets in a similar way.

Sunday, October 16, 2011

The global game industry will generate $60 billions in revenue for 2011. It's not a surprise, but gaming is the only media business growing right now with a rapid growth driven by mobile games, as games are the leading apps on smartphones and tablets.

The game market seems to be changing rapidly from hardcore gamers (mostly teenage boys) to casual gamers with older households (35+yr old), and women (42% of the gamers now), as browsers and mobile are bringing new populations to casual and social games.

The game technical platforms are also shifting quickly to the Web browser. With technologies like HTML5, Canvas, SVG, and WebGL for example, Web browsers have become the most convenient platform for games, and it's now clear that games in browsers are the future. See this interview of the founder of Electronic Arts for more insight and his thoughts on what's happening.

Clearly, that means challenges ahead for the established game companies which have to adapt quickly. That also means great opportunities for new players to come and disrupt the mobile game market!!

Apache Callback is the free open source software evolution of the popular PhoneGap project.

Apache Callback is a platform for building native mobile applications using HTML, CSS and JavaScript. Apache Callback allows Web developers to natively target Apple iOS, Google Android, RIM BlackBerry, Microsoft Windows Phone 7, HP webOS, Nokia Symbian and Samsung Bada with a single codebase. The Callback APIs are based on open Web standards and enable access to native device capabilities, including the device camera, compass, accelerometer, microphone or address book for example, from HTML5 apps.

That's really great news for the mobile app developer community! ... and I'll be watching this project very closely in the next few months!

His book "The C Programming Language" was my first Computer Science book. My first serious encounter with software was a 2-week job to teach the C language to a team of programmers. I was just trying to make enough money to go on vacation, but then I got hooked and decided that I wanted to be a C coder... Feeling sad today.

The HTML5Rocks article showed how to follow all the links in a page and pre-fetch and cache their target pages in local storage. That was a good starting point, but for a mobile Web app to really work well offline you also need to fetch and cache the referenced JavaScripts, CSS, and images.

Here's how I do that in my apps.

I place the following utility Javascript at the top of my page under the <head> element:<script type="text/javascript";gt;
window.appcache = {};

I'm not showing the error handling code here to keep it short, but you get the picture. The get(url) function downloads a resource and caches it, and the script(url, parent) function gets a resource and creates a script element with it.

Then to reference a Javascript script later in my page, instead of writing the usual:<script type="text/javascript" src="foo.js"></script>

If you're loading pages in an <iframe> nested inside a main page as suggested in the HTML5Rocks article, you don't need to repeat the utility script in all your pages. Just have it once in the main page, then in the nested pages just write:<script type="text/javascript">
window.parent.appcache.script('foo.js', document.head);
</script>

Or just set the appcache property on your <iframe> like this:var nested = document.createElement('iframe');
nested.contentWindow.appcache = window.appcache;

Then you can write the same code everywhere:<script type="text/javascript">
window.appcache.script('foo.js', document.head);
</script>

Hope this helps.

To the folks who've pinged me several times this week asking for this tip: Sorry for the delay but I've been kinda busy... I'll show how to fetch and cache CSS (similar to Javascripts) and images (using a data: url and base64-encoded image content) in the next few posts, probably this weekend.

In that review I said:"What HTML5Rocks doesn't describe is how to fetch and cache all the resources referenced by these pages, like CSS style sheets, JavaScript scripts, or images, and that's a lot more work..."

and concluded:"My 2c... The techniques described in that HTML5Rocks post are fine to get you started, but implementing them is not so simple. As usual, the devil is in the details."

Since then a number of folks have pinged me to challenge and ask:"So, how do you fetch the referenced resources? and what are these details?"

Fair question :) I'm going to address it in a series of posts in the next few days. I'll also describe a few additional issues and tricks to make a mobile Web app really work offline, and some solutions with code snippets.

Thursday, October 6, 2011

It describes three HTML5 techniques for optimizing mobile Web performance:

For smooth native-feeling sliding and flipping page transitions, use a translate3d(0,0,0)CSS transform to force the phone's Graphics Processing Unint to kick in and perform hardware-accelerated page compositing.

3D comes at a price though... Hardware acceleration can quickly drain your phone's battery, and some fonts won't look as nice when composited in 3D on the iPhone for example, so you better choose them carefully.

Manually fetch HTML pages and cache them in HTML5 local storage to speed up page navigations and enable the app to work offline (as you have all your pages stored locally).

What HTML5Rocks doesn't describe is how to fetch and cache all the resources referenced by these pages, like CSS style sheets, JavaScript scripts, or images, and that's a lot more work...

Listen to the network online/offline events and detect the connection type (Ethernet, Wifi, 3G, Edge) using the navigator.connection.type property.

Well, the online/offline events are useful but navigator.connection.type is not supported on the iPhone (only on Android), and you can't rely on it anyway as sometimes an overloaded public Wifi will get slower than 3G... What you really needed is a measure of the quality of the end to end network connection in terms of bandwidth and latency, and you can get that by instrumenting your usage of XMLHttpRequest in your client code as well as your server code.

My 2c... The techniques described in that HTML5Rocks post are fine to get you started, but implementing them is not so simple. As usual, the devil is in the details.

That's it. Like magic, you now have a super fast local DNS caching server connected to the Google public DNS servers, which are also much faster than your average Internet Service Provider DNS servers or the Root Internet DNS servers used in the default Mac OS X DNS config.

Monday, October 3, 2011

Trying to relax after an unlucky day which started with a fender-bender... So, to take my mind away from today's unfortunate events I'm hacking some Haskell code tonight and playing around with the Haskell Par Monad.

I've been exploring various aspects of parallel programming for a while now, ranging from low level GPGPU (General-Purpose computation on Graphics Programming Units) using NVIDIA's CUDA and OpenCL to parallel functional programming using Haskell (also tried F#) and Monads like the Par Monad.

Abstract:
Using an open and simple API is a great way to build a developer ecosystem and grow a user base dramatically for Web 2.x (social/cloud/mobile) applications. But building flexible APIs can be a challenge. This session covers how to build a nice Web API from scratch, using open source stacks.

Monday, September 26, 2011

Why would you want to set up an Internet Domain Name System server (DNS) on your laptop? Well, it can be handy to serve test domain names to your iPhone and test the mobile apps you're working on, or to use HTTPS on an insecure network with an TLS/SSL certificate you've created for your test domains.

DNS has a reputation to be tricky to set up, so I thought I'd share the steps to do it on Mac OS X. It's easy, when you know what to do.

Say you want to map the supercoolapp.com domain (an example test domain) and all the *.supercoolapp.comsub-domains to IP address 10.1.1.55 for example.

1) In a terminal window run the following commands:sudo rndc-confgen -a -c /etc/rndc.key
to generate the rndc.key file as it's included by /etc/named.conf.Named is the domain name server, rndc is the Mac OS name server control utility.

Monday, June 27, 2011

New York City is challenging the public to share what apps they want for residents, businesses, and visitors. After 8 days running the challenge there's already more than 300 app ideas! It's amazing to see all the app ideas that people come up with.

Imagine if creating these apps was as easy as blogging about them :)

Well, it shouldn't be too hard actually, with all the nice Web APIs that are available out there now for search, maps, location, photos, shopping etc.

Friday, June 3, 2011

Conclusion from the paper: "We find that in regards to performance, C++ wins out by a large margin. However, it also required the most extensive tuning efforts, many of which were done at a level of sophistication that would not be available to the average programmer."

And my 2c: If you care about performance, for example if you run in the cloud and memory footprint and CPU directly translate to dollars on your cloud monthly bill, if you like programming and are not afraid of spending energy to design and program efficient algorithms, then C++ is still your best bet.

Sure that requires a bit more thinking and perhaps more sophisticated programmers, but you get what you pay for.

And to all the folks who criticize C++ for being complicated, well... in my opinion it's as complicated as you want it to be. It's really up to you as a programmer to think and keep your code simple and, by the way, there's a lot of complicated Java and Scala code out there too :)

Tuesday, January 25, 2011

The last few weeks I've been exploring how to use an iPad, tablet or netbook to write programs in the cloud with just a Web browser connected to my Linux servers.

I've played with the recent Web based source code editors from Google (Code Mirror), Mozilla (SkyWriter), and Eclipse (Orion). They're OK, but a source code editor is just not enough for me...

I also need a browser-based Linux shell window. There's some good ones out there. I've tried Ajaxterm (a little slow and not all keys worked), Anyterm (keyboard issues as well), and finally settled for Shellinabox.

Shellinabox gets pretty close to a real xterm. I've been using it for two days for various tasks including coding (with vi). I like it a lot! and am starting to think that vi in a shell in shellinabox compares favorably to these new fancy Web based editors...

I've patched it to allow a shell to set the browser title using a VT escape sequence in an environment variable (PS1='\\033]0; [\W]\007\\w \$ ' for example), like people usually do with xterms.

The patch is there on my Github account. I find it useful to help distinguish Shellinabox browser windows and tabs and see what directory they're in.

If I find time, I'll post the steps I'm using to configure Shellinabox, build it and connect to it through a secure HTTPD reverse proxy... later this week.

I'd like to use its API to convert SCA composites (XML docs describing SCA component assemblies) into C++0x templates and compile them into machine code on the fly, instead of having to interpret all the XML gorp at run-time.

The Google Project Hosting team just announced the ability to edit programs directly in the Web browser using an online editor powered by CodeMirror. With Mozilla's SkyWriter, and Eclipse's Orion, that's one more way to edit code 'in the cloud' from a connected device.

That works well for me, as most of the programs I write these days run on servers 'in the cloud' anyway (say, big servers on a network instead of my laptop).

I've also been looking at the iPad as a programmer terminal and wondering about the upcoming iPad-2.

So, here's my future programming environment:

White/black screen / 24 x 80 characters
How about an iPad-2, Google CR-48 or similar netbook and a single Web browser window with big fonts?

I haven't been lucky enough to receive a CR-48, so I'm thinking about buying an iPad-2 out and hacking a programmer-friendly keyboard app for it.

No disk, data on a server
Anybody say Cloud?

VI-like
CodeMirror, SkyWriter, a hosted Orion server, or I'll just hack up a vi-like editor on top of one of these projects.

Assembly language + C Well, I have to use Java at work, but on my spare time I like to code some C, C++, Scheme, Lisp, Python or F#. Feels better. I haven't written assembly language for a while, but now that I think about it...

A few good programming books
Receiving the new TAOCP-4a book tomorrow, and will probably spend a few nights reading it.

Web search
Allow for a second browser window.

Some deep thinking
Still thinking...

I already love it!

Hmmm, the new iPad-2's rumored fancy screen should be enough for some 24 x 80 coding, right?