I am the author of 16 books on artificial intelligence and the semantic web. I use this blog to share ideas and code snippets using my favorite languages: Clojure, Java, Haskell, Common Lisp, and Ruby.

Saturday, June 16, 2012

Deciding between two premium hosting options for a new side project

Over the course of a year I spend about 2/3 of my working time consulting for customers and the other 1/3 doing my own projects. I view doing personal projects as a form of continuing education. My last major personal project was my natural language processing (NLP) web service KBSportal.com and at the same time I spent a day writing SleepyBird.us for my wife. The project before those projects was an experiment in a live note taking and scheduling system for consultants ConsultingWith.me which I still use but I have turned off the ability for people to create accounts. I never implemented the sister project ConsultingWith.us. Six years ago my side project was CookingSpace.com which I do still use. For me side projects are a great way to try out new ideas and it is well worth it to me to pass on less interesting consulting gigs to free up enough time for my own stuff.

I am in general interested in the combination of knowledge management and NLP and my new side project combines these interests with a tool that I want for myself: a system to register web resources, notes, and tasks and getting periodic summaries. One of the things I want to experiment with is three front ends (web app, mobile web app, and email) accepting input and displaying one a day and once a week status summaries to help put my time in context - to stimulate reflection on what I have been doing. The other thing that I want to experiment with is a web worker and backend worker architecture.

In the past, I combined web front end code and a background work thread into a single JVM process: basically I wrote Java web apps that had a specific worker servlet whose init method started a work thread. That always worked really well but for my new project I want to have a better architectural fit with platform as a service (PaaS).

Which brings me to the main topic of this article: I view both Google's AppEngine and Heroku to be Premium Services. Premium because no way you get a fraction of the raw resources as using a physical server from a provier like hetzner.de or even from AWS. What you get is the tradeoff on saving admin resources and easier scaling.

AppEngine got a lot of bad press for their price increases but the result is a service that they are unlikely to stop providing. One of the implementation options I am considering is a front end web worker written in Python and a backend worker written in Clojure. For this experiment I doubt that I will have to scale past that, but at least the path to scaling out would be straightforward. The other implementation option I am thinking about is using Heroku with a front end rails app and a background worker using either Ruby or Clojure.