Caltrain updated schedules today, August 31, 2009. I released updated versions of my Caltrain schedule applications last night, so this is a little delayed notice.

My first application is CaltrainPy, which started out as a Python app for my phone, but it did not work so great on the phone after all, at least on Windows Mobile. It is fine for desktop use, though. I added schedule scraping functionality later to support the next two applications. Fully open source and all that.

CaltrainJS is a web version, optimized for Windows Mobile browser. It of course works with other browsers as well, although it hasn’t been optimized for them. CaltrainJS is free to use, but not really open source.

The last and most fully featured one is the Android port, Caltroid, which is a commercial version. I’ve added a little Google Comments gadget on the Caltroid homepage in the hopes that people would find this an easier way to communicate with me on what new features they’d want, what is broken and so on. The Android Market comment system doesn’t let me respond to comments in any sane manner, but the Comments gadget does. The gadget isn’t perfect, but hopefully better than nothing. This is also first time I am testing out Google Gadgets, so time will tell if I’ll keep using them.

The only application to gain small new features is the Python version, which basically just got some properties to identify which schedule it embeds and what schedule date it was confirmed to be able to process. I have some additional ideas for improvements, but due to currently being somewhat sleep deprived those will have to wait a bit. But if you do have ideas, please let me know.

My project is really simple, but once I started looking at running Javascript compression and checking tools I realized I needed to add some automation to the process. I considered three options: Makefiles, SCons and Fabric. In the end I chose Fabric because I didn’t need to do much in the build steps, but I needed a quick and painless way to upload my changes to a staging site for tests. I think Fabric handles the last step best.

My build step is mostly copying files around. I have two Javascript files, one of which is produced by CaltrainPy. The format is pretty compressed already, so I just run sed on it to shorten AM to A, PM to P and take out – from certain fields. Previously I was doing this in the online application, but it makes sense to do this in the build step. I run the ShrinkSafe Javascript compression tool from the dojo project for the second file and combine the two scripts into a single file. I find it somewhat odd that a Javascript compression tool uses Java, but whatever works… The last mildly interesting part is the generation of a sitemap XML file, for which I just wrote a simple Python function. In the Fabric build step this becomes:

The one gotcha I run into with staging is that current version of Fabric does not seem to support wildcards. I first tried to put('*.html', '/path/on/server/') but that wouldn’t work. It seems like such a basic feature that I wouldn’t be surprised if it will be included soon.

For CSS compression I just used cleancss manually. I also looked at cssoptimizer but it was a bit too extreme to my taste. cleancss is based on csstidy so I could possibly integrate that into my build.

For validation I have used W3C validators, but it would be nice to be able to integrate validators in the build step as well. For Javascript it seems JSLint would be a good option. I haven’t spent any time thinking about HTML and CSS validation during the build step, but obviously this would be a nice thing to have as well.

The new feature is the system map. I also realized that I can expand on this work to add support for fares and maybe other similar enhancements in the future. One of the major obstacles holding me back was that I wasn’t sure how to fit this in the very small amount of real estate I had available. I solved this by making the Trains header into a button like the direction and day of travel at the top of the application.

All I can say is that it is really frustrating trying to write Javascript for Windows Mobile. I was able to get the map to display without recreating the whole schedule, but at the price of leaving some clutter after the map (although some could consider the end result a feature). I also found a bug where the scrollbars appear when viewing in vertical mode although they shouldn’t. They go away when you switch to landscape and then back. This bug affects only Windows Mobile devices as far as I know. I also added a documentation page to help users figure out how to use the thing and what all the different symbols mean in the schedules.

The stylesheet and Javascript are now in their own files.

The final piece of work I did was ensure the HTML and CSS passed validation.

Have you ever been looking for a colleague, printer, or just about any resource in an office and being frustrated there was no map showing you where to go? I have good news for you. I just released Solu, which is a “Self-service Office resource Locator and Updater”. Or “cubicle finder”. Or whatever you want to call it.

This project started as a perfect example of “scratch your own itch”. I was doing some research on how to do REST with Python, and remembered jj recommended Werkzeug at some point, so I took a look. Around the same time I was chatting with our new admin, and noticed the office blueprints on the IT cubicle wall. This lead the two us discussing how nice it would be if there was a map to help us new people around. I ended up saying I could program such a system in 20 hours.

So I ended doing a bit more than research, and studied Werkzeug and jQuery. And although it wasn’t strictly necessary I also spent quite a bit of time researching how to make an easy to deploy package of the application, and how to make it easy to try it out. In the end I spent almost exactly 20 hours on the first version, but I realized I could have done it in about half the time had I used Pylons, mostly because I already knew Pylons and also because the things were I struggled with Werkzeug were already provided by Pylons, or easily integrated into Pylons. After the initial version I’ve spent another 20 hours to fix bugs, write tests, and in general getting it to a stage I felt good to release. Even still it is missing some pieces I know how to do in Pylons, but still don’t know for sure how to deal with them in Werkzeug.

Since I developed this application partially on SpikeSource time, it was nice of them to let me Open Source it to everyone’s benefit.

Some of the things I really enjoyed about Werkzeug include:

Interactive Python debugger in the browser

Small enough to make the ramp up period quick

Good documentation

Enough features to provide almost everything I needed

Easy to swap template system from Jinja to Mako (since I didn’t want to spend the time to learn Jinja too)

No internationalization/localization (i18n/l10n) support out of the box

No samples or recommendations on how to pass application and options to views and templates

No samples or recommendations on how to implement sessions

In the end I used ConfigObj to load the options from an ini file. This was a little trickier than I thought at first because I wanted the manage script to work so that I could run with Werkzeug’s server while testing, but I also wanted to retain WSGI deployment possibility. Packaging took some extra reading about setuptools. CSRF is still a little open, although I got pointers to zine.utils.forms and also found werkzeug.contrib.sessions which should make it possible in the way I was thinking about it. I don’t see CSRF as a huge issue at this point, though, given the data this application handles. I also got some pointers on how to implement get_app() function to get the application object from anywhere, so getting the options stored in the app became easy. I still have some open questions about localization, but those might go away once I actually try to do that.

There are some obvious improvements to Solu, like fixing the CSRF issues, i18n/l10n, multiple and resizeable maps, dealing with errors in a more user-friendly way, and overall making it pretty. I could see someone wanting to pull the information from company LDAP database or some such, and hooking this up with general employee database.

Since I spent so much time making it easy to see what the application is about by providing a website with screenshots, a demo site, easy installation and instructions on how to run your own application, I am interested in hearing how I did.

I have been reminiscing about the programming language I have used as a student, as a software engineer, and hobbyist. The list is actually surprisingly long, come to think of it. I think it serves as a useful reminder for students interested in working as a programmer that you are very unlikely to be able to go through your career with skills in just a handful of languages, less alone one. I am also purposefully skipping languages that can not be considered full-fledged programming languages. I am also lumping all variations of a language as one language, more or less. Here’s the list of languages I can remember writing at least one line of code in, roughly in the order that I started using them:

I started with Basic on a friend’s Commodore Vic 20, and then on my own Commodore 64. I used some Basic version on PCs, Visual Basic in a student project and later in small Microsoft Access projects at work.

My university was big on mathematical information processing, and given that I also started as a physics major, the university saw fit to teach me Fortran. I can’t remember ever using it for anything outside those classes, though. I came pretty close when I was working on NIMBUS, which was an online multiobjective optimization package researched and developed at University of Jyväskylä.

I learned Perl on my own when I created an online exam enrollment system for my university. I was too lazy to walk from the computer science lab on the first floor to the third floor where the exam enrollment drop box was so I decided to write the system for my faculty Perl was my main scripting language until 2003.

We had one class at the university that many people dreaded, and that was the class where you had to learn a little bit of Assembly language. I have never had the need to write Assembly since then, but it would have been useful being able to read it occasionally. This is one language I might actually need to invest some time in the future, I think.

I learned this on my own to add some dynamic content on my web pages. I was among the first 10 people in my university to have homepages, and I actually ended up giving some advice to the staff member(s) who were running the webservers. I have since used it on many websites.

Learned while I was an exchange student at the University of Kent at Canterbury. It was a fascinating language, but I could not wrap my head around how to actually use it (I did badly in my assignments).

I had a summer job while I was studing at the university to program questions with answer checking for an online intro math class for the Tampere Technical University. I am still blown away by how they had hooked up Mathematica to the web server backend. Haven’t used it since.

Not sure if I should call this a programming language, but Wikipedia lists some other XML languages so what the heck. Mostly dealt with XSLT when an engineer in my group was working on adding P3P support to Netscape and Mozilla browsers.

My first touch with Python was actually before 2000 when a fellow student raved about it, but as soon as I heard about forced indentation, I walked away (bad experience with Fortran…) Too bad. I wonder how things would have turned out if I had stayed and drank the coolaid then. Next touch was at Netscape, when a colleague raved about it. I actually ended up modifying a little Python program – flawfinder – while at Netscape. But it wasn’t until I started at OSAF back in 2003 when I really had to start using Python. I guess I fell in love with the language in the first two weeks

When I started experimenting with ads, there was one case where the ad code was provided in PHP, but I wanted to alternate between two different ad sources, so I had to learn enough PHP to do that. Quite likely I will need to learn more, as I am thinking about creating a theme for my blog since I can’t seem to find a theme that I like. Incidentally, if you know of nice 2 or 3 column liquid or fluid layout WordPress themes where the main content is on the left, let me know…

* Used only in school.

I listed 20 languages above. A little more than half were required as part of doing my day job. And this does not even list languages that I have had to read but not write, or configuration mini-languages, or XML languages (apart from XSLT), … I don’t expect to stop learning new programming languages any time soon.

Update: I forgot HyperTalk, which I used a little bit on my Macintosh SE/30. I learned it around the same time I learned Fortran and C.

I finally decided to sit down and fix some annoyances in my Caltrain schedule applications written in Python and Javascript. I added the ability to filter trains that don’t stop at either your departure or destination stations. For the Python version I also added the ability to install the package simply with easy_install caltrain. Even nicer, when you have setuptools installed (as you do when using easy_install), the installer will place a script named caltrain in your $PATH, which makes it a snap to launch the application. The magic part in setup.py for the script is simply:

where the line "caltrain = caltrain:gui" means that the script will be named caltrain, and it will call the calltrain.gui function (caltrain is the module).

I also finally made a proper page describing the project. Much nicer than needing to find the last post on this blog about the topic.

I noticed Caltrain has a page listing mobile applications. I wonder what it would take to get them to include my little apps there. I quite prefer the UI in my applications to all the others (except for the PalmOS version). This is what the latest Javascript version looks like:

A colleague pointed out to me that what I am trying to do with battle canvas might still be too complicated and I could simply make do with a scaled image background and drag and drop divs over it. It doesn’t quite tickle the excitement about trying something completely new to me, but on the other hand this is the tried and true method of working on the web today.

Using scaled images and dragging divs has the benefit of getting advantage of libraries (for example, jQuery’s draggable works just fine), but it seems like it will also result in fighting tedious CSS differences between browsers, and things outside the canvas being able to affect the battle canvas by accident.

Nevertheless, I created the (simplest yet) battle canvas in theseries using a PNG background and jQuery UI to implement draggable area effect and hero square: Battle Canvas PNG.

I might have fallen victim to the Golden Hammer. After pondering about the drag and drop of canvas elements, I realized there is an older technology that might be suitable for my purposes as well: SVG. It is just about as easy to make SVG battle canvas as it is to make one based on the canvas tag. I think SVG is a bit more promising because it provides structure and as such you would think providing drag and drop manipulation would be easier, perhaps already done.

I started to experiment by building the SVG canvas using jQuery and the jQuery SVG plugin. Static grid, hero box and transparent area effect were no problems.

Then I found out jQuery supports drag and drop in the jQuery UI. Unfortunately this didn’t work; trying to call draggable() on an SVG element gets you an exception because the code assumes HTML DOM (I think, I didn’t look too closely).

Then I decided I’d try it the same way I did with canvas. Apart from an issue trying to figure out the container’s offsets (hardcoded for now, might not be correct with other browsers but should be able to work this out), the drag and drop works. What is better about the SVG compared to the canvas tag is that I can set the mousedown handler on the area effect rectangle object itself without needing hit testing to see if I started a drag. Possibly the drag could also be done using SVG animation or by erasing and redrawing the dragged object instead of erasing and redrawing everything.