Tag Archives: javascript

My latest released project is focusing on Visualizing Slovenian IT tax spending (139 million euros), the idea here is to take otherwise meaningless numbers and display them visually in a way that tells a story of who is spending how much and on what. The data set comes directly from the government in semi-clean XLS file. Visualization technique I’ve decided on is treemap visualization to represent the data with different box sizes relative to each other.

While visualization itself is nice, there are a two points that you have to be careful about when releasing such visualizations to the public:

Transparency of data and data transformations

In my case, the data set came directly from the government. In order to make sure that everyone can check my calculations I’ve included links to their file as well as provided a local copy in case their version changes or disappears.

You’re loosing and reinterpreting data with every visualization. That is why it’s important to also include transformation scripts so that others can check your work and possibly build on top of it or at least make sure that you didn’t do anything tricky with the data.

I’ve opted for a github repository where I’ve pushed all the associated files: http://github.com/gandalfar/itproracun. It’s a bit chaotic but it should be pretty self-explanatory to any python and JavaScript developer.

Telling the story
Every data visualization is trying to tell a story. It might not be obvious to the visualization author but it helps to identify this early in the process.

I started with just a simple breakdown based on the institutions:

It’s very noisy and it’s hard to compare different institutions to each other. Initial comments to this were that it’s not shiny enough. Cleaning the interface up I came to the following revision:

It’s much cleaner and what basically showed that I need to find an angle to this data. I decided to focus on the ratio between software and services vs. hardware and network equipment. Final version now tells a story of how police is spending a lot of their IT money on network and hardware equipment, while Tax Office is spending much more money on software and services.

Agenda of this last version of visualization should be clear to anyone who takes a few moments to study it.

Other lessons learned

Visualization toolkit should be powerful on one hand, but offer first results without too much work. JavaScript InfoVis Toolkit does this job very well. There are some interesting tidbits that are not entirely clear from the documentation, but become obvious once you start thinking how the rendering works.

The biggest time sink is parsing and cleaning up the data. Don’t expect that the .xls file will make any sense from the programmatic point of view, even though it mostly looks fine when viewed manually. Small parsing errors, moved cells and strange line breaks made parsing this data the biggest challenge.

But wherever you go, CSS selectors that you use, will most probably match styles of the platform. So if you use a widget on different platforms.

You can conquer this within CSS by writing selectors that are more binding than the default way. The one that we did (and is not the best way), we use a platfora of id’s and classes and then we use these names and classes that get injected into web-page and are more binding than the platform ones. This one works great if you use two files: general one and platform specific one.

Second way is to use nesting and “id envelops” – encapsulation and while it’s ugly, it’s the best way:

#zem1 #zem2 #zem3 h3 {

}

the second thing that you should try to use is microformats. After you put your widget into the page, you usually want that widget to do something, and for that we normally use Javascript.

Javascript problems

The variety of problems is large, but a short selection:

Extending Object and Array – it sounds a good idea in a start, but it’s a major problem down the road. It’s usually a bad idea that many JavaScript libraries (like JQuery), use Objects as configuration buckets and if you extend it, it means that all objects will have this and bad things will happen.

The second thing is, that if you have a web-site and you want to have it “extendable”, it’s a good idea not to step all over their feet and potential code.

Private name-spaces – either through anonymous functions or through object hierarchy. The problem is that sometimes you want to change behaviour of methods that are hidden behind namespaces (e.g., changing how dragging boxes works in 2.3 WordPress).

One thread implication: what comes later can rely on what came before. It’s not always true, because JavaScript API’s like DOM functions doesn’t always run in the same thread.

While you can do is to test your JavaScript on slow browser and computers, and on different connections. But this is often not enough, so you should actively protect from this and check to make sure the code and functions are there before you call them.

Their servers are runinng 99.5% open source software. They are mostly Linux users in-house, so they use Firefox as the only real option.

Firefox usage numbers for Interseek:

June 2005Â – 5.1%

June 2007 – 15.2%

June 2008 – 26.9%

Third one is always Opera (not Safari). In June 2008, Opera has 1.2% usage base.

We have Firefox users

We have a lot of Firefox users, and we must offer them a full service experience. We have to give them goodies like toolbars and Firefox search plugins. Publishing these features is good for us but also helps to expand the general Firefox user base.

Our story

in the first half of 2007 we launched Firefox toolbar and search plugin for Najdi.si and Pogodak (.hr, .ba., rs.) search engines.

What is so cool about Firefox extensions?

It’s mostly cool because XUL is cool. If you know how to make a dynmaic client side web application then you will probably know how to do XUL application. If you know how to program using XML, JavaScript and DOM you know XUL. XUL is supported by the Gecko engine.

Only XUL?

XPCOM components written in C++. Integration of other libraries into Mozilla.

Why would be ever need this? Things we can’t do with Javascript, we can do with an XPCOM component (e.g., usage of tcp socekts, etc.) and then use new functions from Javascript.

XUL extension

If you are planning an extension like toolbar or an ultra cool addon for uploading pictures to your new social network web site..

Building UI applications based on XUL. Two popular applications/examples are Songbird and Komodo.

Is your plugin working in Firefox 3?

Secure updates of your application with files hosted on secure http servers or signed xpi files. If you don’t have https server, it’s also possible to sign them with help of McCoy to create keys and fingerprints.

with single quotes, it can be quite annoying to always have to type it twice, since for double quotes (“), TextMate it always does it automatically. This feature is called Auto-Paired Characters.

Here is a procedure how to extend your Auto-Paired characters to also include ‘, in the default list (or any other character for that matter).

Go to Bundle Editor, and locate Text bundle (you might have to turn it on using Filter List in bottom left corner). Now locate Text -> Miscellaneous with (p) next to it. Inside it you should see a snippet of text that says: