Categories

Blogroll

I have been playing with Massachusetts Bay Transportation Authority (MBTA) Realtime API in recent weeks and ended up learning a lot about this transport agency which operates the largest fleet of public transport in New England area.
MBTA has a web based realtime API which can be used by developers who want to use the data to create new applications or just explore with the data.

Lots of the data provided comes with very little documentation in itself. Even when the documentation is provided it is pretty much lacking any context unless you read the blue book provided by MBTA. Here is a simple run-down of everything MBTA related, especially related to it’s subway system.

MBTA runs four rapid transit train lines, bus services, boats and commuter rails. All four of them are Electrical Multiple Units type of trains, with varying number of coaches per train. Out of four transit train lines, three of them are based on heavy rail system containing around 6 coaches per train with multiple doors on each coach. The Red Line train has 6 coaches, each coach contains 4 doors on each side of the coach. The Orange Line has 6 coaches and has 3 doors on each side of the coach. The Blue line has 6 coaches and has 2 doors on each side of the coach. The last line is Green Line which is actually a Street Car containing just two coaches and just one door on each side and might vary as there are multiple generation street cars serving the green line.

Blue line runs between Bowdoin in central Boston near Beacon Hill and ends at Wonderland in Revere. It is just one single line and does not split anywhere.

Red Line runs between Alewife at Somerville and runs south splitting near JFK/UMass where one section goes to Braintree and other goes to Ashmont.

The Orange Line runs between Oak Groke in North to Forest Hills in SouthWest.

The Green Line starts at Lechemere in Cambridge which is the Northern section and goes South till Boyslton Street, turns West and eventually splits into four sections are multiple stations. The four sections end at Boston College (B Line), Cleveland Circle (C Line), Riverside (D Line) and Heath Street (E Line). Even though there are four sections, not all trains run from each end to Lechemere. Some trains end way before the last stops on each section, not all trains from B, C, D or E Line go all the way upto Lechemere.

If you were confused by the above description, let me introduce you to the concept of “route”. A route is any two places where a service exists. Even in cases like Orange Line where there is no split of line, there can be theoretically more than one route. If there is a weekend service which travels from say Back Bay to Wellington station instead of going all the way till the ends of the line, that would count as another route on the same line.

We expect D line to go all the way from Riverside to Lechemere, but most of the D line trains end at Park Street instead of going ahead to Government Center, Haymarket, North Station, Science Park and eventually Lechemere. Infact many times MBTA serves a separate train between Haymarket and Lechemere. It’s like instead of having one route from stop 1 to 10, we have two routes on the same line from stop 1 to 6 and then from stop 8 to 10.

Next I would like to emphasise the difference between station and a stop. Take for example State Street station. Two lines – Orange and Blue Line have a stop in this station. Blue Line and Orange Line both end up having two stops each in station – one in each direction. Now let’s imagine theoretically, if Eastbound Blue line stops twice in the same station in two different parts of the station (if the station was huge), then Blue Line would have two stops in Eastbound direction with one stop in Westbound direction. That would make Blue Line have three stops in a station.
Not all the stops need to have a station associated with them. A bus stop at the corner of the street is not within a station. We can have more than one route crossing a stop is the most fundamental element. Stop can be associated with route and stop can be associated with a station. Thinking practically, a route cannot exist without stops and a station cannot exist without stops.

When I am working on integrating zeitgeist with any other tool as a plugin, addon or writing a completely new application, a lot of times I felt the need to run a live monitor of all the events which are being inserted. Since zeitgeist uses dbus, so the obvious answer would be to use dbus-monitor, but this tool is a bit difficult to use in my opinion. When you are working on something which needs zeitgeist, last thing you need to do is to spend next few hours dealing with dbus-monitor.

I ended up writing a small GUI application which connects directly to zeitgeist daemon and receives a notification everytime an event is inserted. I released it 2 months before, but never got enough time to further work on it even announce it’s availability. The application is very easy and just lists down all the events. Select any event and it will give you all the information about it. Just to clarify, this is a developer tool.

When I first started programming, working on projects both in Open Source and Corporate world – I wish someone made me aware of the Do and Don’t. For a long time, I wanted to sum them up and keep it as a checklist.

Commit Messages

Make sure you put a commit message which makes sense. It is true that a good commit message might be difficult to come up with, but please avoid messages like “Fixed bug” or “Rendering”. Which bug did you fix? What is the bug number? What is the bug about? What rendering? What did you do with rendering? Did you fix it? Did you add rendering capability? Did you remove rendering capability?

It is a good practice to add a line of commit message title, followed by a single blank line and then an optional detailed description of the change which might not be obvious. If you made a commit which alters the working or behavior of the component, you might want to explain why you took that decision in the description section of the commit message

Check In Early, Check In Often

Jeff Atwood has an excellent post on this recommendation. My personal motto is to check-in when I am done for the day and pull in the changes when I start working for the day. I have a friend whose motto was “Every morning, you wake up and do ‘svn up’, even if you haven’t brushed your teeth”.

Keep the change-set manageable

Avoid any such scenario where your commit diff is 2000 lines unless you are nuking a component or any special scenarios. If the component you are working on enters the source tree in a single commit with 4000 lines of diff after 5 days, then STOP. THINK. Is this really the correct way? Can you do it in a better way?

If your single diff contains truck load of different features with 25 files affected, then it is a exercise in frustration for anyone who is attempting to review it. Try breaking it down such that you are able to follow “Check In Early, Check In Often”.

Learning to resolve merge conflicts

To be honest, after working on git for long time, I am still have a sigh face when I encounter a merge conflict. It isn’t always easy to fix it, but don’t forget that merge conflicts do happen and you will need to deal with it even if you collaborate on a piece of software in a very smart manner.

Get your code reviewed

No one’s code is perfect. At times, we are insecure of our coding abilities and become overly protective of our code quality. It took me time to learn that code reviews are not just about improving quality of the code, but it is a great opportunity to learn. Even if you don’t have a better programmer around, get it reviewed by your peers whom you consider at par with you.

Try reviewing code, that helps too – you learn better ways of implementing features and ideas.

If someone criticizes your code, resist from taking it personally. Usually the person criticizing your code has got nothing against you.

Start Questioning

You don’t need to catch your team lead by the neck and demand the answer for choosing this method. Question yourself! If the lead of a project has chosen to use MVC pattern or decided to use Smarty as the templating engine, then don’t just take it for granted. Think over it. Try again! Why does this problem needs a MVC solution? Why does this needs to be implemented in Rails?

Recently I resigned from my job and went back to graduate school for my masters in US. I had to leave my desktop back in India and take my 6 years old along. Even though it works, it is a living horror. I am somehow able to manage my coursework using this laptop and cannot use it anymore for any open source development.

This laptop overheats and burns my legs, has a broken wifi card which forces me to use an external WiFi adapter. The touchpad buttons are broken and using the touchpad requires secial talent.

One of the USB port, optical drive doesn’t work at all. Battery is completely dead. If I pull out the power cord, it switches off the same way as your TV turns off – not even a single second of backup. I recently got a new charger else this laptop would turn into a paperweight. The only things which works are keypad, display and the bare necessary things like RAM, processor to use the laptop.

Donations

I know it sounds shameless, but being on a student education loan and in future I might have to use all the sanctioned amount, I am not in a position to buy a new laptop. I have $400 with me and I am aiming for a customized Thinkpad T430 which should last for another 4 years. It might cost upto $1000 including taxes and shipping. If you are happy with my Zeitgeist and Activity Log Manager work, I would like to humbly request you to donate any amount if you are willing. If I don’t make to $1000, I would have to try to figure out other ways to fill the gap.

I have not been able to work on any Ubuntu, Zeitgeist or any other open source project for last 4 months. Partial list of reasons are immigration stuff, figuring out academic requirements and preparations for this change.

If you are in India, you can deposit it in my State Bank of India account. Account number 00000020085201096 and IFSC code is SBIN0004456

What to expect

If I reach the target and get that laptop, these are the things I have in mind. These are the TODO and possible work I would like to do

We all love privacy and we take it seriously. Most of the people whom I have met are pretty much vigilant about their data and which makes complete sense to have a tool which can you can use to fine tune your privacy settings. Till now most of our experience with Privacy Settings are related to Facebook which makes me feel like

Layman control center?

If you are a Zeitgeist user, it is pretty expected to expect a Privacy application. Yes, we have it. Today we are releasing it. The first release was a hastily assembled up together application using pygtk (gtk2) which served it’s purpose well for the time being.

Older version of Activity Log Manager

Soon it was decided in Ubuntu Developer Summit in Orlando that Activity Log Manager will be included in Ubuntu Precise to provide greater privacy controls to user. We improved the looks based on the design provided by the Canonical Design Team.

The interface was kept as simple as possible. This first release lacks a few things as of now and some more UI polish is pending including localization support. Both of them will land sooner.

It has been integrated in Gnome Control Panel (which works only on Ubuntu). In distributions where it cannot integrate in Control Panel, it runs as a standalone application.

Here is how it looks

Entry in GNOME Control CenterBlacklisting ApplicationsFiles and Folder blacklistingActivity Log Manager History tab with the Icon

The Zeitgeist Team worked on it and ported the application to gtk3 and vala in a few weeks which was a pleasure (and I finally learnt vala. Finally!)

Jono founded Ubuntu Power Users group a few months back. Since then lot many things were discussed but due to being a bit late in the cycle.

Power Users Group got a fresh new start and it was advised to hand over the bigger responsibilities to James Gifford. If any of you don’t know him, he a a very young yet highly energetic community member and an active member on AskUbuntu.

James, every initiative has it’s high and low in activity. When Power Users was started there was many good discussions but now it looks like the team really needs a strong leadership. Do not be demoralized if you find that you are the only person who is working under power-users banner. There might be other people interested in contributing, but they might not feel like contributing when they find themselves alone. It is always more encouraging and consoling to find that the project you are contributing to has an active leader and your work is actually going to make a significant impact. When there is a leader who is working tirelessly, I feel more confident because even if I am not able to contribute, still there is someone who is making sure that my work will reach the wider audience.

Being a leader is tough. Being a maintainer of a software project is also very tough. It is sometimes easier to be a person who just commits code than being a maintainer. You have to make sure that all the nuts and bolts are assembled together in the correct order. Contributors might only make separate pieces, but you have the responsibly to assemble them together and make a working product. Sometimes you even have to make sure that your assembled piece works properly when it itself has to act like a nut to an even bigger piece of software.

I admit having the feeling of “Why? Why am I the only person who is working on it? Does it mean no one else is interested?”. The answer is that probably many other people might be interested, but they are just too busy in another piece of software or community which they maintain. They might be not getting enough time to contribute to the project in which I or you work.

That said, James please make sure that atleast Ubuntu Tweak is included in Ubuntu archives starting Precise. The Ubuntu developers had their own valid share of criticisms for this product, hence it is time to address the issues. I can help you in support and documentation part (on AskUbuntu).

As I announced earlier we had opened User Survey for Zeitgeist for general feedback and assessment of our own work. It was also to help us understand what are the user requirements.

After 5 days, we have received 261 responses. Instead of going in details and boring you, I would like to keep the post as short as possible and provide detailed analysis in the next post.

Our survey contained questions related to general Zeitgeist questions, Activity Journal, Activity Log Manager, Datasources and Community aspects. Users were also asked about their distro of choice and optional name, age and Country.

Distribution demographics

The biggest share of responders were using Ubuntu – 136. 30 of them were using derivatives. I am also happy to see Fedora and Gentoo in the list.

The other 10 distros included Mandriva, Chakra, Kubuntu (which could be part of Ubuntu and it’s derivatives)

How did you come to know about Zeitgeist

Online blog/news-outlet and Planet make up a huge percent of users. It is evident that people do follow these two very closely. Notice that total does not add up to 100% as just hearing the name might not be sufficient for someone to figure out what it is.

How much do you know about Zeitgeist

Pretty much obvious conclusion. The user know what it is but most of them are not geeky. Looks like those 6 “I hacked on the codebase” was selected by contributors.

Do you know what is Zeitgeist Datasources

One half of my brain says “Teach them what it is” and the other side says “No. Not everyone is very interested in being force taught”. Guess what? I take the central stand and will publish a short article on what all components of Zeitgeist do.

Which of these Datasources do you use?

This is a VERY VERY important chart. This tells us the adoption of datasources esp when datasources are not present in Ubuntu, Debian, Fedora and openSUSE (AFAIK for last two). I think they are present in Gentoo and Arch.

I am really humbled that so many people like Firefox datasource even though it had suffered so much with regular xulrunner API breakage. With our new datasource which uses js-ctypes, all the API headache is gone. Gedit. Banshee and Chrome too score pretty high

Which datasources would you like to see in the near future?

I never thought Thunderbird would come out as first. My expectations were with Gnome-0shell which is actually second.

Good news is that Thunderbird datasource is in progress. Mark Jtully is working on it. Gnome-shell would is also in progress.

Since other were 76, I would like to name a few, though I would prefer a separate post for this. The number 76 has led me to make up my mind for a special treatment.

Let me list a few:

Novacut

Jason. Is that you? If yes, then contact me on IRC 🙂

git-repos

Personally my favourite too

Pidgin

Work in progress

Empathy

Telepathy plugin already present

Clementine

Even I am excited about it

KDE

Work in progress. Stay tuned for the next blog post

Heard about Activity Log Manager

Activity Log Manager is Administrative tool for Zeitgeist. You can temporarily stop logging, blacklisting certain applications from logging information, blacklisting certain mimetypes, clearing all history between time range etc. It is a handy tool. A new version is going to be out soon.

It is really sad that half of the people never heard about it. Looks like we failed at spreading the word. By the way there has only been a single release.

How much are you involved with the community?

40% of the people are users. “Nor am I interested” is not a harsh phrase. It simply means “I use so many applications and it is not possible for me to be involved in each and every community”.

15% of the respondents are eager to join the community. Probably you all can help us in testing. This is where most of the manpower is needed.

In case you have problems, where do you ask your questions?

Two-third of the people do prefer forums. Mailing lists and IRC still trump over Q&A sites. I think many people have answered this not only for problems but also for general questions.

In case you are an Ubuntu user, you can ask your question on Ask Ubuntu and tag the question with zeitgeist tag.

Are your problems related to Zeitgeist solved?

Did I tell you that this chart has boosted my morale? 70% of the people didn’t have problems with Zeitgeist. I now really guess that these people were asking general questions or discussing about zeitgeist and replied to the above question.

Whatever be the case, we still have 10% unsatisfied users from the samplespace. 10% is a significant number. Hope we can bring it down by the next survey.

Few interesting comments we received

More focus on the related items of Zeitgeist

Looks like we really lacking here. Thanks for reminding

The ability to put zeitgeist in private-mode

It is already present in Activity Log Manager

I’m a software developer and do lots of testing every day. I need a way of blocking activity from my development/testing trees

There is a “Logging Active” toggle button in Activity Log Manager

It[Activity Journal] needs to be abstracted into Unity

Isn’t it already? Probably the implementation is different. There is no journal view in Unity

It doesnt show all my files

You need more datasources. In other words we need to provide more datasources which you need to install.

It’s [Activity Journal] good, but shouldn’t we be providing this information through the UIs made available by the host OS?

It is somewhat done for Unity. Next up in Shell and Plasma. In any case none of them show the Journal view as Activity Journal

I really like being able to see what I did, and when I did it. The thumbnails for code snippets, videos, and pictures are also rather nice. Congrats on making a easy-to-use computer journal 🙂

All I can do is smile

Please integrate it into Gnome Shell if it light enough.

This is also my personal wish. It might take some time

It would be nice if these datsource plugins are included by default without the user having to do it manually. This way it just works.

This has to be done at distro level. This is something even I would love to see