Present Perfect

2011-3-2110:17 am

When I look at the list folder in my Evolution these days it’s just a depressing sight. So manny mailinglists I’ve accumulated membership to that go unread now.

I think the model of mailing lists is just fundamentally broken. I just subscribed to the cdparanoia mailing list because I had a bug filed against morituri related to the sample-offset option. And you have to subscribe just to be able to post.

So the process to get some question answered is:

ask to be subscribed

wait for the request mail to arrive, which arrives somewhere in my admin/list folder which I keep forgetting about because of server-side .procmailrc rules

wait some more, because the greylisting on our mailserver probably delayed the arrival

reply to it

wait some more, you’re now subscribed

send your question

wait some more

update my Evolution folder subscription selection, because another .procmailrc rule (that I probably stole from jdub) automatically filters the mail into an unfiltered subdirectory which is a catch-all for all mailing lists; but it only shows up in the list when the mail actually arrived of course

check the mail in the list, which is usually only mine

remember to check that folder over the next few days for possible answers

end up accumulating years of cdparanoia list mails that I won’t ever read because the friction of this process is just too much and it’s just easier to waste bytes across the net

Of course, I can just change my account settings on a particular list to not receive mails, until I have my next question, but that’s still painful.

My garden is in dire need of weeding here, but I don’t even know what direction to take it in. Anyone who feels they have a good handle on the mailing list problem want to share their approach with me ?

In particular, I’ve been googling for a solution to a very particular problem: the not noticing mails on mailing lists that are directly for me – either as a reply to one of my questions, or because I’m being CC’d somehow. My filters file all mails to mailing list in their respective folders, and I tend to miss the mails addressed to me a lot. Surely there is an easy procmail recipe that, when I’m in CC in some other way than just through the mailing list, also leaves a copy of the mail in my INBOX ?
Surely there must

2011-3-146:42 pm

Having various machines, some with homedirs passed on across distro versions, I somehow ended up with both Download and Downloads directories. Not to mention that my Firefox and Chromium instances were downloading everywhere, and the silly mental friction of this one letter difference is pissing me off. I was renaming one to the other on one machine, and probably doing the opposite on another.

So, no more. Instead of figuring out what the spec says, I just created a fresh user on my Fedora 14 laptop and did:
$ ls
Desktop Documents Downloads Music Pictures Public Templates Videos

So, that’s what it’s going to be. Never mind that I hate with a passion something ugly like Music vs Videos (where do I put audio podcasts then ?), I’ll just follow like rank and file.

2008-11-81:13 pm

The short version is ‘I can finally live with the favicons in my delicious extension thanks to a script I wrote and after upgrading to beta 2.1.005.

The long version is, well, longer.

This morning’s yak shave story begins a bit over 5 years ago. That was the time when I was inbetween jobs, and already had my laptop for the new job, which was going to evolve into being ’employee number 1 at Fluendo’. I had decided I wanted to try the ‘living life from only one computer’ approach to working and hacking. Some people do this instinctively, some people don’t choose and end up with crap on all their machines, but me, I like inventing processes where there is no need to and sticking to it.

So, for a long time, I mostly worked only from my laptop, and this served me fine in a lot of cases. My main motivations were ‘I want to be able to work anywhere I am’ and ‘If I’m used to a laptop I will always be able to be productive’. The first proved very useful whenever I was travelling, and the second is what made me not want or need a second screen at work when all developers around me were making it a political point.

The biggest problem, however, is that often the work you do really does need the higher CPU, the more RAM, or the bigger disk that your average desktop easily has over your laptop as soon as your laptop becomes a year old. Also, your laptop wears out a lot faster if you have it running all the time. So, at some point, after knowing for sure that I am able to work constructively on my laptop, I decided that I should also make it possible to work from a home desktop and a work desktop.

Make no mistake, to me this is an incredible paradigm shift. While on its own there is no real problem to habitually work on three computers, to someone like me it is incredibly annoying to have my flow broken by various small details that made up my previous ‘work environment’ on my laptop that I have not yet successfully migrated to the other machines. If you have only one machine, adapting it to your workflow continuously is easy. If you have three, you constantly bump into the things you forgot to synchronize across all three at the moment where you least want it.

From day 1 the plan was thus to make sure that everything that is important to me would be synchronized across all three machines as often as I needed it. Additionally, I would make sure not to start using any new tools that weren’t able to work this way, and move my current set of applications in a direction where they would be easy to sync. It would also be important to make sure I can work offline as much as possible – I spend time in travel and on planes that I want to put to good use.

So, over the years, I’ve:
– created an svn repository for all my private notes, documents, tests, and code
– created a public svn repository for the same stuff that can be public
– picked a Getting Things Done application called yagtdthat uses a text backend so that I can put the data in subversion to keep it synced across all machines but still work offline
– tried to move my contacts to an LDAP server so that Evolution can use it (which I still consider a ‘fail’ because Evolution’s LDAP backend is still crashy, hangy, and practically unusable for everyday use)
– moved my calendar to various Google Calendars (up to having a public calendar called ‘Where is Thomas’ so friends and collagues know what country to expect me in)
– actually paid for an application on my phone – the wonderful GooSync Zaheer recommended me which allows me to synchronize all my Google calendars to my phone, in two directions
– moved all of my bookmarks to Delicious for obvious reasons
– used the Delicious firefox extension to replace my bookmarks folder toolbar – Instead, I use the ‘Favorite tags’ view and use delicious tags like ‘@daily’ (borrowing the at symbol from GTD’s contexts) to have quick access to all the sites I use on a daily basis. This works great because as soon as you drag a bookmark to one of those favorite tags on the bookmark bar, it is instantly available on all three computers where I have this set up.
– put all of my passwords into a revelation database that I put in my subversion repo (not 100% ideal, since managing conflicts is impossible, but hey…)
– wrote two scripts, called ‘start’ and ‘stop’, which I execute when I start or stop working on a machine, and which sync stuff from subversion and commit stuff to subversion that is important to my workflow (my GTD list, the password database, …)

None of this is perfect or ideal, but most of it is about as good as it gets for my usual workflow. Lately, I’ve started focusing on improving these individual elements.

For example, contacts. I really wanted all my contacts to be available in Evolution on all my computers, and on my phone, syncing every way. And bonus points for anything that would finally let me know when my friends have their birthday, a pet peeve for like the last 10 years.

I read some Evolution code and figured out a lot of reasons why the LDAP code is bad, but most of my inquiries (over IRC, on the mailing list, in bugzilla, or poking some friendly hackers (who will remain anonymous)) gave me no idea about why the code was the way it was, who is responsible for maintaining it, and if my ideas for improving were actually any good. I did get a lot of feedback from people agreeing that it sucked or wanting me to work on their pet feature for LDAP support though.

Similarly, calendars. Around the same point, I tried the Google Calendar backend for Evolution as well, because I was getting fed up with not being able to use Evolution for this, and what seemed to be touted as a major feature for a new GNOME release didn’t actually work very well. I couldn’t figure out at all if it actually supported more calendars than just your primary Google calendar (which I hardly use – I have 10 others), and it didn’t work very well at all.

I found a java daemon that is supposed to synchronize your Google calendar with local .ics files which should then work for Evolution, but in practice works very badly for two-way sync and Evolution doesn’t seem to notice the file changing anyway. All fixable, but fixing hacky solutions isn’t very motivating. And as I mentioned before, while I usually set myself up with a process then stick to it with an almost religious fervor, as I had done for the last years with Evolution – for the first time I was seriously considering completely dropping Evolution.

I saw that Conduit seemed to have some Google support, but after talking to John it seems that the support for Google Calendar got broken and removed in a recent refactoring (I abstained from noting that, by definition, if you broke or removed code then it isn’t a refactoring), so it wouldn’t work at the moment.

Moving back to contacts, it came to the point where I deactivated my LDAP backends, and tried to go back to using the local one and figuring out other ways to sync. I noticed that GooSync offered the option to sync to Google Contacts, which I had never used before, so I tried that. I found that Conduit did have support for Google Contacts, but I could never get Conduit to sync more than 20 contacts. If I were to hack on anything to fix it, it would probably be Conduit, because I have a feeling it wouldn’t be too hard to get it fixed. While my Evolution contacts were being updated on the fly through conduit, I could never get them all. And after a few 3-way syncs I ended up with a bunch of triplicate contacts in my Google Contacts.

But at the same time I noticed something utterly astounding that I would never expect from any Google product. And sure, it’s a detail. But somehow I have this opinion of Google that they are so good at getting some of the small details right in their applications. I was so surprised to find out that Google Contacts does not allow you to store birthdays. It’s just not there. Even my phone lets me do that and shows the birthdays on its calendar. I was sure it was me who was unable to figure out where they store it, until I found a whole bunch of posts from users sharing the same disbelief. Amazing.

Here’s a company with some of the smartest engineers in the world, who’ve made a successful web mail client, a successful calendaring application, and some technology for storing and using contacts across all of their applications. Yet no one in that company, with all their 20% times, thought it would be a good idea to have birthdays in contacts and have Google Calendar show you a calendar of birthdays automatically all the time ?

I left the whole ‘synchronizing contacts’ plan on the top shelf right now, while I try and figure out whether I’m going to:
– ignore birthdays and clean up my Google Contacts
– figure out if I can write enough code to make Conduit sync 3-way between my phone, all my Evolutions, and a collection of vcard files in my subversion repository.

So, after that disappointment, there was one more really annoying problem in my ‘synchronize everything’ approach – since upgrading to Firefox 3 favourite icons have been a huge mess. I had about a 30% chance of getting correct favicons, with the other 70% split between ‘not getting a favicon’ or ‘getting the wrong one, forever’. The delicious extension was even worse at it, usually not able to show more than 5 of them correctly in my toolbar. Seems like a very small detail, but when you have a lot of bookmarks the icons really help in finding the right one consistently.

After various betas and updates of the extension that never actually fixed the problem, a few posts to the yahoo group, little more than promises from the developers that ‘things are changing’, this week they launched a new beta that takes the first step in a new direction. It wasn’t clear to me that it would actually solve the problems, but the notes said that they moved from an RDF file to an sqlite database for storing the synchronized bookmarks. And A developer replied that this new beta should fix my particular problem.

So I updated, tried, and at first nothing changed. I stopped Firefox, opened up places.sqlite and dropped the moz_favicons table. Restarted Firefox, and it refetched a bunch of favicons (some of them wrong – my homepage got the GNOME foot for example), and the delicious extension did get a bunch more favicons, but not all of them by a long shot.

If the extension had been open source I would for sure already have taken the time to look into the code, and fix the bugs – that’s how much this irritates me.

But, with the information that now everything was stored in an sqlite backend, I decided to look for the database and see what was wrong. And after 2 hours of hacking (double the time it took to write this overly long blog post) I have a simple python script that adds or fixes the favicons in the ybookmarks.sqlite database.

It’s probably dangerous, eats your babies, breaks your database (so back it up first), and takes ages to run as it downloads each page in your delicious account, but for the first time in two years I can finally see that last.fm favicon in my @daily bookmarks folder!

It’s Saturday morning going onto 13:15, and finally I can start doing what I was actually planning to do today, and in no way related with the final straw that caused me to fix my favicons.

2008-10-2912:18 am

Some principles and observations come back in many aspects of life.

One drive I have in everything I do, whether professional or personal, is to remove friction. Removing friction helps to stay in a flow and focus on the important things. (One of the bad sides is that I’m prone to yak shaving because of this, although I think it pays off). I like removing friction today for my flow of tomorrow.

A recurring observation I have is how, in any company of a reasonable size, certain small things just don’t happen automatically anymore, because a sense of shared ownership gets diluted when there are more people involved. Typical example is ‘the shared fridge tends no lose the entropy battle’.

And another observation is how, in many cases, an overengineered system ends up being extremely unusable.

Today, those three met when I was in a meeting and I wanted to put on the lights, but I turned on the way-too-loud AC instead.

What was the problem ? Pretty simple. The remotes for lights and AC all have 6 buttons. Left ones are On, right ones are Off. But there are no labels. So these programmable remotes can control whatever you want. It’s just that usually all you want is to control lights and control AC.

So, someone had labeled the remote using a Dymo (I love Dymos), but apparently got them switched around. Then someone else put scotch tape on the labels, and put a very small almost invisible double arrow on there to indicate ‘hey, the labels are the other way around’.

But no one takes the time to just put the labels on correctly. No one feels annoyed enough to fix it for the next person. Amazing how people leave this easily fixed piece of friction around. (Guess what I did during that meeting)

The story of our AC and lighting is another one entirely, but it’s overengineered to the max. Here are the things that suck:

like I said, remotes are programmable. Sounds like a great idea, but sucks in practice if you really always only control lights and AC. The third row of buttons is never used. I’m sure the engineers went ‘Hey, wouldn’t it be cool if we made it programmable so that if we ever think of something else than lights or AC, we can just reuse the same remotes ?’ I can hear them discussing it right now!

hence, the remotes are unlabeled. Nuff said.

the remotes have no light that indicate whether they work at all. Batteries go flat – the bare minimum is a light that flickers when you push a button.

You need to sort of aim the remote at a receiver in the ceiling. But they’re pretty hard to spot, and the range is not very big.

Hence, wouldn’t it be nice if you actually got a feedback light on your remote that indicates ‘the receiver noticed the command ?’

If the receiver manages to receive the button press, apparently the signal goes to some central location where the whole machinery is housed. I imagine an ENIAC computer filling the whole basement somehow. It takes ages to get from ‘pressed a button’ to ‘lights go on’ or ‘AC goes on’. I mean, easily, 10 seconds.

All of these huge usability mistakes combine to make turning on or off lights or AC into a giant button mashfest not seen since the days of playing Summer Olympics joystick waggling and button twiddling at the arcade was in fashion.

All of this, of course, is in addition to all of the usual problems with AC – the controls for temperature are also unlabeled (and in a company with new people arriving constantly, there will be at least one person thinking that left is complete minimum and another person thinking that right is, and they will fight over this passive-agressively), it never provides the amount of cold people want, some people smoke in the building anyway so it gets shared for all of us, and filters never get changed as often as promised. Oh, and a few times a year, water drips from the ceilling (and it’s not that much fun to find your laptop turned on and wet in the morning).

And the kicker is, the whole system can be programmed on a time basis to get turned off at certain intervals. I still don’t know if it’s a good thing or a bad thing that, around 22:00, the lights turn off, and that tells you to either go find the remote or to go home because it’s long past time to do so.

2008-6-263:57 pm

Every time I want to reply to a comment on my blog I am stupefied at how badly commenting works in WordPress.

Here’s what I can’t figure out:

Why can’t I comment on a specific comment and start a thread of discussion ?

Why can’t I easily quote some other comment ?

Why can’t I comment from the admin interface ?

But these probably are relatively minor. The number one thing that annoys me is how this comment system does not help at all in creating a discussion. It would be so much better if, assuming there was a way to comment on a specific comment:

your comment to someone else’s comment would get automatically mailed to that person (if he left his email)

that mail contained a link that would take him to “create a new comment on this comment”

This way, you can keep a discussion going without too much mental friction. Imagine how I would have to do it now – I would have to comment in general (using the @ symbol to indicate at which commenter I’m directing my comment), send them a mail (hey, I saw your comment, and replied), cut and paste my comment or send them a link to it, and ask them their opinion. Then if they still cared enough, they would give it to me, I’d have to ask them if they’re ok with posting their comment again, and then manually post it again. And so on.

Right now often I don’t reply because I’m pretty much sure these people aren’t spending their free time going back to comments they’ve written to see if they got a response. So much possibility for goodness wasted…

My hope is that one of you knows of a plug-in that just Does The Right Thing here. Care to share ?