David's Blog

Fewer lines of code is better. Given that it sill solves the problem. Less code means less debugging, less testing and fewer bugs. A framework has to make sure that the programmers using it can express concepts succinctly and still clearly. Also, the framework or library should not encourage the developer to hide essential parts of the code just for the sake of succinctness.

Succinct is good

I guess an example would be handy right now. The example I used in my talk at "Mathema Campus" earlier this year is from Paul Graham's article Take the Arc Challenge. The challenge is:

Write a program that causes the url said (e.g. http://localhost:port/said) to produce a page with an input field and a submit button. When the submit button is pressed, that should produce a second page with a single link saying "click here." When that is clicked it should lead to a third page that says "you said: ..." where ... is whatever the user typed in the original input field. The third page must only show what the user actually typed. I.e. the value entered in the input field must not be passed in the url, or it would be possible to change the behavior of the final page by editing the url.

Imagine solving this challenge using your favourite web framework... In Apache Wicket, for example, one would have to write 2-3 Java Classes, 3 XHTML-Files and an XML file to solve the challenge. In JSXP it's just about the same effort.

This is the solution written in ARC, the Lisp-dialect created by Paul Graham:

The beautiful thing about this solution is that it is really short, but still does not hide anything essential from the reader: Everything from the original requirements can be found directly in the code.

Reasons

There are several reasons why less code is better. An important one is Working Memory Capacity (which I have explained in an earlier article from this series). Programmers have to remember large parts of the code correctly so they can modify it. They have to "Load" it into working memory - This is what gets programmers into "flow". Succinct code means that the programmer can "load" more code more quickly.

There is also some evidence that the number of lines of code a programmer writes per hour is independent of the programming language - also the number of bugs per line of code seems to be pretty independent of the technology used (see for example this paper by Ericsson). So, less code means less bugs and more productivity.

Finding errors is easier too: There is less code where the bug can be hidden, and there are probably less tests affected.

Too succinct

Can a piece of code be too succinct? It can, if it hides things that are essential to understand what the code does. Consider this piece of Ruby code:

while file.gets
print if ~/third/ .. ~/fifth/
end

This code reads all lines of a file, and as soon as one line matches the string "third" it starts printing all the following lines - until a line matches the string "fifth" (this is what the ".." operator does - it acts like a toggle switch).

But how does it do that? There are no variables involved whatsoever!

The code works because several ruby functions use global variables by default if no other parameters are given: "file.gets" saves its result in the global variable $_, the "~" operator can match against $_, and if "print" is called with no argument it prints $_.

Something very essential is hidden here: How the data flows through the program. A better - and still very succinct - solution would be:

while line=file.gets
puts line if line=~/third/ .. line=~/fifth/
end

Conclusion

Less code is better - unless it hides something essential about the solution. Make sure that programmers who use your framework or library can create succinct solutions. Don't force them to write boilerplate code or configuaration files where defaults would do.

Think about one of your favourite frameworks or libraries. Does it force you to create boilerplate code or does it enable you to create succinct, elegant solutions?

I am just at the beginning of my journey - My journey towards becoming a better software developer, towards understanding how teams work and towards helping others. I will always be at the beginning. I hope I'll never forget that.

Yes, I am a Certified ScrumMaster. I am also a Kindergarten Graduate. Both were really good places to start, not good places to stop.Kate Oneal

Whatever we have learned so far is just a good place to start. But I think there is no good place to stop - ever. Sometimes it is hard to admit that. And sometimes it is too easy to become arrogant. I sometimes have to remind myself that there is a lot more for me left to learn. Even about things I think I already know (*).

If there is no good place to stop, then the way ahead is longer than the way already traveled. Because it never ends. This means that I am still at the very beginning (or very close to the beginning). But that's a good thing. Nothing ever really ends. There is always a chance to learn. There is always a way to improve. To make something in our world a little bit better.

Molly Grue: But what if there isn't a happy ending?
Schmendrick: There are no happy endings, because nothing ends.The Last Unicorn

When was the last time you thought you didn't need to pick up a book or go to a training because you already knew the topic? Were you really right?

(*) For example, I just bought the Design Patterns book because I realized that I never read it. I learned most of the patterns from other sources.

Up until a few months ago, I used to work on side projects mostly after work, for just about an hour or two. Often I did not really get much done. Now I have set aside one day per week where I work on my side projects, and this greatly improved my productivity. Then, a couple of weeks ago, I won a book about the pomodoro technique in a contest organized by the Trello team, and so far I am quite impressed: It's amazing how much you get done when using this technique!

One day per week

After I had finished the project with my last client in February last year, I had some weeks where I could work exclusively on side projects. During this time I wrote JSXP2 which is almost a complete rewrite of JSXP and I wrote most of the code for gclimbing.com. I was very productive.

Before and after this time I was working full time for clients. I only worked on my side projects for a few minutes to a few hours after "real work". Getting things done was much harder when working like that. I was tired after a long day of work, and the relatively short time I was working on my side projects meant that I never really got into "flow". Still I was quite productive some time, but mostly when I had longer blocks of time where I could work on a single project.

This year, I decided to set aside (at least) a whole day per week to work on my own stuff. I only work on client projects for four days a week. Monday is my side project day. This is when I write code for my side projects, try to learn new things, write blog posts, read books, prepare talks and so on (*).

Now that I have a whole day per week I get much more done. I know that not everybody can work in their "day job" for only four days per week, because most employers will not allow this. Anyway, setting aside large chunks of time for side projects once a week works better than working every day for 30-60 minutes - at least for me.

Pomodoro technique

A few weeks I discovered the pomodoro technique because I won a the book about it. It is quite simple: Write a TODO list for the day. Work for exactly 25 minutes, then take a break. This is called a "pomodoro", because the time is measured with a kitchen timer shaped like a tomato (The italian word for tomato is pomodoro. A differently shaped kitchen timer might work too). Log all interruptions during these 25 minutes and try to minimize them. Log how many "pomodoros" (pomodori?) you spend on every item from the TODO list.

The book about the technique is interesting and easy to read. It is also quite short. I have been trying this technique a couple of times now, and so far I really like it. When working for 25 minutes without any interruptions you can get a lot done. And the breaks make sure that you can concentrate for the full 25 minutes in the next pomodoro. I think I'll need some more days of using this technique until I get really used to working like that. Still, even after a few days I see improvements in productivity yet I don't feel overworked, so this technique really seems to work!

(*) I also read books and work on blog posts when riding the train an on other occasions. But: Different Story ;)

How can we make sure that our code is extensible? How can we enable or even encourage the users of our library to use the library in new, interesting and unexpected ways? First of all, Simplicity is really important here. Also, we should support inversion of control (IOC) - and if possible we should not constrain our users to a specific IOC container. You can deliberately create "leaks" in your abstractions. And, last but not least, lambda expressions and closures can be an awesome tool for providing extensibility.

Simplicity

Only simple code is extensible.Useless abstractions in the code, trying to anticipate future requirements and even adding design patterns where they are not needed can make the code unnecessary complicated. Just as I have mentioned in my blog post "Simplicity", the benefits of these things are local and grow linear, while the drawbacks have global effects and grow exponentially.

Inversion of Control

Inversion of control - and especially dependency injection - can make extending existing code easier (even if it violates the direct call pattern - at least somewhat ;) ). This does not mean that your framework should be an IOC container. It also does not mean that you should force the users of your framework to use a specific IOC container.

You should just make sure that the framework or library you created can be used with an IOC container. Wicket, for example, lets you specify an "Injector" which is called every time a component is created to inject dependencies into this component. There are several Injector-implementations that support popular IOC containers, like Spring or Google Guice.

Leaks

Not necessarily. You can even create such "leaks" deliberately if it makes extending the framework easier! For example, Apache Wicket abstracts the fact that you are dealing with a web container. But sometimes, when writing a web application, direct access of the container would be handy! And it does not really make sense for the wicket developers to wrap all of the functionality of the container.

So, when you really need access to the container, you can get it:

getRequest().getContainerRequest()

You can do this too: Create your abstractions for the common case - for the 90% of the use cases where creating abstractions is easy. But, when these abstractions are not enough, allow your users to access the underlying technology in a controlled and defined way.

Lambdas and Closures

We want to write a function that generates accumulators-- a function that takes a number n, and returns a function that takes another number i and returns n incremented by i.

(That's incremented by, not plus. An accumulator has to accumulate.)

In Javascript the solution looks like this:

function accgen(n) {
return function(i) {
return n += i;
}
}

A function that creates another, anonymous function. How would the solution look in Java? Well, you can not solve this problem in Java - That is, you can not write a (reasonably short) solution that fulfills all the requirements. You might think "Well, nice, but what does that have to do with extensibility?". With anonymous functions and closures it is very easy to write reusable algorithms, like map reduce.

Of course you can simulate a lot of this behaviour in Java too, but there is a lot more boiler plate code you have to write. So, if your programming language supports closures and anonymous functions, use them to make your libraries and frameworks extensible.

When writing a framework or library, we have to simplify things and create abstractions. But: With every new concept or abstraction, our users have to learn more. They can not simply forget about the underlying stuff. And they can not simply ignore other concepts. This does not have to be a problem, we just have to think about it.

The law of leaky abstractions

Abstractions make working on large code bases possible: We don't want to deal with low level stuff. In fact, we can not deal with it - A large project would not be feasible if we had to write it in assembler. Or if we had to take care about disk I/O or scheduling. So, we need libraries that create new abstractions and/or simplify things for us.

Wicket

This means that the abstraction, in some cases, does not work anymore. The user has to "dig deeper". The underlying concepts "leak" through the abstraction. For example, Apache Wicket tries to hide the fact that we are dealing with HTTP, a stateless protocol. Writing a web application with Wicket feels a little bit like writing a desktop application: There is a flow of events, components pass data to callback methods, etc. The communication over HTTP is not an issue, ever.

Except when it is. Every wicket component has a strange method:

setOutputMarkupPlaceholderTag(boolean)

To really understand what this method does and why it is needed, one has to understand how AJAX works and that we deal with a stateless protocol here. This method is only needed when a hidden component should be manipulated with an AJAX call. Hidden components are normally not written to the output (HTML) at all. The AJAX request is a new, stateless HTTP request, and the only thing at the client side that can be manipulated is the HTML from the last (stateless) HTTP request. Because of this, when an AJAX result should manipulate the hidden element, there has to be a HTML element which can be replaced with the AJAX result - the "Markup Placeholder Tag".

Moq - .NET

Another example is Moq, a neat mocking framework for .NET. Moq is great: It uses lambda expressions to configure mocks - They are checked by the compiler at compile time and refactoring safe. Configuring the return value of a method looks like this:

mock.Setup(
foo =>foo.DoSomething(It.IsAny())
).Returns(true);

This means: If a method called DoSomething is called on a given mock object ("foo") with a string parameter (It.IsAny() - the value of the string doesn't matter), then return true. There is some magic going on behind the scenes to make this work, but normally you don't have to care about that. Except when the magic stops working: Like, when you want to mock a protected method. Then, suddenly, you have to write different code:

If you want to truly understand why this difference exists you'd have to understand how the magic works in the first place and why this is not possible with protected methods. The easier way is: Just use the second code snippet for protected methods. This probably works for most of the users of this framework, but when there are unexpected problems, they still have to dig deeper.

What do do

So, all abstractions we create will be leaky. But we have to create them, otherwise developing large system might not be possible anymore some time in the future. For every new concept our users potentially have to deal with 2 concepts: The new one, and the old one. But inventing and creating new concepts is part of the essence of our job as programmers.

Are these "leaks" a bad thing? Not necessarily, I think. We just have to be aware of them when designing a framework. And we have to make sure that they happen at defined places, and in defined situations. We have to know the limits of our abstractions and concepts. And we have to look at our framework or library from the perspective of our potential users. Both are a good idea anyway.

Anyway, we should be really careful with creating new abstractions in the first place. We are always at risk of causing more trouble for our users than what they get as benefit: They have to remember more concepts and their programs become potentially harder to read, write and debug.

Last week I tweeted "Just *try* to tell a construction worker to use a cheap plastic drill because a hilti is too expensive." - A statement that I shamelessly stole from a co-worker. To most people I know, the idea of professional construction workers using cheap, crappy tools seems hilarious. But what about software developers? Well...

When I tell people that I use an expensive mouse and keyboard, or that I bought a mobile workstation for 5x the price of a cheap laptop, or that I have a really cool office chair, I often hear something like "Whoa! Why do you waste your money like that?"

And when it comes to software... "A text editor for $60? Don't you know Windows comes with one out of the box?" Or "Why do you need a $499 Profiler? Aren't there cheaper ones?".

Good tools are (sometimes) expensive. [*] But they are always worth it. Working with bad tools is not only inefficient, it is also frustrating. And frustration kills productivity.

I hear these comments about wasting money even in our own industry - from fellow programmers and from my customers! I worked for several customers where it was not possible to get a second monitor. One customer wouldn't even let me bring my own. I never had a good mouse or keyboard when I had to work on computers provided by my customers. And don't get me started about office chairs...

Of course I could use the cheap keyboard and mouse that came with my computer. And I could sit on a chair from Ikea. I could work on a $500 laptop. I could use the built-in profiler of Visual Studio or Java. The construction worker could drill holes with the $50 drill from the discount store.

But: I use these things 8 hours a day - or longer. I want them to feel right. I don't want my hand to hurt after using a cheap mouse for a day. Good tools allow me to do my work more efficiently. And for some kinds of problems the cheap tools won't work at all or take way too long.

Good tools are (sometimes) expensive. [*] But they are always worth it.

[*] Andsometimestheyarenot. But that's a completely different story. Also, expensive tools are not always good.

A framework or library should make it really easy to write code. It should be obvious how the library can be used and how the functionality can be extended. Also, we should help our users save keystrokes by supporting the auto complete features of our IDEs. And we should make it easy to write testable code and to find errors in the resulting code.

Don't repeat yourself (DRY)

And by "yourself" I don't mean "yourself" (the framework developer), I mean your users. That is: Allow your users to follow this principle - do not force them to repeat themselfs. For example, Ruby on Rails has several features that enable writing code without repeating stuff.

With other frameworks, DRY is very hard or even impossible. Take android for example. This is some content of a layout file for an activity:

myactivity.xml

<Button android:id="@+id/submitButton"/>

But wait! The id has to be unique across all activities, so better prefix it:

<Button android:id="@+id/myactivity_submitButton"/>

Now we repeated some information: That the button "submitButton" is part of the activity "myActivity". When you write the Java class for this activity it gets worse. You have to tell the system that yes, the layout "myactivity.xml" is the layout for "MyActivity" and that the "Button" object with the name "submitButton" corresponds to the button with the id "submitButton" from the xml file:

In a similar way, Apache Wicket forces you to repeat some information from the HTML files in the corresponding Java classes: You have to create the same components and the same structure in the Java class. This is one of the reasons why Oliver Szymanski and I started the JSXP project.

Auto Complete

Never underestimate how much help the auto complete feature of your IDE can be for your users - and how annoying it can be if your framework does not support it well.

Just try this in a Java IDE:

JButton button = new JButton();
button.set[Ctrl]+[Space]

(Or whatever your shortcut for auto complete is) You'll get an endless list of methods without any way to sort or filter them. For example, from this list you can not filter the methods that only effect the view.

In one version of JSXP2 (my re-write of JSXP that is not publicly available) I have also created some code where auto complete does not really support you, as a user: In a view controller in JSXP, you can get all the components of the page that are definded in XML by their "jsxp:id". But auto complete gives you this:

Only the last suggestion here is relevant to what you want to do: "getElementLoginForm" is the first method here that returns an element from the HTML page. To get meaningful auto complete suggestions, you have to type "getElement" before. Later I tried to group these methods into a protected object, which works a little bit better: In this version, you can access the login from with el.loginForm();.

Finding Errors

Help your users to find or - even better - avoid errors. Let me start with a negative example again: In Apache Wicket, you have to match literal strings from the HTML file in your Java code: The "wicket:id"s from the HTML and the ids in Java must be the same. And the components must be nested in the same way.

And you can only find errors by starting the application - or unit tests with WicketTester. You should absolutely use WicketTester. So, yes, you can find these kinds of errors automatically. But the situation is still pretty bad:

It is really easy to produce these kinds of errors

You find out pretty late - only when running the program

There are a lot of "false positives": Changes that should be compatible but still cause this error, because the structure has changed a little bit.

The thing is: Many of us use a statically typed language. And this has some advantages [*], at least it should have. One of the advantages is that we know very much about a program during compile time. We should use this knowlege to detect errors early - And most of the time, this is not done, so I see a lot of wasted potential here.

The thing is: Actually, we don't know anything about the program. The compiler does. So, if we can somehow phrase our expectations about the program in a way the compiler understands the compiler could make sure that our expectations are met. This is easier than you might think: The Java compiler understands Java, so we have to write about our expectations in Java. And to make sure the error is not caused by our expectations, we should generate this code.

I guess you think right now "an example would be really handy at this point". Well, ok ;) The second reason Oliver and I wrote JSXP was to eliminate certain kinds of errors.

When you defined an element in the view (HTML file) you should be able to make sure that it is available in the controller (Java file). And when you change these files in an incompatible way, you get a compiler error. A code generator is responsible for bridging the gap between HTML and Java: From the HTML file we generate code that lets you access the elements in a checked, type safe way, by calling a method. When you change the HTML or the Java code in an incompatible way, the method call can not succeed, so the compiler shows you an error.

I think it's time for some code, but not from JSXP but from another project where I experimented with this technique. resdroid is a project that tries to bring the concepts of JSXP to Android. It generates code to make sure all IDs are unique and to provide a compile-time-checked, type-safe way to access widgets within an activity. Check it out, but beware: It is just a proof of concept at the moment!

The activity from the example above would look like this when you use resdroid:

myactivity.activity.xml

<Button android:id="@+id/submitButton"/>

From this file, the resdroid generator creates the real activity layout file where all IDs are prefixed so the IDs are unique across your App. It also creates a base class for your activity, where the button is defined as a protected member of class "Button". The layout file and the button are initialized correctly in "onCreate", so after the super call you can rely on it being available. Your own activity class could then look like this:

Conclusion

Make writing software based on your library or framework easy for your users! Make sure they don't have to repeat certain kinds of information. Think about auto complete when designing public interfaces. Make testing easy and make sure certain errors can not be made or detect them at compile time.

[*] And also several disadvantages, I know. But this is an entirely different topic. Maybe for another post.

Today I wanted to write the second part of my Framework Design mini series - but then I found out something interesting about HTML5 canvas. I will write the framework design post later this week.

Soooo.... I was again experimenting with the HTML5 canvas. When I was writing scribblingspree.com, I found out that the canvas uses it's own width and height properties when calculating coordinates, no matter how big it really is. This means, the following canvas will work as if it was 100x100 pixels large, even though it really is larger, depending on the screen size:

So, in the above canvas, a line from [0, 0] to [100, 100] will be drawn across the entire window, not just from the top left corner to the point [100px, 100px]. This behavior was pretty consistent across browsers - at least on the desktop, but also on Android - so I calculated a factor based on canvas.width and canvas.scrollWidth and used that to calculate the correct coordinates.

Also, when processing mouse events, the event object has properties offsetX and offsetY. These properties represent the mouse position relative to the element on which the event occured. So, these properties can be used directly for calculating the coordinates within the canvas as described above.

This did not work on the iPad. Not at all.

It seems like the browser on the iPad uses the actual size of the canvas to calculate coordinates. Setting the width and height of the canvas to be equal to the scroll width and scroll height did it for me, but it only works as long as the window is not resized. On resize this has to be done again.

Also, the touch event on the iOS browser does not contain the properties offsetX and offsetY. This means that I have to work with the page coordinates of the touch event. But there is one little inconvenience with using the page coordinates: One has to know the top and left offset of the canvas, which is not immediately available in JavaScript. With this neat trick I came up with the following solution that works on the iPad and on the desktop:

Update: sendResponse now uses a TextRequestHandler instead of writing the response directly. This prevents an IllegalStateException "Header was already written to response" when sending the data.[/Update]

A while ago I wrote the blog post A wicked problem with jQuery and Wicket. Back then I did not find a solution to the problem. I found some information on Stack Overflow, but I could not make it work properly.

Here is the problem again: On the client (browser), I want to use jQuery to create an AJAX request and send some data (encoded as JSON) to the server, which runs on Wicket 1.5. The server should send some JSON data back.

Now I have a version that works, and I want to share it here.

The HTML page

I have a base page which include jQuery on all pages:

BasePage.html

Then there is the component where the AJAX request should be sent. This component is a wicket panel and is meant to be included in a page. The including page must load the jQuery script (in my application, all pages include the jQuery script, since all pages are derived from BasePage).

The Java-Code: Component.java

The component class contains an inner class for the AJAX behavior. It also contains 2 member variables: The WebMarkupContainer that represents the canvas and the ajax behavior. We also have to override onBeforeRender (which is explained later).

Note that we add the ajax behavior to the component, not to the canvas. I guess it does not really matter where it is added, but I have not tried to add it anywhere else. The above code works for me, so why bother ;)

Now we have to add the callback URL of the ajax behavior to the output. The Javascript code reads the callback URL from the attribute "my:canvas.callback" from the canvas. We can not do this in the constructor because the method "getCallbackUrl" requires a page. In "onBeforeRender" the parent page of the component is available, so we put the code there:

Note that the super call "super.onBeforeRender();" is mandatory! (Side note: I don't like it when frameworks do that. A super call should always be optional. Maybe I should include this somewhere in my mini series about framework design.)

The AJAX behavior

Ok, we are almost done here! All that's still missing is the AJAX behavior. Here we have to override the method "onRequest":

Update 2: The behavior itself can set the attribute for the callback URL. Then you do not need to override "onBeforeRender" in the page anymore. For this, the behavior has to be added to the canvas component, not to the page:

And now we're done! This code has been tested with Apache Wicket 1.5.2 and jQuery 1.5.2 (The matching version number is a coincidence. Really!). I have modified the code a little bit for this blog posting - that is, I shortened it a little bit.

My overall impression of the conference is very positive, but there were a few negative points, so let me start with them:

Beverages (coffee, water, ...) were only free during some official breaks, otherwise you had to pay for it.

You were supposed to pay 30 cents to go to the toilet. There is no way that I'll pay for that after paying 150 Euros conference admission! Using the toilets must be included!

Some of the sessions were rather boring, and the first keynote by Prof. Jean-Pierre Seifert was outright bad (unstructured, lots of bullet points, often he was talking about a completely different topic than what was on the slides, ...). (Side note: Why do speakers from the USA deliver better sessions than than german native speakers?)

But: There were some really good sessions too.

My personal favourite: "Dragonbot" by Adam Setapen from MIT Media Lab. He told us about a robot they develop which does amazing stuff and is powerd by an off the shelf android phone. He also showed some videos of earlier prototypes, including the incredibly cute Leonardo.

Also really good: The Keynote "What's new in Android" by Mike West from Google. The topic here was (mainly) Chrome on Android. The talk was delivered very well and he had beautiful, easy to understand slides. The topic was very interesting too. I think I should try Chrome for Android in the next couple of days.

There was a session about developing for Windows Phone, and I must say, I am impressed. It all looks really easy (My WPF experience would really come in handy here), and the emulator does some amazing things. And: The emulater is fast. A completely different but also very interesting technology was shown in another session by Motorola Systems: Rho Mobile is a way to develop cross platform apps for all kinds of devices (even Windows Mobile!) using Ruby, HTML5 and JS.

The session "Developing fault tolerant Android applications" by Andrew Levy from Crittercism was very interesting too: It was no sales pitch at all, he provided some interesting statistics and common real world problems. He had nice slides and delivered his presentation very well (of course, I mean, he is american :) ).

I also tried some phones: The Xperia S from Sony is great - I want one of those! The Xperia sola (announced yesterday) is a really nice phone too. I also played a little bit with some windows phones at the Nokia booth. They really feel solid and the operating system is very smooth, I think these are good phones. I guess I won't switch though - I just like android better (at least for now).

Pages

My name is David Tanzer and I have been working as an independent software consultant since 2006. I help my clients to develop software right and to develop the right software by providing training, coaching and consultanting for teams and individuals.