Developer and other things

Welcome to my little corner of the internet. I make things with computers, usually enterprise (shudder) websites but sometimes desktop applications or iPhone apps.

I enjoy working with various technologies and learning about how to improve my craft by trying out new things. That is not limited to learning methodologies & patterns like functional programming, but also development in a broader context, such as lean/agile funtimes.

You can find me on twitter.com/quii. If you have any comments on the stuff I write I would be thrilled to hear them, even if they're mean.

Unused features are technical debt

Technical debt is often described on a low level, where developers will take some kind of shortcut in order to deliver a feature quicker; at a cost to future development which must be “paid” sometime.

However technical debt can manifest itself in many different ways and one way it has reared it’s head on a project I am working on is by unused features.

Delivery culture

Agile development has many flavours and in my experience it has been broadly about improving a team’s capacity to release software by improving processes.

For the team I work on, it has worked out great. We generally deliver things in good time and we work well in a productive and enjoyable environment.

Right from the very top of the business at even the board level, the whole company is focused on delivering software and features to the areas of the business who need it. In our company we have a number of departments who fight for the features they want on our main website; plus a number of other products. These requests eventually filter down to the development team and we build them.

From there, that’s more or less it; and that’s the problem.

When a company becomes delivery focused, it forgets that it should be value focused.

Build - Measure - Learn

I’ve been reading Lean Startup and it focuses heavily on validating your assumptions when building products. It seems obvious really; you need to verify that the features you make actually have value.

From my (admittedly limited) experience with agile, teams invest a lot of time in terms of reviewing processes in order to deliver things more efficiently; but it doesn’t matter how efficiently you deliver a feature that isn’t used; you are wasting money by doing so.

This can sometimes be a hard sell to the business. People have forged jobs for themselves by making decisions on what should be built; based on their previous successes. It takes quite a culture shift for a business to acknowledge that it’s assumptions about it’s customers could be wrong.

However unless you validate your learning with actual real hard numbers, all that you are doing is guessing.

When you consider how expensive software development is, guessing is not good enough.

This is not to say that we shouldn’t build anything without some peer reviewed, scientific thesis. In fact, building something and learning that customers don’t want a feature is actually a really valuable piece of knowledge for a business to learn about their customers.

Lean startup talks about this in greater detail and I really do encourage you to check it out.

So why is it technical debt too?

We have established that building unused stuff doesn’t make business sense (duh); but it has many other unintended side-effects, which I will illustrate with an example.

I am currently working on a project which is taking some useful code we have within our big-blob mothership product and trying to extract it into smaller useful services for the rest of the business.

While doing this we invested a lot of time trying to figure out how we can refactor certain areas of functionality so that they carry on working as we break up the system.

Of course, a BA comes along who happens to look at Google Analytics and then tells us “oh well only a tiny percentage (like 0.0001%) of our users use that anyway”.

facepalm

When you don’t measure the success of features but leave them in the system, developers have no choice than to treat all the features on the product equally.

Developers have to be mindful of these features when trying to change the system to meet the business’s needs; and this can have a considerable cost.

I felt frustrated, and I’m sure the business would too, knowing the time and effort wasted; simply because we don’t know our product (and customers) well enough.

Side effects of unused features

I am quite confident that the business realises that a number of features on the site are much less important than others; but probably think “well what’s the harm in it being there?”. I shall try and debunk this line of reasoning here.

Every unused feature effects a businesses’ ability to deliver value and makes the product worse.

Build time

Builds can often slow down the velocity of a team. For us functional tests with Selinium have often really slowed us down. Unused features are a pointless way of slowing down your build.

Refactoring

The best commits are the ones where you delete code as it’s less code to maintain.

These so-called harmless features can reduce the malleability of your product because every time someone wants to change code they might have a dozen tests fail for a feature no one cares about.

Additionally, developers might think that generalising code to fit a number of different features could be worthwhile effort; but what if the dev could just delete them instead?

More to learn and understand

By having lots of features in a product, it simply becomes harder to understand; this again makes it more difficult for your team to deliver. In particular when making larger changes.

Demotivating

Knowing that you have to work with code which isn’t actually useful, is a frustrating and demotivating thing for developers to face.

On occasion we have measured some feature’s success and it was really exciting to see when a feature we made was really successful.

Every story should be like that.

Performance

A number of features we know intuitively (or with analytics) that are not used, are slowing down the delivery of features which are actually useful because they are on the same web page and happen to be backed by slow queries.

UX

Simple is good right? Why are you cluttering your search page with features no one uses? Unused features make your product harder to use and distract users from what you actually want them to do.

Moving on

There are plenty of tools now to validate learning. Metrics are trivial to assign to features and A/B testing frameworks can help you compare features and how a certain configuration can be more beneficial than another.

Rapid prototyping is an excellent tool to quickly test a business’s hypothesis before investing too much development effort and as I said, every failed hypothesis is still a positive thing for the business as it improves it’s understanding of it’s customers to help it make more informed decisions later down the line.

I feel with the advent of agile development, teams took on more of a responsibility for delivery in exchange for empowerment. I think the next step in the maturity of an organisation is for the people responsible for picking features to work closer with the development team; which have the tools and expertise to help them learn more about their customers and make better decisions.

If you can create a learning culture within your business you can improve the motivation of your organisation, learn more about your customers and be more efficient.

Intangibles

Things become more difficult when you are unable to measure the success of a feature reliably.

This might be because of a salesman promising X sales if feature Y is implemented on the website. The development team has little choice and makes the feature but then sometimes this “vital” feature isn’t used.

This usually comes about by customers not knowing their own needs, thinking they want a feature and demanding it from a salesperson; but they don’t need it.

Even though the feature is unused, it still technically has given value to the business; at the customer’s expense.

But eventually, the customer’s perception as to how valuable your product is, may diminish. When you base your business on these intangible things, you are running a risk.

In these cases it would be nice to try and work with the customer to try and find features that they want and that they will actually use, as chances are other customers might find them useful which could then lead to more sales than if you just left the customer with their pile of unused features.

It would be exceptionally brave to contact a customer and say:

“You demanded and paid for these features but we notice you haven’t used them, is there any way we can make them useful to you?”

but it could lead to some valuable insights about your customers and open up new opportunities.

Other examples of intangibles

“It’ll improve SEO”

The person purchasing is not the end user, but is buying on behalf of the users.

Conclusions

Software development has almost always been trying to become more efficient. Whether it is with different programming languages, new development techniques and different styles of project management.

Producing only what is valuable is vitally important but in many domains trying to define what is valuable can be hard, but still very important. Without this understanding the business will find it hard to find new ways to appeal to customers; which is too short-term of a viewpoint.

Thankfully these techniques are widely known these days and many companies can attribute their successes to them. In Lean Startup, it discusses how Facebook really understood their users and that helped them become what they are today. I have been to talks where engineers talk about how they often experiment and hypothesise what their users will like and measure it’s success.

It is up to development teams to evangelise this kind of approach upwards, to show how the business can get more out of it’s development; if it is willing to work more with developers.

Scala - Just enough rope to hang yourself

Last week Dan and I did a presentation at the Springer offices for the Functional Media meetup about our experiences with Scala. We thought it would be interesting to talk about the mistakes we had made with Scala as we thought that would be more interesting than rehashing a number of other talks about why Scala is or isn’t good.

Scala is a very feature rich language and because of this there are many ways of doing things. This was how Scala was designed, to allow developers to express themselves in a flexible way in comparison to languages like Java.

This flexibility is a double-edged sword and I think this not only effects our team but can also effect the OSS community around Scala. Some developers see the power in being able to write DSLs or construct cool type systems which they think will be really useful but aren’t actually as generalised and good as they think and can hamstring other developers. As Jeff Atwood says writing re-usable code is hard and that is especially true when with a flexible language like Scala.

I can actually appreciate some of the author’s sentiments. If I started Scala without the aid of working with some very experienced programmers in a pair programming environment I imagine I would find it a nightmare too.

His complaints about def and val to me seem like inexperience in his team with Scala, which then means the resulting code is difficult for everyone. Again, this is Scala’s flexibility hurting a team. I actually think good Scala code reads really well, but it takes some discipline and knowhow. Scala can tempt you into writing code you think looks great but is very hard for other people to read.

Lessons from our talk

If you want to have a productive team working with Scala, we feel you need

An enthusiastic team - A team that really wants to learn Scala where individuals demonstrate new powerful ways of getting stuff done to the rest of the team

Pair programming - This helps newer people get up to scratch with Scala.

Code review - You want to encourage your team to learn how to weild Scala’s power but at Springer we make sure we have weekly team code review where we go over code so everyone understands

Question things - “The functional way” is not a catch-all argument winner. Leveraging Scala’s power is great but readability is the most important factor in code for us.

Monitor build times - You cant really avoid Scala’s compilation problems. But if you invest a little time with CI to measure it, you can take steps to manage it.

Generalists

There seems to be a demand on a lot of developers to be “generalists” who know a number of languages and can be productive with them. This is a perfectly valid goal as it means developers generally have a broad and open mind about technical approach.

I’m not entirely sure if Scala is well suited to this as to write good Scala requires you to put effort into learning and appreciating the language. It is a power tool and it means you have to spend time to know how to use it responsibly so you can write concise, readable code.

Some would say this is a bad thing, but after working with Scala for a number of years now and if I was to change language I would want to make sure I picked a language which offers the same power and flexibility of Scala; otherwise I would probably feel like I am fighting the language to get things done.

I totally understand when people accuse Scala being cryptic and bloated, but I think that mainly comes from either using a horrible library of just being inexperienced. I think the true power of Scala is that it gives me the opportunity to express the intent of my code without the need of complicated boilerplate. However, it takes time to get good at it and to be honest that’s what I like about it, there’s always more to learn.

The Chris Validation Pattern

After nosing through some Scala FP books I have been trying hard to write small and simple functions and then compose them together. This post will show an example of where this has taken some needleslly complex code into something terser and easier to understand.

Previously

When getting our feet wet with Scala we would more or less still write fairly traditional OO code with some nice functional goodies mixed in.

This is one of Scala’s great strengths and it creates a relatively low barrier of entry for OO programmers; but there is a wealth of cool stuff you can achieve with it without having to blow the dust off your copy of The Gang of Four.

In this scenario we are re-implementing an old piece of functionality, which is user registration validation. The old Scala code to be fair it’s completely fine. It’s unit testable, fairly easy to understand and gets the job done.

My colleague and I (also named Chris); when re-implementing this feature for a new shared service thought we could use our growing Scala & FP skills to make something a bit nicer; and to show off. I think it’s important not to rest on your laurels with Scala and really try new things out when possible. It’s not just Java without semicolons.

The Problem

You wish to apply a number of validation rules against something and have a list of error messages to return eventually to a client.

def validatePassword(password: String): Set[ErrorMessages]

The old code

For the first iteration of code there was a class for validation which had one public method which took the item you wished to validate. Within the class were a number of private functions that took the item and checked to see if the item was valid and returned a sequence of errors.

The respective functions are all called at the public function and the sequences are mashed together into one sequence of errors.

The validation functions have two responsibilities, applying the business rule and then returning a collection of errors when appropiate. By coupling these responsibilities the functions become less re-usable.

Imagine if the private functions were public so you could unit test them. To unit test the sub functions which contain the actual business logic, you need to check for the error message returned, rather than whether the field passes the validation or not.

It doesn’t look enough like a regular expression, which is how you know you are winning at Scala :)

Our Approach

When reading most FP books, they usually emphasise the idea of creating small “pure” functions which you then compose together to achieve greater pieces of functionality.

The ultimate solution: The Chris Validation Pattern

As I said, the colleague I did this was also called Chris so it’s not an entirely egotistical name. Anyway

By thinking in terms of functions as being “pure” and only having one purpose, the resulting code was created fairly organically.

The first goal was to make it so the validation functions only dealt with applying a rule and letting the caller know if it passed or not

This means that unit testing these functions becomes very easy and not dependant on the error messages. Now that they are decoupled from any domain specific error messages; you could see how they could be refactored into something more generic, such as “stringIsTooShort”.

The next task is coupling each function with an error message, which is obviously done by creating a map.

You then simply run all the functions against the value you are testing, filter out ones that didn’t pass and then take the right side of the Map; which results in your list of error messages.

passwordValidation.filter(_._1(input)).map(_._2)

Ok, so to a non-scala developer it may not be completely obvious what that line does; but if you read into it, you can see it’s only really performing two operations over a map.

Granted, this isn’t the most impressive piece of code but I think it illustrates well how by being really strict in the responsibility of your functions and decoupling things you can compose useful functionality elegantly.

Edit

Thanks to Dan for pointing out that you can use the values method of a map to retrieve the values rather than the wonky .map(_._2). I also found that filterKeys can make this code more readable too.

So, ultimate version:

passwordValidation.filterKeys(_(input)).values

My Christmas Song

In our house we thought it would be fun for each of us to make a Christmas song.

I found it a lot of fun, it was nice to sit by myself, take my time and try and make something okish. Generally when working with other musicians, they have been better than me and I always felt I needed more time to figure stuff out.

Granted, the bass is really the only well done bit, as I dont play lead guitar, sing, piano or even sing. Despite all this, it does seem to come together!

In previous attempts to make songs I have generally always struggled to make lyrics, so I just made up a stupid depressing story and ran with it.

The “story” revolves around a woman, who’s life takes a turn for the worse because of austerity times. She ends up on the streets and gets hold of some drugs. In a sea of strange hallucinations she convinces herself that she is a star that needs to climb to the top of a Christmas tree.

Tragically, the Christmas tree she is climbing is The Shard and this ultimately leads to her demise.

Enjoy.

Edit - forgot lyrics & chords

Just in case you want to play along at home?

Verse 1

C F
And santa watched her fall
C F
Dropped right into the Thames
Am G
And no one, gives a toss
Am G
And no one, gives a toss

Verse 2

And the reindeers were confused
Because she couldn't fly-y-y
Santa has a gift for you
It's a can of special brew

Chorus

Dm C
She was climbing up The Shard
F G
She thought she was a star upon a tree
Dm C
And she crashed down hard
F G
While we had some christmas cake and tea
C
Homeless and boneless
F
Pennyless and helpless
x2

Verse 3

Pigeons pecking at her brains
Deck the halls with boughs of blood (falalalala)
Death came down your chimney
Death came down your chimney

Understanding Javascript Objects

I have been reading the book Javascript Patterns, which is fairly easy reading and I would recommend it if you wish to understand Javascript a little better.

This post will describe JS objects, which is something you’ll come across when you’re writing almost any kind of JS.

Objects in Javascript are essentially hash tables of key value pairs. The values can be almost anything; strings, functions, other objects, etc.

It is worth noting that these objects are mutable which can offer some cool features but obviously has it’s drawbacks. This is why understanding scope is very important because otherwise you can get some unexpected behaviour when your object gets changed by something at runtime.

Here’s a basic example of this mutability, where we define an empty object and then add a property.

New

Using new will implicitly return the this object that you create in the function. You dont need to do var this = {}; - that is also done for you.

In other words what happens is similar to the first example in this post where it makes a new empty object and then adds the properties defined in the function.

Most importantly, the newly created object will also inherit the prototype of the function you defined.

Prototypes?

Javascript uses prototypal inheritence. A prototype simply defines methods that an object referencing it can run.

When you try to run a method of an object, if it cannot find the method locally to the object it will then try to find it in the object’s prototype.

Like most things in Javascript, prototypes are mutable which can offer some interesting (and insane) power to a programmer. You could have a bunch of widgets on a web page and based on an event, such as a user action change the functionality of all of them by changing the definition of their prototype.

Returning to the Person constructor function before, I defined a function called sayHello. Every time this constructor function is called and an object is made, a copy of that function will be made inside that object. This is inefficient and should be refactored to use a prototype instead.

This will potentially make unit testing easier too as you can simply create a mock object which has the prototype you wish to test.

Therefore, if you think your object will be created more than once and has methods you should definitely consider including the functionality in a prototype to save memory.

Dont forget new

Unfortunately forgetting new will not cause any obvious errors. What it will do is bind new to the application’s Global object, rather than the “new” this which would normally be implicitly returned. In a browser, this would be the window object; fail!

However, as far as I can tell it wont protect you against not using new

Constructor functions vs Object literals

So it seems there are two ways to create objects but which should you use?

If you wish for an object to just contain data then you should use object literals. It is also acceptable to use object literals if you are sure that you wont make another “instance” of your object to have behaviours attached to it.

If you feel you will have more than one instance of your object and in particular if you wish to add behaviour and alter it accross all instances then create it using a constructor function.