A journey in software development

Category Archives: Agile

Those who have worked with me have most likely heard a few times and probably also got bored by me saying we have to establish a core set of common values before we do any code writing. What I mean by that is that you can’t have a team work in the same general direction without the team actually believing in the same things. Take a thing like being test driven. If you have a split in the team and not a common consensus of wether or not automated testing is good or bad for your team and your product, you’ll end up having tiring discussions and not establishing a climate for establishing a good culture. Worst case; you might hurt your velocity directly by not adressing a bunch of the elephants in the room and over time establish a declining velocity and not notice that it is declining.

Elephants

Yes, there always seem to be an elephant in the room. And they are so hard to get out of the room. Subjects that are impossible to discuss, because you really can’t get reach an agreement. Every place I’ve been to have these, and in my experience, there seem some be a tendency to more than average in software development. I think this comes from a bunch of reasons, but in my experience it boils down to not adressing the underlying issues and also in many cases, really understand why we go to work. In our industry there are some elephants seem to be quite common, things like my favorites; performance and the “keep it simple stupid”-principle. Performance is by some the trump card, whenever a discussion gets out of hand, you might run the risk of this card; hoping it will kill the discussion and we can move along.

Business Value

Why we as developers go to work is something that can be hard to remember. Our job is to add business value to the place we work. It is so easy to forget this fundamental thing and end up doing things that is not related to this, often based on an established culture of developers having the ability to just ignore this fact. By having the power to create something in code makes us experts in our fields, we do something that not many people understand what is. This power comes with great responsibility, we should not abuse this just because we want to do something exciting. Its easy to bullshit a manager into thinking that using technology X will solve all problems, while you could have done this with a different approach in shorter amount of time but you never got the chance of trying out X. Don’t get me wrong, we should always look at tools that actually improve our productivity or helps us get our business value done more accurately or delivered faster or better. Just don’t make up excuses to get to try it out to get it on your resumé.

Lack of understanding

Lets face it, not everyone wants to learn new tricks all the time. And that is actually fine. Being conservative to change can help balance out those who wants to change everything all the time. That being said, there are also those who are conservative because its convenient, because it suits them to not necessarily go in the right direction and hinders development and moving forward. A great way to protect your own job for someone who has been there since the dawn of time and really just want to continue doing the tasks that has been mastered. In our industry I think you’d be crazy to even try to want something like this. Our industry moves quite rapidly compared to most others and knowing only things that belong to the past is not necessarily a benefit. I think this is also important to understand for some of the elephants in the room, people protecting themselves; they just don’t necessarily understand the new ways of doing things and don’t have the full motivation to actually bother learning it. A good manager should be able to pick up on this and make sure the team is motivated and keep the risk down for the company by actually facilitating for learning.

The User

Who are these users we hear about. Adding business value is good for business, but not thinking about the user and keeping them close could be a disaster for business. So even though you think you’re adding something to your system that you believe will add business value, the users might not want it because you made it in a way the users won’t understand. This is something I see all the time; developers making software look like their development tools and thinking the users will just intuitively get it. Heck, I’ve done this myself on numerous occasions. Its like we get tunnel vision, thinking that everyone thinks like us. They don’t! We are possibly the worst frame of reference for what a user wants. This is one of the things that I think brings more elephants in the room that are not up for discussion. Because we’re used to do something, that wasn’t good for the user, we should just continue down that path.

Being pragmatic

This I hear a lot, and it gets presented as an accusation, as if one is not pragmatic. Short and simple; it is an abused term. It is something that gets twisted to fit as a trump card and thrown on the table to stop discussions. It doesn’t really reflect a value of any sort, its just a way of hiding any core value of actually bubbling to the surface. Its often related back to the “keep it simple, stupid”-principle and is just an instrument used to belittle the other person(s) in the argument. I look at other professions whenever I encounter things like this and ask my self, would they throw this card — ever? Take a plumber, would he pragmatically drop putting in a pipe or even worse pragmatically decide to not comply with building regulations. We can do much better than this. The KISS principle is a really good example of something that has completely different meanings depending on the person you talk to as well. For some this would mean; “… put everything in a stored procedure in the database…” type of KISS, while for me it means I adhere to the SOLID principles amongst other things. I don’t like these terms – they don’t bring anything positive to the table.

Core Values

Back to the original intent of this post; establishing core values. This is not an easy task, it is something that can take a while and it is not something you delegate to one person and hope it solves everything. It is a team effort. The team has to do this together and they are going to have to work together to get the wheels smoothly running. An approach that I’ve had great result in slight variations with, is to let every team member write down on three post-its the three most important things to them. One is not to discuss the items with anyone else, the items can be anything related to what the person consider important to be able to do their job. Every team member has to do it themselves for themselves. Put this up on a wall and let every team member present the case for their items and what they mean. You then optimize the wall by grouping the things that are the same. We now want to cut the list down, we can’t have all the items as our core most important values. Then you give the team 3 votes they can distribute on the things they now find the most interesting with the knowledge that we’re not keeping them all — chose wisely. You should now have a ranked list the things with the most votes are more important and you do a cut off at the number of items you decided to keep. I usually tend to keep it at 10. With smaller teams you might consider increasing the number of post-its and votes, so that you get more than your cutoff, even after grouping. A variation of this that I’ve used was after a disaster of a release at a company were we had a yearly release-cycle. After the disaster and weeks of firefighting I put the team together and asked them to come up with 3 post-its each for how we could make the next release even worse. This triggered something very interesting, people got really creative and every one had great ideas of how to really sabotage the next release. We then voted for the things we found most relevant and that we wanted most to do. We then took the list and we converted it into things we should not be doing and then had a cut off of 10 items that became the law of the land for the development team.

What you now have produced is something that can kill the things used to kill discussions, you have a list of things you all have agreed upon is important. It is the compromise, the things that you as a team believe in. Whenever nonsense arguments are being thrown into the mix, the different trump cards or elephants that sit in the room, you can refer back to the list.

Sometimes it is not clear cut what things mean, and you can end up having discussions about the meaning of it all. Try to capture what people are saying when presenting their post-its, put this in writing and get the team to read through and commit to this. But even after doing that, it might be hard to understand. A tool that can help with getting a common understanding is doing pair-programming as part of how you work. Circulate who pairs and start building the dynamics into the team. You end up discussing some of the items on the list and you will eventually break down barriers and create a team that at the very minimum executes more in unison. Typically the elephants can actually be taken out of the room and you will hopefully and most likely also address the lack of understanding and get to the need of learning much easier.

An example of the process is below, this is from a real exercise.

The things that made it into the list

The things that didn’t make it into the list

The board of votes

Conclusion

Though probably not perfect, it is a way of getting the conversation going and making everyone aware of the fact that we have to be on the same page. You should never underestimate the importance of having the team thinking pretty much the same, and you need to address the core belief system in order for you to be close to getting your team thinking in the same terms. Having a team that anticipate each other and can have a dynamics together that pulls in the same direction gives higher velocity in the long term and also helps people keep motivated. Nothing is worse than not being motivated to go to work because of elephants and established truths. It can feel constraining and for someone not in the same mindset, completely arbitrary because the world has moved on ages ago.

Bottom line; communication is super important and probably the hardest thing to get right.

I was at a client a while back and was given the task to asses their code and architecture and provide a report of whatever findings I had. The first thing that jumped out was that things did more than one thing, the second thing being things concerning itself with things it should not concern itself about. When reading my report, the client recognised what it said and wanted me to join in and show how to rectify things; in good spirit I said:

We approached one specific class and I remember breaking it down into what its responsibilities were and fleshed it all out into multiple classes with good naming telling exactly what it was supposed to do. When we were done after a couple of hours, the developer I sat with was very surprised and said something to the effect of; “.. so, single responsibility really means only one thing..”. Yes, it actually does. A class / type should have the responsibility for doing only one thing, likewise a method within that class should only do one thing. Then you might be asking; does that mean classes with only one methods in them. No, it means that the class should have one subject at a time and each and every method should only do work related to that subject and every method should just have the responsibility of solving one problem. Still, you might be wondering how to identify this.

Lets start with a simple example.

Say you have a system were you have Employees and each and every one of these have a WorkPosition related to it enabling a person to have different positions with the same employer (Yes, it is a real business case. 🙂 ).

The Id property is a classic pattern, but in domain driven design you would probably not use an integer as Id but rather a natural key that describes the aggregate better. For an Employee this could be the persons social security number. This means that an integer won’t do, but something that can tackle the needs of a social security number. For simplicity in this post, I’ll stick to primitives and do a string, normally I would introduce a domain concept for this; a type representing the concept instead of using generic primitives – more on that in another post.

We want to introduce this, but we’d love to keep the Entity base class, so we can stick common things into it, things like auditing maybe. But now we are changing the type of what identifies an Employee, and it’s not the same as for a WorkPosition; C# generics to the rescue:

Great, now we have a generic approach to it and get auditing all in one go.

We’ve just created a nightmare waiting to happen. We’ve made something generic, lost a lot from the domain already just to save typing in one property; Id (yeah I know, there are some auditing there – I’ll get back to that soon). We’ve lost completely what the intent of the Employees identification really is, which was a social security number. At least the name of the property should reflect that; Id doesn’t say anything about what kind of identification it is. A better solution would be going back to the original code and just make it a little bit more explicit:

That made it a lot more readable, we can see what is expected, and in fact we’ve also decoupled Employee and WorkPosition in one go – they weren’t coupled directly before, but the property named EmployeeId made a logical coupling between the two – which we might not need.

Another aspect of this would be bounded contexts; different representations of domain entities depending on the context they are in. In many cases an entity would even have different things that identifies it, depending on the context you’re in. Then Id would be a really bad name of a property and also having a generic representation of it would just make the whole thing so hard to read and understand. Normally what you would have is an underlying identifier that is shared amongst them, but you wouldn’t necessarily expose it in the different bounded contexts. Instead you would introduce a context map that would map from the concepts in the different bounded context to the underlying one.

Back to auditing – don’t we want to have that? Sure. But let’s think about that for a second. Auditing sounds like something you’d love to have for anything in a system, or in a particular bounded context, one could argue its cross cutting. It is a concern of every entity, but I would argue it is probably not something you need to show all over the place; in fact I’ll put forward the statement that auditing probably is an edge case when showing the entities on any dialog. So that means we probably don’t need them on the entities themselves, but rather make sure that we just get that information updated correctly in the database; this could be something we could do directly in the database as triggers or similar, or make sure everything goes through a well-defined common data context that can append this information. Then, for the edge cases were you need the auditing information, model only that and an auditing service of some kind that can get that information for the entities you need.

Fess up

Ok. So I have sinned, I’ve broken the Single Responsibility Principle myself many times, and I will guarantee you that I will break it in the future as well. In fact, let me show you code I wrote that I came across in Bifrost a few weeks back that got the hairs on my back rising, a system called EventStore:

I’ve stripped away some parts that aren’t vital for this post, but the original class some 60 lines. But looking at the little code above tells me a couple of things:

Its doing more than one thing without having a name reflecting it should do that

The EventStore sounds like something that holds events, similar to a repository – but it deals with other things as well, so…

The API is wrong; it takes something that is uncommitted and it saves it – normally you’d expect a system to take something uncommitted and commit it

The solution to this particular problem was very simple. EventStore needed to do just what it promises to do in its name, all the other stuff is coordination and by the looks of it, it is coordinating streams of uncommitted events. So I introduced just that; UncommittedEventStreamCoordinator with a method of Commit() on it. Its job is then to coordinate the stuff we see above and the EventStore can then take on the real responsibility of dealing with storing things, and in fact I realised that the EventRepository could go at this point because I had tried to solve it all in a generic manner and realised that specialised EventStores for the different databases / storage types we support would be a lot better and not a lot of work to actually do.

Another thing the refactoring did for us was the ability to now turn of saving of events, but still get things published. By binding the IEventStore that we have to an implementation called NullEventStore – we don’t have to change any code, but it won’t save. Also what we also can do is to introduce the ability the EventStore itself to by asynchronous, we can then create something like AsyncEventStore that just chains back to the original EventStore, but does so asynchronously. All in all it adds more flexibility and readability.

Behaviors to the rescue

All good you might think, but how do you really figure out when to separate things out. I’ll be the first to admit, it can be hard sometimes, and also one of them things one can’t just be asked and necessarily be able to identify it within a heartbeat – sometimes it is a process getting to the result. But I would argue that thinking in behaviors makes it simpler, just for the simple fact that you can’t do two behaviors at the same time. Thinking from a testing perspective and going for BDD style testing with a gherkin (given, when, then) also gives this away more clearly; the second you write a specification that has and in it, you’re doing two things.

Why care?

Took a while getting to the why part. It is important to have the right motivation for wanting to do this. Single Responsibility Principle and Seperation of Concerns are principles that have saved me time and time again. It has helped me decouple my code and get my applications cleaner. Responsibility separated out gives you a great opportunity to get to Lego pieces that you can stitch together and really get to a better DRY model. It also makes testing easier, testing interaction between systems and more focused and simpler tests. Also, separating out the responsibility produces in my opinion simpler code in the systems as well, simpler means easier to read. Sure, it leaves a trail of more files / types, but applying proper naming and good structure, it should be a problem – rather a bonus. It is very different from procedural code, you can’t read a system start to finish – but I would argue you probably don’t want to that, it is in my opinion practically impossible to keep an entire system these days in your head. Instead one should practice focusing on smaller bits, make them specialised; great at doing one thing. Its so much better being great at one thing than do a half good job at many things.

Since I started working professionally back in 1994 with software development I got very interested in processes and methodologies. Over the years I’ve worked with quite a few types; Waterfall, Cowboy, eXtreme Programming and lately we’ve been working with applying MSF Agile. Back in 1996 I started doing test-driven development, on games as the matter of fact. We had unit tests for basically everything, some of them needed to be interactive seeing that we couldn’t exactly mock visual stuff that needed the human eye to approve, but nevertheless we were TDD based.

I’ve been working with agile software development for the last 4-5 years, or should I say, trying to work with agile software development. All processes has a certain set of rules that needs to be followed and I remember when I had training in XP they said that you need to obey the 12 pillars of XP. The developers at work, myself included, couldn’t really figure out why that was so important and decided to apply the pillars we found interesting. Mistake number 1.

We applied XP for about a year and a half and found it to be lacking certain elements to it. We felt the process was missing key quality assurance elements and was very exclusive to developers. Mistake number 2 we made was that we hadn’t really applied eXtreme Programming, so who were we to judge. Anyhow, we tagged along and discovered MSF Agile and found it very interesting straight away. It had the roles we felt were missing and process elements that would fill in the missing gaps of what we felt were missing for quality assurance. We jumped aboard the train and never looked back (until today).

Mistake number 3; I found myself believing to my core that we had applied MSF Agile, at the time of introduction I was manager of application development and was working a bit more with processes and planning. I sincerely believed we had applied MSF Agile with all it’s artifacts. Last spring I decided to leave the company for a job that looked very interesting (My personal mistake number X), only to find out that the company I started at was not interested in applying any form of methodology unless it was named adhoc. I decided to leave them after only 2 months and was quickly approached by the company I quit 2 months earlier and got an offer that was acceptable. I went back, this time being only a senior software developer and team manager for a smaller team (4 people, including myself). I decided at that point that I’d had it with all the management stuff, I wanted to primarily write code, but it wouldn’t hurt to have some control over a specific part of the application we are working on. Anywho.. After a couple of months I started to see something very strange. We weren’t working according to any process, or at least any process I knew. I got really pissed off and started ranting openly about why aren’t we working according to MSF Agile (My personal mistake number X+1). The truth was that when I was too busy planning our next big project and going around believing that everyone was working according to the process, people didn’t know the first thing about the process. No one had read the process guidance on our sharepoint site even. I kept ranting and was honestly really fed up that there hadn’t been anyone who took charge and applied the process we had agreed upon.

And now back to the title of this post. Why do I think agile is fragile, in fact I think most processes are fragile in the sense that if you don’t apply the process correctly, the process will collapse under it’s own weight. Agile is fragile is just a cool title, I honestly think the statement applies to all processes. So, why are they so fragile. Take for instance “embrace change” that agile methodologies has in it’s core. The concept of having code that can be changed, refactored, feature change at any given point in time demands that you apply TDD. Without unit tests you would really need to be a cowboy in order to dare to start a major refactoring job. The unit tests serves as a safety net that assures you that the code you are rewriting will still result in a working piece of software. Another example is pair programming. In order to achieve full global ownership to the code, pair programming must be applied. The beauty of pair programming is that you spread the knowledge about the code throughout your team like a wildfire. If you drop this and let developers specialize, global ownership is lost. Pair programming is probably the pillar that is hardest to apply in a team. Many developers will argue not to be part of it at all. Pair programming is not an aspect of MSF Agile. That’s probably why global ownership is not a part of MSF Agile. Yet another example is the fact that the customer should be available all the time. By customer one means an actual customer or a representative for the customer that is located within the team or is available within the organization. Without having the close relationship to the customer, you will not be able to see if the project is on the right track between iterations. An agile project needs to correct itself all the time, this is done through the use of iterations. The length of the iteration can be 1, 2, 3 or 4 weeks. It is not recommended in most cases to have them go for more than 4 weeks. If you don’t have the customer available at these junctions, you will not be aware of any mistakes you might have done. That means you’re back to waterfall were you have a test period at the very end of the project. A very important aspect of all agile methodologies is that all roles work actively for every iteration. That means that you’re planning, writing unit tests, writing code, acceptance testing and so on for every iteration. Loose one of the activities and it all collapses. I could go on and on about other fragile relationships in agile methodologies, my point is that you must be really careful about applying a whole process no matter what process you decide to apply.

The beauty of having time of work (I’m on my paternity-leave or sick-leave, not sure which one yet) and really not think about work, totally disconnected from work (I’m not even reading emails..), is that you get a lot of time to think about stuff like this. I’ve come up with a couple of rules that I believe is important when trying to apply a software development process :

1. Whenever you want to apply a process, one would think that having management with you should be sufficient. Wrong. You need the entire team. Every single member of the team must embrace the new process. They don’t have to like it at first, but they have to agree to be willing to try it out and work according to it.

2. You need to be really strict when applying the process. You can’t afford to not apply the whole process. Most processes will collapse if you do so.

3. Stretch to fit – not shrink to fit. You can adjust the process so it fits to your needs by adding to it, never removing!

4. You need at least 2 persons in the organization that are not only masters of the process, but also really enthusiastic about it. They will act as motivators for the others.

5. Train everyone! If everyone is trained in the process, and if possible, certified in it, you’ll automatically have a process-cop in every single member of the team. Together the team will apply the process and correct any mistakes that is bound to happen.

I’ll probably come up with a couple more rules as the days fly by sitting here doing nothing. Feel free to drop me a comment if you have input to the madness above.. 🙂