Your code is trying to tell you something.

I havewrittenelsewhere that people, not rules, do things. I have written this in exasperation over some people claiming that TDD has ruined their lives in all manner of ways. Enough!

People, not rules, design software systems. People decide which rules to follow and when. The (human) system certainly influences them, but ultimately, the people decide. In particular, people, not TDD, decide how to design software systems. James Shore has recently written “How Does TDD Affect Design?” to offer his opinion, in which he leads with this.

TDD does not by itself create good or bad designs, but I have evidence (see “Complexity and Test-First 0”) suggesting that it does create different designs.

Keith’s comment triggered me to think about how practising TDD has affected the way I design software systems, of which this article represents a summary. I might add more to this list over time. If you’ve noticed an interesting pattern in your designs that you attribute to your practice of TDD, then please share that in the comments.

More value objects, meaning objects with value-equality over identity-equality. I do this more because I want to use assertEquals() a lot in my tests. This also leads to smaller functions that return a value object. This also leads specifically to more functions that return a value object signifying the result of the function, where I might not have cared about the result before. Sometimes this leads to unnecessary code, and when it does, I usually find that I improve the design by introducing a missing abstraction, such as an event.

More fire-and-forget events. I do this more because I want to keep irrelevant details out of my tests. Suppose that function X should cause side-effect Y. If I check for side-effect Y, then I have to know the details of how to product side-effect Y, which usually leads to excessive, duplicate setup code in both X’s tests and Y’s tests. Not only that, but when X’s tests fail, I have to investigate to learn whether I have a problem in X or Y or both. Whether I approach this mechanically (remove duplication in the tests) or intuitively (remove irrelevant details from the tests), I end up introducing event Z and recasting my expectations of X to “X should fire event Z”. This kind of thing gives many programmers the impression of “testing the implementation”, whereas I interpret this as “defining the essential interaction between X and the rest of the system”. The decision to make function X fire event Z respects the Open/Closed Principle: inevitably I want X to cause new side-effects A, B, and C. By designing function X as a source for event Z, I can add side-effects A, B, and C as listeners for event Z without changing anything about function X. This leads me to see the recent (as of 2014) trend towards Event Sourcing as a TDD-friendly trend.

More interfaces in languages that have interface types. In the old days, we had to introduce interfaces (in Java/C#) in order to use the cool, new dynamic/proxy-based mocking libraries, like EasyMock and JMock. Since the advent of bytecode generators like cglib, we no longer need to do this, but my habit persists of introducing interfaces liberally. Many programmers complain about having only one implementation per interface, although I still haven’t understood what makes that a problem. If the language forces me to declare an interface type in order to derive the full benefits of abstraction, then I do it. At least it encourages me to organise and document essential interactions between modules in a way that looser languages like Ruby/Python/PHP don’t. (Yes, we can implement interfaces in the duck-typing languages, but Java and C# force us to make them a separate type if we want to use them.) Moreover, the test doubles themselves act as additional implementations of the interfaces, which most detractors fail to notice. They might argue that I overuse interfaces, but I argue that they underuse them. Interfaces provide an essential service: they constrain and clarify the client’s interaction with the rest of the system. Most software flaws that I encounter amount to muddled interactions—usually misunderstood contracts—between modules. I like the way that the interfaces remind me to define and refine the contracts between modules.

Immutability. As functional programming languages have become more popular, I’ve noticed more talk about mutability of state, with an obvious leaning towards immutability. In particular, not only do I find myself wanting functions more often to return value objects, but specifically immutable value objects. Moreover, thinking about tests encourages me to consider the pathological consequences of mutability. This happened recently when I wrote “The Curious Case of Tautological TDD”. Someone responded to the code I’d written pointing out a problem in the case of a mutable Cars class. I had so long ago decided to treat all value objects as immutable that I’d even forgot that the language doesn’t naturally enforce that immutability. I’ve valued immutability for so long that, for me, it goes without saying. I reached this point after writing too many tests that only failed when devious programmers take advantage of unintended mutability, such as when a function returns a Java Collection object. I went through a phase of ensuring that I always returned an unmodifiable view of any Collection, but after a while, I simply decided to treat every return value as immutable, for the sake of my sanity. Functional languages push the programmer towards more enforced immutability, and even the eradication of state altogether. I feel like my experience practising TDD in languages like Java and Ruby have prepared me for this shift, so it already feels quite natural to me; on the contrary, it annoys me when I have to work in a language that doesn’t enforce immutability for me.

How has TDD affected the way you design? or, perhaps more importantly, what about the way TDD might affect your designs makes you uneasy about trying it? I might have some useful advice for you.

I’m publishing this as a “rough cut”, so I apologise to everyone annoyed by having to download a PDF to read this article. I have my reasons. Some of the links in the document don’t work; the links to external web sites, however, should. A handful of people have already suggested improvements and reported problems, which I appreciate, particularly the cordial and civil manner in which they’ve done it. (Hint.)

(In response to complaints, I have removed the embedded PDF viewer. It was an experiment. Thank you for your consideration.)

Recently Bob Marshall opined that refactoring code is waste. This reminds me of passionate discussions from a decade ago about testing: should we classify testing as a value-added activity or as an unavoidable waste? I’d like to change the question a little, but first, allow me to play the ball where it lies.

If you haven’t read Bob’s article yet, then do so now. You’ll find it quite short; I read it in a few minutes. I composed this as a comment to Bob’s article, but it expanded to the point where I chose to promote it to a short article. You might say that I refactored my writing. With that segue manufactured…

Is editing waste for a writer? Why don’t writers simply write the right words/articles/books/sentences the first time? So I think it goes for programmers. I think of refactoring as editing for programmers. Since I plan to refactor, I don’t have to program like Mozart and “get it right” in my head before writing it down. This helps me, because often I don’t see trouble with code until I’ve written it down, even though sometimes drawing its structure helps me enough to spot trouble.

Sometimes problems don’t emerge until long after I’ve written it down and the situation changes, putting pressure on an old choice or negating an old assumption. Absolutely/permanently right-first-time seems to require clairvoyance. Writing any code entails risk.

Even so, I agree that we programmers don’t need to deny our own experience just to fit some arbitrary goal of taking tiny steps and refactoring towards abstractions. (This has got me in trouble with some people who declare what I do “not TDD”. As they wish.) Sometimes I can see the abstractions, so I go there sooner. Sometimes that doesn’t work out, so I refactor towards different abstractions. Often it works out and I’ve skipped a handful of tedious intermediary steps. One could measure my “expertise” in design by measuring the additional profit I can squeeze out of these trade-offs compared to others. (No, I don’t know how to measure that directly.) I think we broadly call that “judgment”.

A Question of Intent

I find refactoring wasteful when I do it out of habit, rather than with a purpose. Nevertheless, I don’t know how to have developed the judgment to know the difference without making a habit of refactoring. (Of course, I like to think that I do everything always with a purpose.) I encourage novices (in the Dreyfus Model sense) to force code into existence primarily through refactoring with the purpose of developing that judgment and calling into question their assumptions about design. That reasoning sounds circular, but I have written and said elsewhere how refactoring helps programmers smooth out the cost of maintaining a system over time. I can only assert that I produce more maintainable software this way, compared to what I used to do, and that refactoring plays a role. I really wish I knew how much of that improvement to attribute to refactoring. Refactoring still saves my ass from time to time, so it must pull some of its own weight.

I would classify refactoring as waste in the same way that I’d classify verification-style testing as waste: since we don’t work perfectly, we need feedback on the fitness of our work. Not only that, but I refactor to support not having to future-proof my designs, because of the waste of building structures now that we don’t intend to exploit until later. Which waste costs more? I find that open question quite interesting.

References

Bob Marshall, “Code Refactoring”. In his article, Bob surmises that programmers can’t quite “get it right” in their heads, and highlights refactoring as potentially a self-fulfilling waste: if we assume that we have to live with it, then we will choose to live with it. I leave the parallel with #NoEstimates as an exercise for the reader.

Gemma Cameron, “Is Refactoring Waste?”. I noticed Gemma’s article on Twitter and it led me to read Bob’s. She mentions that she plans to experiment with a TDD microtechnique that I use often: noticing while ‘on red’ that a little refactoring would make it easier to pass the test, and so ignoring (or deleting) the test to get back to ‘green’ in order to refactor safely. I don’t always do this, but I consider it part of the discipline of TDD and teach it in my training courses.

“The Dreyfus Model of Skill Acquisition”. Wikipedia’s introduction to the topic. All models are wrong; some models are useful. I find this one helpful in explaining to people the various microtechniques that I teach, when I follow them and when I don’t.

If you like to teach test-first/test-driven techniques, then you have probably stumbled over configuration problems in your programming environment. This happens to every presenter at least once. We have a variety of ways to handle the problem, and today I’d like to share another one with you.

Google Spreadsheet can run Javascript, which means that we now have a ready-to-go approximation of Fit. It took me only a few minutes to figure out how to write a Javascript function that operates on the value in a spreadsheet cell. After that, I recorded ten minutes of video demonstrating the environment.

Yes, I wrote the code test-first, but not test-driven. Please hold your cards and letters. I wanted to demonstrate a testing environment and not TDD. Even so, with ATDD or BDD, we programmers often receive a batch of customer tests and make them pass one by one, and sometimes we don’t need additional programmer tests to have confidence that we’ve built things well. Looking at the final design for this solution, I don’t think that a strict test-driven approach would have improved anything. If you disagree, then please share your approach with us!

So maybe not an epic, age-old battle, but a battle nonetheless. The battle over the correct ordering of the Four Elements of Simple Design. A battle that I’ve always known I’d won, but until recently, could never justify to others. I can finally declare victory and bestow upon you the One True Sequence.

Calm down; I’m joking.

Seriously: it has always bothered me (a little) that I say it this way and Corey Haines says it that way, and somehow we both have it “right”. I never quite understood how that could happen… until now.

I have improved the code samples to make them closer to valid, working Python.—jbrains

At some point, you know I had to write this article. Let me state something clearly from the beginning.

In spite of the title, I use mocks freely and happily.

I do not intend with this article to join those who either shun mocks or appear to shun mocks only to praise them. I plan to share a simple example of using mocks as a stepping stone towards a potentially more suitable design, but you must not interpret this as a message against mock objects.

There. I feel better now. I trust you to carry my message the way I intended it. Do not betray that trust.

I asked my Twitter community which HTTP client to use. Certainly not HTTParty, because cool people don’t use HTTParty anymore. One takes these risks when one falls out of the loop with the latest and greatest tools.

Everything went fine until I needed to follow a redirect. What happened from that point forward could form the basis of a solid one-act play. I’ll spare you the horror and send you directly to the punch line.

At Øredev 2013, I asked managers to tell me what makes their job difficult. I spoke to one manager who told me an all-too-common story about facing immense pressure from the board of directors to deliver more features more often and similar pressure from the programmers to give them more time to refactor. More important than the details of this situation, I’ve heard dozens of stories like these over the years, and while one single piece of advice can’t possibly address every significant aspect of these situations, I have found that most of the people who tell me these stories identify with what I’d like to share with you.

Warning. You almost certainly have more to deal with in your exact situation than this advice can cover. You might need some more direct help. I can offer that to you, but for now, read on.

Legacy code imposes an unbounded, inexact tax on the cost of all future features. This explains why I run events like Legacy Code Retreat and why I write about improving the design of software systems. This further explains why I’m willing to accept the risk of people labeling me as an “over-engineerer”. What they label “over-engineering”, I consider managing risk. As part of managing risk, I like to test-drive changes to legacy code, and implicit dependencies almost always get in the way of starting. I’m going through this right now, working with Octopress, the blogging software that I use here.

Let me state this clearly: I really like Octopress. If you intend to interpret what follows as me attacking Octopress, then stop reading and go away. Thank you.

I love having tricky conversations on Twitter. I know a lot of you don’t, because you can’t express yourself clearly in 140 characters or less, but I love it for precisely that reason. The constraint makes it easy to voice objections, which makes assumptions visible and encourages us to challenge them. You’d be amazed at the things that people object to – issues that you thought were settled a long time ago, which you take for granted, at least within certain of your social circles.

This Twitter conversation reminded me of one of my own, long-standing assumptions… the one upon which I based the name of this website.

@ecomba@jbrains The more I hear this notion expressed in summary, the harder it is for me to keep liking it.

You owe it to yourself to read the entire conversation, so click on the Tweet’s timestamp and read further. Allow me to summarise: Letting your code tell you where it wants to go does not mean conducting a seance, speaking to the dead.

This is not in my design toolbox

In case that conversation link has long-since died, let me state very clearly that when I write code, it might look like I use a Ouija board, except that I know that I’m controlling it. I’m looking for signals in my code to indicate design problems, then I decide which problems to address right now, then decide what to do about them. Some signals hit me more strongly than others. Over time I have developed an awareness of and a sensitivity to an ever-growing number of different signals that indicate potential design problems.

I just find it more fun to say that my code tells me where it wants to go. Please don’t interpret this too literally.