I'm trying to wrap my head around TDD, specifically the development part. I've looked at some books, but the ones I found mainly tackle the testing part - the History of NUnit, why testing is good, Red/Green/Refactor and how to create a String Calculator.

Good stuff, but that's "just" Unit Testing, not TDD. Specifically, I don't understand how TDD helps me get a good design if I need a Design to start testing it.

To illustrate, imagine these 3 requirements:

A catalog needs to have a list of products

The catalog should remember which products a user viewed

Users should be able to search for a product

At this points, many books pull a magic rabbit out of a hat and just dive into "Testing the ProductService", but they don't explain how they came to the conclusion that there is a ProductService in the first place. That is the "Development" part in TDD that I'm trying to understand.

There needs to be an existing design, but stuff outside of entity-services (that is: There is a Product, so there should be a ProductService) is nowhere to be found (e.g., the second requirement requires me to have some concept of a User, but where would I put the functionality to remind? And is Search a feature of the ProductService or a separate SearchService? How would I know which I should choose?)

According to SOLID, I would need a UserService, but if I design a system without TDD, I might end up with a whole bunch of Single-Method Services. Isn't TDD intended to make me discover my design in the first place?

I'm a .net developer, but Java resources would also work. I feel that there doesn't seem to be a real sample application or book that deals with a real line of business application. Can someone provide a clear example that illustrates the process of creating a design using TDD?

TDD is just a part of whole development methodology. Of course you will need to employ some kind of design (either up-front, or better evolutionary) to get the whole thing together.
–
EuphoricMay 29 '13 at 17:49

3

@gnat: It's an inquiry into why the TDD books do not make the design process clearer.
–
Robert HarveyMay 29 '13 at 18:17

4

@gnat: It was your edit, not mine. :) See my change to the title of the question and the body.
–
Robert HarveyMay 29 '13 at 19:00

7

If you have read Robert C. Martin's work or maybe watched one of his videos, you'll see that he often has a design in mind but he isn't married to it. He believes that his pre-conceived notion of the right design will emerge from his tests, but he doesn't force it to. And in the end, sometimes that design does, and sometimes it does not. My point here is that your own prior experience will guide you, but the tests should drive you. Tests should be able to develop or debunk your design.
–
Anthony PegramMay 29 '13 at 19:34

3

So it's not really about testing, it's about design. Only it's not really helping you with design, so much as helping you validate design. But isn't that !@#$ing testing?
–
Erik ReppenMay 29 '13 at 23:17

11 Answers
11

The idea of TDD is to start with testing and work from that. Thus, to take your example of "A catalog needs to have a list of products" could be seen as having a test of "Check for products in catalog" and thus this is the first test. Now, what holds a catalog? What holds a product? Those are the next pieces and the idea is to get some bits and pieces put together that would be something like a ProductService that will be born from getting that first test to pass.

The idea of TDD is to start with a test and then write the code that makes that test pass as the first point. Unit tests are part of this yes, but you aren't looking at the overall picture that is formed by starting with tests and then writing the code so that there aren't blind spots at this point since there isn't any code yet.

Test Driven Development Tutorial where slides 20-22 are the key ones. The idea is to know what the functionality should do as a result, write a test for it and then build a solution. The design part will vary as depending on what is required it may or may not be that simple to do. A key point is to use TDD from the start rather than try to introduce late into a project. If you start with tests first this can help and is likely worth noting in a sense. If you try to add the tests later, it becomes something that may be put off or delayed. The later slides may also be useful as well.

A main benefit of TDD is that by starting with the tests, you aren't locked into a design initially. Thus, the idea is to build the tests and create the code that will pass those tests as a development methodology. A Big Design Up Front can cause problems as this gives the idea of locking things into place which makes the system being built to be less nimble in the end.

Robert Harvey added this in the comments which is worth stating in the answer:

Unfortunately I think that this is a common misconception about TDD:
you can't grow a software architecture by just writing unit tests and making them pass. Writing unit tests does influence the design, but
it doesn't create the design. You have to do that.

@MichaelStum: Unfortunately I think that this is a common misconception about TDD: you can't grow a software architecture by just writing unit tests and making them pass. Writing unit tests does influence the design, but it doesn't create the design. You have to do that.
–
Robert HarveyMay 29 '13 at 18:12

4

@RobertHarvey, JimmyHoffa: if I could vote up your comments a 100 times, I would!
–
Doc BrownMay 29 '13 at 19:29

9

@Robert Harvey: I am glad you wrote about this common misconception: I hear way too often that one just needs to sit down and write all kinds of unit tests and a design will just "emerge" spontaneously. And if your design is bad, it is because you did not write enough unit tests. I agree with you that tests are a tool to specify and verify requirements for your design, but "you have to do " the design yourself. I totally agree.
–
GiorgioMay 29 '13 at 20:33

@Andres F: I know the story about sudoku, and I think it is very interesting. I think some developers make the mistake of thinking that a tool (e.g. TDD or SCRUM) can replace domain knowledge and their own efforts, and expect that by mechanically applying a particular method good software will magically "emerge". They are often people who do not like spending too much time in analysis and design and prefer to code something up directly. For them, following a particular methodology is an alibi for not doing proper design. But this is IMHO a misuse of TDD.
–
GiorgioJun 4 '13 at 10:34

For what it's worth, TDD helps me come to the best design more quickly than not doing TDD. I would probably come to the best design with or without it. But that time that I would have spent thinking it through and taking a few stabs at the code is spent writing tests instead. And it's less time. For me. Not for everyone. And, even if it took the same amount of time, it would leave me with a suite of tests, so that refactoring would be safer, leading to even better code down the line.

How does it do it?

First, it encourages me to think about every class as a service to some client code. Better code comes from thinking about how the calling code wants to use the API rather than worrying about how the code itself should look.

Second, it stops me writing far too much cyclometic complexity into one method, while I'm thinking it out. Each extra path through a method will tend to double the number of tests I need to do. Sheer laziness dictates that after I've added too much logic, and I have to write 16 tests to add one condition, it's time to pull some of it out into another method / class and test it separately.

I'm trying to wrap my head around TDD...
To illustrate, imagine these 3 requirements:

A catalog needs to have a list of products

The catalog should remember which products a user viewed

These requirements should be restated in human terms. Who wants to know which products the user previously viewed? The user? A salesperson?

Users should be able to search for a product

How? By name? By brand? The first step in test-driven development is to define a test, for example:

browse to http://ourcompany.com
enter "cookie" in the product search box
page should show "chocolate-chip cookies" and "oatmeal cookies"

>

At this points, many books pull a magic rabbit out of a hat
and just dive into "Testing the ProductService", but they don't
explain how they came to the conclusion that there is a ProductService
in the first place.

If these are the only requirements, I certainly wouldn't leap to create a ProductService. I might create a very simple web page with a static product list. That would work perfectly until you get to the requirements to add and delete products. At that point I might decide it is simplest to use a relational database and an ORM, and create a Product class mapped to a single table. Still no ProductService. Classes like ProductService will be created when and if they are needed. There may be multiple web requests that need to perform the same queries or updates. Then the ProductService class will be created to prevent code duplication.

In summary, TDD drives the code to be written. Design happens as you make implementation choices, and then refactor the code into classes to eliminate duplication and control dependencies. As you add code, you will need to create new classes to keep the code SOLID. But you don't need to decide ahead of time that you will need a Product class and a ProductService class. You may find that life is perfectly fine with just a Product class.

OK, no ProductService then. But how did TDD tell you that you needed a database and an ORM?
–
Robert HarveyMay 29 '13 at 21:33

4

@Robert: it didn't. It's a design decision, based on my judgment of the most effective way to meet the requirement. But the decision could change.
–
kevin clineMay 29 '13 at 21:41

1

Good design will never be produced as a side-effect of some arbitrary process. Having a system or model to work with and frame things on is great but test-first-TDD, IMO, hits a conflict of interest by also selling itself as something that will guarantee people won't be bitten unexpectedly by side-effects of bad code that shouldn't have happened in the first place. Design requires reflection, awareness, and forethought. You don't learn those from pruning the auto-discovered symptoms off the tree. You learn them by figuring out how to avoid evil mutant branches in the first place.
–
Erik ReppenJun 6 '13 at 5:31

I think the test 'add a product; reboot the computer and restart the system; the added product should still be visible.' shows where the need for some sort of database comes from (but that could still be a flat file or XML).
–
yatima2975Jun 6 '13 at 13:25

Others may disagree, but to me many of the newer methodologies rely on the assumption that the developer is going to do most of what the older methodologies spelled out just out of habit or personal pride, that the developer is usually doing something that is fairly obvious to them, and the work is encapsulated in a clean language or the cleaner parts of a somewhat messy language so you can do all the test business.

Some examples where I have run into this in the past:

Take a bunch of spec-work contractors and tell them their team is
Agile and Test First. They often have no habit other than to work to
spec and they have no concern over the quality of the work as long as
it lasts long enough to finish the project.

Try and do something new test first, spend much of your time ripping
tests as you find various approaches and interfaces are crap.

Code something low level and either get slapped for lack of coverage,
or write a lot of tests that do not amount to much value because you
cannot mock the underlying behaviors you are tied to.

Any situation where you don't have enough of the underlying mechanics in place ahead of time to add a testable feature without writing a bunch of underlying untestable bits first, like disk subsystem, or a tcpip level communication interface.

If you are doing TDD and it is working for your, good for you, but there are a lot of things (whole jobs, or stages of a project) out there where this just simply does not add value.

Your example sounds like you aren't even to a design yet, so either you need to have an architecture conversation, or you are prototyping. You need to get through some of that first in my opinion.

I am convinced that TDD is a very valuable approach to the detailed design of the system - i.e. the APIs and the object model. However, to get to the point in a project where you would begin to use TDD, you need to have the big picture of the design already modeled in some fashion and you need to have the big picture of the architecture already modeled in some fashion. @user414076 paraphrases Robert Martin as having a design idea in mind, but not being married to it. Exactly. Conclusion - TDD is not the only design activity going on, it is how the details of the design get fleshed out. TDD must be preceded by other design activities and fit into an overall approach (such as Agile) that addresses how the overall design gets created and evolved.

FYI - two books I recommend on the topic that give tangible and realistic examples:

TTD drives design discovery by test failure, not success, therefore you can test unknowns and iteratively retest as unknowns are exposed ultimately leading to a complete harness of unit tests - a very nice thing to have for ongoing maintenance and a very difficult thing to try to retrofit after code is written/released.

For example, a requirement may be that input can be in several different formats, not all are known yet. Using TDD you would first write a test that verifies the appropriate output is supplied given any input format. Obviously this test will fail, so you write code to handle the known formats and retest. As the unknown formats are exposed through requirements gathering, new tests are written before the code is written, these also should fail. Then new code is written to support the new formats and all tests are rerun reducing the chance for regression.

It is also helpful to think of unit failure as "unfinished" code instead of "broken" code. TDD allows for unfinished units (expected failures), but reduces the occurrence of broken units (unexpected failures).

I do agree that this is a valid workflow, but it doesn't really explain how high-level architecture can emerge from such a workflow.
–
Robert HarveyMay 29 '13 at 21:33

1

Right, a high-level architecture like the MVC pattern, isn't going to emerge from TDD alone. But, what can emerge from TDD is code designed to be easily testable, which is a design consideration in itself.
–
Daniel PereiraMay 29 '13 at 22:28

... many books pull a magic rabbit out of a hat and just dive into "Testing the ProductService", but they don't explain how they came to the conclusion that there is a ProductService in the first place.

They came to that conclusion by thinking about how they were going to test this product. "What sort of product does this?" "Well, we could create a service". "Ok, let's write a test for such a service"

A functionality can have many design and TDD will not completely tell you which one is the best. Even, if tests will help you build more modular code, it can also lead you to build modules that suits tests requirement and not production reality. So you have to understand where you are going and how things should fit in the whole picture. Put otherwise, there are Functional and Non-Functional requirements, don't forget the last one.

Concerning design I refer to Robert C. Martin books (Agile Development) but also Martin Fowler's Patterns of Enterprise Application Architecture and Domain Driver Design. The later especially is very systematical in extracting the Entities and Relations out of the requirements.

Then, when you get a good feelings of the options available to you on how to manage those entities, you can feed you TDD approach.

TDD helps a lot, however there is important part in software development. Developer should listen to the code that is being written. Refactoring is 3rd part in TDD cycle. This is the main step where developer should focus and think before go to next red test. Is there any duplication? Are SOLID principles applied? What about high cohesion and low coupling? What about names? Take a closer look at code that is emerging from the tests and see if there is something that needs to be changed, redesigned. Question the code and code will tell you, how it want's to be designed.
I usually write sets of multiple tests, examine that list and create first simple design, it doesn't need to be "final", usually it's not, because it's changed when adding new tests. That's where the design comes.