Thoughts about software development

Agile people still don’t get it

I just attended Test-Driven Development presentation which represents everything that is wrong about the way Agile advocates are trying to evangelize their practices. I don’t have anything against the presenter in particular, but it’s really time for Agilists to rethink the way they communicate with the real world.

Here are a few comments on his presentation.

One of the first slides that deeply troubled me claimed the following:

Tests are (executable) specs.

If it’s not testable, it’s useless.

First of all, tests are not specs. Not even close. Somebody in the audience was quick to give a counter-example to this absurd claim by using a numeric example ("how do you specify an exponentiation function with a test?") but my objection to this claim is much broader than that. Relying on tests as a design specification is lazy and unprofessional because you are only testing a very small portion of the solution space of your application (and of course, your tests can have bugs). Tests also fall extremely short of having the expressiveness needed to articulate the subtle shades that a real specification need to cover to be effective.

This claim is part of a broader and more disturbing general Agilist attitude that is usually articulated like "Your code is your spec", along with some of its ridiculous corollaries such as "Documentation gets out of date, code never does".

Anyone who claims this has never worked on a real-world project. And I’m setting the bar fairly low for such a project: more than five developers and more than 50,000 lines of code. Try to bring on board new developers on this project and see how fast they come up to speed if all they have to understand the code base is… well, just code. And tests.

I am currently getting acquainted with a brand new project that is not even very big, and while I understand Java fairly well, there is no doubt in my mind that for ten minutes I spend trying to understand how a certain part of the application works, a five-line comment would have given me this knowledge in ten seconds.

The second claim, "If it’s not testable, it’s useless" is equally ludicrous and a guarantee that at this point, the audience you are talking to is already looking at you as a crackpot.

Software is shipped with untested parts every day, and just because it’s not entirely tested doesn’t mean it’s bad software or that the untested parts are "useless".

Agilists just don’t understand the meaning of calculated risk.

Early in the development cycle, it’s perfectly acceptable to go for a policy of "zero bugs" and "100% tests". But as the deadline looms, these choices need to be reconsidered all the time and evaluated while keeping a close eye of the final goal. Very often, Agilists simply forget that their job is to produce software that satisfies customers, not software that meets some golden software engineering scale.

Anyway, let’s go back to the presentation, which then proceeded with the implementation of a Stack class with TDD. Before spending thirty minutes on a live demo of the implementation of a Stack class (are you impressed yet?), the presenter warned the increasingly impatient audience that they should "not pay too much attention to the Stack example itself but to the technique".

And that’s exactly the wrong thing to do.

Look, we "get" TDD. We understand it. Frankly, it takes all of ten minutes to explain Test-Driven Development to a developer who’s never heard of it: "Write a test that fails and doesn’t compile. Make it compile. Then make it pass. Repeat".

The hard part is applying it to the real world, and showing the implementation of a Stack will soon have everyone leave the room with the thought "Cute, but useless. Now let’s go back to work".

It was even worse than that, actually: The presenter kept taking suggestions from the crowd but he declined all those that didn’t fit in the neat script that he had in hands at all times. These suggestions were good, by the way:

"What should we test now?"

"How about: if we pop an empty stack, we get an exception"

<a bit embarrassed> "Mmh, no, let’s not do that" <hand waving, look down at notes and proceeds while happily ignoring the other raised hands>

To be honest, I am becoming quite suspicious of Agile practices for that reason: all the presentations I have attended and books that I have read are always using toy implementations as examples. Stack, List, Money, Bowling… enough already! Let’s talk about TDD for code that interacts with clustered databases on laggy connections built on 500,000 lines of code that was never designed to be tested in the first place (and: yes, I read Michael Feathers’ book, it has some good and some bad, but it’s not germane to Java and TDD so I won’t expand on it here).

And please, avoid smug and useless answers such as:

"A lot of the classes I have to test are hard to isolate, do you have any advice regarding mocks?"

"Well, if you had started with TDD in the first place, you wouldn’t be having this problem today".

Fundamentally, I am disturbed by the Agilists’ dishonesty when it comes to presenting their arguments. They offer you all these nice ideas such as Test-Driven Development and Pair Programming but they never — ever — disclose the risks and the downsides. To them, Agility is a silver bullet that is applicable in all cases with no compromises.

The truth is that these practices come at a price, and for a lot of organizations, the price gets high very quickly. Agile development will never go far if its proponents keep ignoring these organizations and make condescending comments to its members.

I like Test-Driven Development. I really do, and I’m fortunate enough to work on a project that lets me use TDD most of the time. But the truth is: at times, I don’t do TDD because implementing a feature quickly is more important than a fuzzy feeling. And I’m also aware that TestNG is an open source project with less than five developers, all of them on the bleeding edge and aware of the latest advances in software engineering.

And this is my main beef with Agilists: I strongly suspect that most of them are spending their time on open source projects with like-minded fellows, but none of them have any experience what companies whose survival depends on shipping software have to go through to organize huge code bases growing thousands of lines of code every day under the combined push of hundreds of a developers, all with their personal background, education and bias.

So here is my advice to Agilists: get real now, or you will become irrelevant soon.

176 Comments

I have seen some companies use Agile and do it well, but it is not the “end all, be all” of development. It also took a lot of pain to get there and now they are locked into the Agile method in it’s entirety. If it ceases to work for them they will have to go through a lot of pain to get out of doing the entire Agile model.

In my current job, we employ some Agile practices, but only the ones that make sense for our company. The more I have studied and looked at all the different methods out there the more I’ve become convinced that custom made processes, tailored to the companies culture, make the most sense. I try to stay away from becoming dogmatic about any one idea, but I’m buying more and more into the Context-Driven typology all the time.

Agreed.. there is a common mistake that is made by engineers (and managers) on both side of the Agile argument.. they mistake tools and process for discipline when THEY ARE NOT. Common sense and good discipline make things better, but are rarely practiced in any organization I’ve been in. You can apply this to all the disparate comments made for this blog entry. Nothing is a replacement for discipline, and with good discipline many processes can be made to work well.

I’ve used TDD for 15 years now, thinking in terms of unit testing for many years now and tend to have very few bugs occur. I managed at Oracle for quite a few years where we had very fast turn around releases that some folks might call sprints.. but where we were using what folks would call waterfall style documentation (design docs kept up to date, formal design reviews, etc.).

Folks take both sides religiously, but when it comes down to it, these common sense details will prevail:

o Customers like to change their minds after they see stuff, so plan for that.

o Thinking out designs ahead of time is better than just coding.. I’ve seen too many folks paint themselves into corners that could have been avoided by designing and analyzing up front.

o We don’t always have the monetary luxury to have two programmers working on the same piece of code

o Good (and maintained) design documentation protects against programmer turnover.. and every programmer who has been around knows that its one thing to read the code, but if it is truly convoluted puzzling out what it should have done based on previous requirements, absent of documentation, can be an exercise in extreme frustration. If the one guy that wrote it is gone, you can be truly in the hurt locker.

Again… the common idea behind all this is good discipline.. and achieving good discipline in an instant-gratification style world is the TRUE challenge. Agile vs. Waterfall is a theoretical argument, the real world challenges us with problems that don’t neatly fit into either.. which means look, adapt, and react in a disciplined manner. THAT is the path to real project success.

If I were interviewing you for a job, asked you for your thoughts on Agile, and you told me that you have never even heard of a an Agile project that was successful, I would thank you for your time and show you the door.

It shows that you don’t really have much professional experience, or you are so set in your ways that you will be dismissive of any counter-evidence.

There’s really no point in listing out the ever-growing number of companies who are transitioning to Agile and seeing good return on their decisions in the process. You will just blow it off and keep doing what you’re doing.

Sean Art: Is it really the candidate’s fault if they have never seen a successful Agile project? It feels like you are filtering out candidates on criteria that are completely out of their control, a bit like judging someone based on their skin color or where they were born.

“Agile” people still don’t get it!
I’m with you there:
The problem is, they Don’t get “Agile” either..
“Agile” seems to be a re-tread of the production manufacturing methods we used in component manufacturing in the 1980s at National Semiconductor. And that was a J.E.Deming re-vamp of USA WW2 production methodology that was re-learned from the Japanese who picked it up in the ’50s.
We supplied components to the mainframe manufacturing plants at IBM in France. They wanted zero production defects, and achieved that. There was no incoming inspection testing of our components or their circuit boards on their production line. Our components were supplied at a failure rate of 0 parts per million. (Obviously we production-tested 100% and then sample-tested the hell out of them prior to shipping). The program was called “Ship To Line” where the components came off the delivery truck and went straight to their main board assembly in a Just-In-Time kanban system that made circuit boards to very short product runs in a sales-order-driven line. Win-Win because we sold standard parts on huge mark up based on the added value to the customer of not needing to test, or keep stock.
Software Production:
I guess – in Agile- they are trying to work software production in a kinda Kanban way (Kanban is where the production line pulls all the way from the orderbook not pushes based on some production quota).
A software product (as opposed to a system) is issued on a release schedule – a bit like components coming off a production line I suppose. I like the concept of scrum-agile where there is a flexible backlog of requirements, a stack of finished code modules ready to be assembled, and the spec is only set in stone for the next release during a 2-week sprint.
As with all these schemes, the entire system becomes a lifestyle akin to a cult. Many of the consultants have never worked in it for real. They are more like groupies.
One thing needs to be created in the customer’s mind. Value. If they want to be lazy and indisciplined about requirements – fine. But they have to be made to pay more for that. Work to a 2-week sprint assembly of code modules that can be changed in backlog? – better. That allows it to be done less expensively and their price is less. That is Win-Win.
Components of whatever nature need to fit. They need to be pluggable and replacable, forwards and backwards compatible without complete system retest.
These can’t be made as 1-offs like a piece of art and then fettled-to-fit.
It isn’t so much “Waterfall vs Agile” but “Engineered to fit” vs “Sculpted like art”.
Agile is a way of life – not some optional training course you send individuals on. Everyone must be signed up to it, live it, think it, dream it and make it work. You can’t think you are “Doing” Agile and then say “Sorry – pair programming is too expensive”.
“Agile vs the ‘Real World’”
Agile and the other cultural approaches seek to redefine the real world. The real world is a cold and friendless place where young coders’ energy is traded to prop up indulgent, sloppy practices, billion dollar software projects fail and Individuals are bullied into committing to ridiculous deadlines that nobody believes in – and-it costs minds and bodies. Changing the culture with semiconductor parts was a great experience I participated in, it worked superbly well and was good business.
I’m hoping for great things from Agile-scrum. I just hope I’m on the winning side in the rough Rugby game they call working in Software Development.

Your claim that agilists “never — ever — disclose the risks and the downsides” may have some truth to it, but whenever you are advocating in a significant change in methodologies (for example from waterfall to agile), it can be difficult to gain corporate stakeholder buy in when you focus on the disadvantages and failures.

Kent Beck: “And yes, people in the agile community jump on you as a failure when you admit that things went wrong. This fear reaction keeps us from a lot of useful learning.”

A methodology such as agile (be it XP, Scrum, or a combination) and agile practices such as TDD, continuous integration, quick turnaround to customer, have pros and cons. I have found the biggest challenge in getting a team to adopt TDD is dedication and skill. Sometimes your team just needs to get their job done and TDD can appear to add more work to their plate. Once the tests start getting stale and breaking and if only some of the programmers are writing tests, then the managers end up supporting the disabling of test in order to move forward.

TDD requires a lot of discipline and skill. Often developments teams are not made up of expert programmers. And sometimes expert programmers can even resist because they have been successful without TDD.

In the real world I agree with Nathan, TDD is not a silver bullet, you need to practice “good discipline.” If you can integrate TDD into your team in a disciplined and healthy way, then your team will probably benefit. TDD is not new, but it has recently become more popular, and in many firms, standard practice. I am a big supporter of TDD and have personally seen the benefits, from small projects to huge enterprise applications running Fortune 500 companies.

My recommendation is that you explore it if your team is willing and learn from it. If you don’t like it, don’t use it. But don’t blame the agilists for not giving you all the downsides, instead see for yourself. Your will learn the downsides and challenges of implementing TDD pretty quickly.

Here are some of the things I like about TDD:

1. Interruptability: if you write a test first, and you get pulled away to a dreaded meeting, you can get back to work faster because you simply rebuild and run your tests to remind you of what is next and exactly what problem you were trying to solve before you were pulled away, since you wrote the code that would fail until you implemented the solution.

2. Less fragility: I’m less concerned about fixing and refactoring code when I can rerun the tests before checking in. Sometimes in large complex enterprise projects, a small change can break some distant dependency. We’ve all been there where we are afraid to make a change to someone else’s code because the app is so fragile. TDD reduces stress and makes programming more enjoyable.

3. Refactoring: a benefit from #2 is that you can keep a clean house. Nothing worse than code that “smells” and is not fun to debug or navigate. TDD allows developers to constantly refactor and delete dead code so you don’t have an increasingly growing pile of dung. It is much more fun to maintain a codebase that is 25% of the size and clean.

4. Less time wasted: when a test can pinpoint a bug in seconds that can take hours without test coverage, you realize that TDD can often reduce the time to develop software over the course of the project.

Disadvantages/Risks:
1. No buy in by developers or management: this is the biggest risk to TDD. If you don’t have people who realize the benefits, it will never work for your team.

2. Dependencies: TDD should be implemented in a way that has minimal dependencies. Depending on a database connection, network connection, web services, etc.. can lead to fragile tests. There has been a ton of work in this area, especially mocking frameworks, to get around this issue. Done right, your tests should test only your logic, not external components.

3. Hard to add TDD later: TDD is best to have in place before you start coding. It is very difficult to add TDD to an existing project. Often the application architecture will evolve in a “testable” way when you are using TDD. You will write your code and use application frameworks like ASP.NET MVC instead of ASP.NET WebForms for example to make testing easier and more natural.

I have been using TDD/Agile for the past 8 years. I have *only* on real projects, enterprise applications. Not only that, I’ve done production support, so I understand what it’s like to have things go wrong in production, and have to figure out from code and logs why something didn’t work.

TDD is not about a “fuzzy feeling”, it is about proving that a section of code does what it is supposed to. Writing code without tests is like tight-rope walking without a net. There’s a chance that you’ll get it right, but if you don’t there’s nothing to catch you. I can think of a number of times where code went into production, where developer made simple error of returning the opposite boolean than intended. (a very easy error to make).

“Agilists just don’t understand the meaning of calculated risk.” I think you don’t understand the meaning of calculated risk. I work on trading applications, where a simple error can result in large dollar losses. Unless your users have a high tolerance for programming faults, you’re better off not shipping untested code.

And in large applications, of the sort you claim Agilists don’t do, often have code which is very hard to reach through conventional functional testing.

I must say that I totally disagree with the notions in this post and a majority of the comments. There seems to be the perception that “Agile” is nothing more than Unit tests and leaving comments. When nothing could actually be further from the truth. Following this logic most people who left comments here opposing “Agile” would probably deduce that “Agilists” don’t need a specification before they start coding. That my friends is not “Agile” that is “chaos”.

Agile is a methodology, and like any methodology is open to corruption and misimplementation.

As a freelance agile developer, I have yet to actually work in an organisation that has actually fully implemented the Agile methodology correctly. What they have actually implemented is an “AgileFall” and alot of misunderstandings.

I will probably in a couple of days post a reply blog explaining exactly what I mean.

@RoobNoob keep taking those pills dude, and when the men in the van with the little white jacket show, don’t fight them off.

Surely the problem is that he had a bad experience with poor a poor traing company / presentation.

The guy has been a victim of the untrained trainer training course, wherby some outfit has simply added an “Agile ( Test-Driven Development presentation)” course to their portfolio, read a few books and tried to give a course. The students are then simply victims of the bad information.

The one thing I got from this is something I already knew. There is no silver bullet.

Techniques are just tools. Just as you won’t use a hammer where a screwdriver is needed. You won’t use technique A when B is needed. Nor will you build a building using only a hammer. Design and testing are both needed in engineering. Software is a bit more difficult as we also require capability for expansion for technologies we don’t yet know. So documentation is not optional.

As far as I can tell, it’s a set of ideas, many of which are common sense, many of which are utterly perverse. People end up picking at the worst ideas to criticize it and the best ideas to pitch it.

It’s like the “cloud” hype. Maybe a few people out there have a coherent idea of what it is, but even if that has merit, it neglects the fact that on a societal level it won’t work, for the exact same problem that Agile tries to solve in the first place: we can’t achieve perfection.

Ideally you’d take the Agile ideas that work for you and leave the rest, but if cherry-picking Agile principles results in dismal failure as some of the Agile practitioners here point out, then in the bigger picture, Agile should not be encouraged.

Tests like all programming can be written poorly. You can have bugs in your unit tests. Poorly written tests can be long and laced with dependencies that make maintaining the tests a nightmare.

I’m in a .NET project that started a year and a half ago, which has dependencies on Entity Framework scattered throughout the application, which is causing scalability problems. Due to all the dependencies, it’s been decided to start from scratch rather than try to re-write the code. However, TDD was not used for this project, and I can’t help but think that if TDD had been used and enforced, it would have pushed the project to be more loosely coupled, and a complete rewrite might not be necessary.

Really, the problem was lack of IoC more than anything else, but since tests encourage loosely coupling, it is a step in the right direction. So, I’m not opposed to companies being religious about TDD, BDD, or DDD. However, as stated, these are just tools that can help (not force) developers to adhere to good practices and reduce (not eliminate) bug counts.

From ’01 thru ’08 I managed a software engineering organization responsible for maintaining and enhancing a marine container terminal operating system; it supported the planning of the unloading/loading of vessels, yard management, supporting a variety of complex set ups (RTG, Straddle, Chassis, automated, etc). In addition it offered sophisticated vessel scheduling, and yard allocation, and truck optimization capabilities. It consisted of north of 2 MLOC, was started in the late ’80′s, written in in Apple PASCAL, converted to the PC, and later ported to C++. Get the idea! It was as real of a beast as you can imagine. No one knew how it worked!

It took years, 2 to 3, before an good engineer could do work on it worry free; exceptional folk were able to do it in less time. Eventually we hit into the idea of using TDD to help us out. We automated 180+ manual tests suites and declared that they established how certain features worked; if anyone asked how certain functionality worked we would refer them to its test suite/s; yes, yes, yes, I know that this was not the case, a lot of untested code could be there, but it did not matter since no one knew about it! The tests became the documentation. Eventually we had over 700 tests that required a lot of HW to run in less than 4 hours.

New engineers used the tests to “play” with the app and learn it; customer support folks used them to assess problems in the field., etc. Our CSAT went from less than 3 out of 5 to 4.2+. I believe the automated tests had a significant impact on these results.

Now, was it Agile? I don’t know. The point here is that these fads have good and bad things. It is up to the practicioner to understand what these concepts and tools can do for them.

The Agilists are often evangelical cultist zealots who will go on a witch burning if there is someone who “doesn’t get it”. Kudos for fighting back. I’ve seen people destroy projects by using the creed as an excuse to not document or comment, and just run ahead at 200 mpg until the project was destroyed in a fiery crash as road-kill on the Agile autobahn.

The founders are making tons of money promoting their system with exaggerated claims of 1000% “Productivity” gains.

Also, for those who talk about the Agile “methodology”–I’ve read the founders specifically calling it -not- a methodology, but a certain set of techniques for dealing with complex systems.

In the job market right now–generally speaking, if you don’t act like Agile and design patterns are the new religion, they act like you’re stupid, so everyone pretends to know something about it…not sure that many really get the main points and are able to use them effectively.

It did just dawn on me the other day, the thing about people over processes…that’s actually kind of cool.

Having been a “saved” agilist from some 15 years now I must admit the article has some good point. There are som good counter-points too. In agreement with the article author I’d say …:
- a few lines of code documentation, especially on the over-all algorithm goes a long way towards understanding code quickly and does not take a long time to deliver as you are coding. Agile does NOT say you should not do this. Some overly zealous agilists may, but there is no general agreement.
- agile does NOT say no documentation (it just says oral communication is usually preferable), in fact – agile mostly talks about documentation of requirements and how almost pointless it is to trust written big up front specification documentation. There is no general agile view on how well to document the system itself or not. This is really up to the product owner to decide.
- if you do all with user stories that you can including acceptance criteria, there is actually quite a bit of “specification” for just a 40 hour work item – the big difference we don’t do this up front before we start we do this while we are working supplemented with lots of oral communication and discussion, just in time – not ahead of time.
- agile done properly is far more disciplined than any specification driven process I’ve ever seen – we just have discipline as we go, rather than up front before we start walking. Discipline, discipline and discipline again is the way to succeed with agile. Coaching clients this is one of the most difficult cultural changes things to understand and make happen – agile requires far more discipline than they’ve ever seen before. And involvement, focus and fun.

Admittedly, I don’t really know what Agile is. I think I got some sense from reading this post. However, one thing that immediately occur to me is your point about “If it’s not testable, it’s useless”. Well, your examples suggest that you completely missed the point. Note that no one says that if it is not TESTED, it is useless.

But really, if your code *cannot be tested* (i.e. is not testable) for compliance with the spec or the definition, it means that you don’t have a WAY to KNOW it is correct. Even not theoretically. Then, what use does it really have?

Hi,
Having spent all week trying to use 2 commercial codes that have more bugs from lack of testing. I believe that codes that are not actually tested waste countless hours for real people that are trying to use your codes

The other main advantage of an Agile development is that you, when followed correctly, end up with a code and user interface that is actually useful to the end user and does not subject them to unnecessary burdensome point/clicks/reformatting inputs/steep(almost unclimbable) learning curves GUI’s. The problems with the standard approach is that it builds in difficulties that can not be easily re-engineered.

Totally agree here with you on the article. Some Agile practices are not very practical and real. You just need to adapt them to your team. As with any advice, methodology, we have to use what makes sense and ditch the rest.

I think you are confusing “untestable” with “untested”. They are not the same thing. Untestable means no test can be devised to test something. Not tested means simply no test was executed to exercised a peace of code.

Honestly, I don’t think a lot of people write untestable code. Code that runs in conditions that are extremely hard to reproduce in non-real world environment/data set etc.

I don’t think anyone disagrees with you on shipping untested code. But it’s not the same as untestable code.

I’ve been practicing Agile for a couple years now, and I still can’t get my teams to adopt TDD. They are, howevere, writing functional and unit tests. I’ve always liked TDD as a concept, but putting into practice is difficult.

I really like Agile, as it’s more predictable than traditional project management. I think the key to my success with it is that I don’t have any non-technical project managers on my team.

Nice write-up, nice to see it’s still getting traction (and comments) so many years subsequent to its posting. Yes, many Agile “practitioners” are treating Agile as a cult, with dogmatic (and naive) ideas and with arbitrary, unproven (and sometimes ridiculous) assertions presented as the-one-and-only way to succeed. TDD is a technique/approach/pattern like any other, helpful in some cases, useless in others, worth having in one’s toolbar in all. Presenting it as a commandment is, well, indeed childish.