"I was in this Agile shop, and they did 2-hour stand-ups, how can Agile help productivity?" — "You were clearly doing it wrong, stand-ups are not status reports to managed and they are time-boxed to 15 minutes overall" — "Oh, you are invoking no true scotsman, you!!!111"

Agile processes, I observed, failed when they were not, in fact, agile (look it up in the Oxford dictionary). Management wants to create an impression of adopting new methods, while not letting go of all their favorite toys like micro-management and long-winded status reports (so they can score better in their bosses' or customers' eyes). Or developers believe the new process will magically relieve them from the burden of responsibility, accountability, or thinking as such.

Which all makes me think that there must be something really irresistible about cargo cults.

Or that people who claim decades of programming experience are off by an order of magnitude evaluating their own intelligence. Because common sense is a thing, motherfucker. For example, take a look at this comment:

As a Program Manager, while Waterfall techniques could frequently end up with late or over budget, or both, projects, at the end of every project (I oversaw 5 multimillion dollar projects using Waterfall methods) we at least had a working application that met the original specifications.

What good is being so proud of a working application meeting the original specifications if the said specifications are no longer relevant? Is this some kind of a masturbatory relief after six months of stroking one's mind to the point it's completely numb? I thought we already absorbed the idea that software projects are there only to satisfy a specification. They are there to make the motherfucking revenue, or to enable people to make the motherfucking revenue.

The guy at the Exhibit A tries to make a point that Agile fails because developers are too stupid to reason about why a particular rule works, thus we need a new shiny methodology (called GROWS, without a clear explanation what it even is) to accommodate stupid developers. Why not educate or get rid of them instead?

TL;DR version: programmers are by no means a kind of an intellectual elite. The humanity is a total failure. Makes me sad.

What good is being so proud of a working application meeting the original specifications if the said specifications are no longer relevant?

Yeah...how many things did the users recoil in terror over when they saw what they'd really asked for? That stuff is hard, and I don't see a way around iteration. I came across this post this morning, which seems apropos:

The guy at the Exhibit A tries to make a point that Agile fails because developers are too stupid to reason about why a particular rule works, thus we need a new shiny methodology (called GROWS, without a clear explanation what it even is) to accommodate stupid developers.

Go back to your quotes from Slashdot above. Do you think that:

Developers chose to have no QA department?

Developers chose to have 2-hour-long standup meetings?

How do you propose educating developers would help either of those scenarios?

TL;DR version: programmers are by no means a kind of an intellectual elite.

Duh?

Look at the shitty software produced by them. Especially in the open source world, where there's practically no management at all. If they are some "kind" of intellectual elite, they're the "kind" who doesn't give a flying shit whether they produce good software or not.

I personally agree that Agile really sucks at greenfield development. It only works if you're modifying an existing, working, product-- and even then it only works if your modifications are small enough to be completed in a 2-week period of time.

The only way Agile works for greenfield is if you extend the length of a sprint to ludicrous amounts, or completely trash the demo requirement. And then programmers on your team bitch about it being no longer Agile.

It's a natural consequence of the waterfall methodology. You collect the requirements and build the spec in the beginning and then go away for a while and come back with a finished product. That's not to say that everything is no longer relevant.

The only way Agile works for greenfield is if you extend the length of a sprint to ludicrous amounts, or completely trash the demo requirement

Depends on the project. I've done greenfield projects before that, after a couple of weeks setting up the basics, lent themselves very well to weekly or biweekly "here's what the application does that it couldn't do last time" demos

If you go a single sprint without anything demonstrable, you're "breaking the rules" of Agile. That's my exact complaint.

Currently, I'm working on back-end C# code to figure out health insurance spending account rules and rates, there's no development team on Earth that could start from scratch and have a demonstrable product for that after 2 weeks. 2 months? ... maybe.

(Of course I couldn't demo it even if my part were done, since the front-end guys generally run at least a month behind the back-end development, and there's nothing to demo until you have a UI.)

Our solution is just to trash the entire "demo day" concept completely. Which means we're no longer doing "Agile", we're just "inconveniently splitting a several-month-long task into tons of individually-useless 2-week long tasks". A.k.a.: adding tons of management overhead for no discernible benefit.

I've noticed a lot of theory, tooling, and methodologies assume standalone compiled applications, which is about the easiest use case. Web applications, integrations between COTS, API-only applications, SSIS packages or batch jobs, all kind of have to shoehorn themselves into tools meant for traditional applications.

Yeah, I was thinking of relatively small desktop apps where it's the same person doing the UI and the business logic. In that case, it's not hard to make a load of non functional buttons and wire them in as you get to the logic, or add screens as you go.

If you're doing backend stuff it is more tricky to demo without writing your own shitty UI to plug in to the logic, and then people complain about the shitty UI instead of the shitty logic

To anyone with familiarity with the history of software development, the answer to your question was obvious. You've displayed some savvy in this area before, so I'm still confused why you'd ask such a braindead question.

You've displayed some savvy in this area before, so I'm still confused why you'd ask such a braindead question.

I think "backing-up assertions with data" is the opposite of braindead. You know me well enough that none of this should have come as a surprise.

And you even admit here that the assertion comes out of nowhere because it's "obvious". Well you also know I don't do "obvious", and you know I'm likely to say "a lot of 'obvious' things are utterly wrong", and I don't know why you posted that unless you're suddenly suffering from amnesia.

It's "obvious" that the stomach ulcers my aunt suffered from all her life are caused by stress or some other emotional state. It's so obvious that no medical researchers even bothered to attempt coming up with a cure for decades. Then, shortly after she died, Barry Marshall and Robin Warren said "fuck 'obvious!'", actually studied the problem, and determined it was a simple bacterial infection all along.

Look, of all the millions of waterfall projects that have been done, how many have had the specifications change during development? 10%? 50%? 90%? 100%? How do you know? How does wft?

Yeah, I have no clue what the numbers are, but I'd be surprised if it was very far from 100%. It is the whole reason we went from it to things like spiral design and now to agile. Because people don't know and can't predict this stuff without doing it and trying it and seeing what works.

It's "obvious" that the stomach ulcers my aunt suffered from all her life are caused by stress or some other emotional state. It's so obvious that no medical researchers even bothered to attempt coming up with a cure for decades. Then, shortly after she died, Barry Marshall and Robin Warren said "fuck 'obvious!'", actually studied the problem, and determined it was a simple bacterial infection all along.

And I'm sure that you guys even get all of your specifications correct for just a short sprint and never need to revisit anything. Now scale that up to the whole damn thing. There is nothing subtle about people thinking they need one thing and only realizing later that it's not what they need. There is no mysterious waving of hands about a cause, like the stuff you brought up there.

I have not seen a Waterfall process in decades. As soon as there is any type of "change order" it is not waterfall.

I have seen hundreds (200-400) of projects that claim to be "Agile", yet have no correlation to the 12/4 of the Manifesto

I regularly work on complex project like blakeyrat posted (even some in the insurance industry). I have never had a problem with a functional demonstration each sprint (I do advocate 3 week sprints, but that is not so much of a change).

Having "front end guys" and "back end guys" is usually an indication of an organization that will have problems in being agile. Having thin vertical slices where UI<->DB can be worked on by one team for a small segment of the application domain is usually much better.

For Greenfield, Agile also works great. I just finished bidding on a project for a piece of avionics test instrumentation. Before project kickoff the product backlog already has over 200 user stories (which have not yet been ranked).

Is that valid? The stuff shifting during our sprints is what part of the requirements to do first, not what part of the requirements are valid at all. In fact, since the requirements are based on Federal Law, they've been remarkably stable on this particular project.

Having "front end guys" and "back end guys" is usually an indication of an organization that will have problems in being agile.

The front-end is using radically different technologies. (Single-page apps, or whatever the latest buzzword is compared to WebAPI 2.0.) I'm sure a holy angelic perfect being like yourself has no trouble hiring employees who know both, but us mere mortals have to compromise.

And no, I had no part in picking either technology so don't come in here and bitch at me for "my" stupid decisions.

For Greenfield, Agile also works great. I just finished bidding on a project for a piece of avionics test instrumentation. Before project kickoff the product backlog already has over 200 user stories (which have not yet been ranked).

The funny thing is I'm not even saying Agile doesn't work. For all I know, it's the best method. My entire career I've been at companies that either have team sizes too small to bother with any development methodology at all, or that blindly use Agile (or some variant of it). I just don't know, and haven't experienced any alternatives.

In fact, since the requirements are based on Federal Law, they've been remarkably stable on this particular project.

This is a particular sort of stability where some projects have an advantage. But in a waterfall project, you'd have planned out what each screen / page looks like before you started writing. Not just the requirement that you need to frobnicate the listicle.

The move away from spatial interfaces has been a 20-30 year trend, and produced shitty results. However, it's so established now that you can't even assume the average programmer knows what the term "spatial interface" means anymore.

Well, yes, that is their choice, since ONLY developers should be in these meetings. If the developers choose to allow the product owner to attend, then that person is an observer only, and should remain SILENT,

Again, it is a matter of organization. There is definitely a need for many "quality" related issues, but personal responsibility is the most effective means of achieving actual quality. Having a distinct department is really counter productive.

Of course, it is the 'simple past tense" of shall... and Shall has the definition (among others) of "will have to, is determined to, or definitely will: will have to, is determined to, or definitely will:

So it is a matter of the developers simply (simple and easy are different, and there may be side-effects) say "You want Agile, this is how agile is done, now GTF out".

Capers Jones found that projects that had only developers as testers tended to succeed less than 25% of the time, while projects that had a distinct testing department succeeded 90+% of the time. The separate QA department that doesn't have a personal bias toward the code like a developer would and doesn't answer to development leads is key to getting good results. The most successful companies have > 3% of their IT staff as dedicated SQA professionals. http://sqgne.org/presentations/2012-13/Jones-Sep-2012.pdf

Again, it is a matter of organization. There is definitely a need for many "quality" related issues, but personal responsibility is the most effective means of achieving actual quality. Having a distinct department is really counter productive.

Yeah, QA is not a silo! At least, not if you want it to be effective...

You need testers, though, as Capers Jones' study points out. I personally would rather have them detached and paired with functional-unit teams, though...there's no sense in trying to lock QA away from devs.

At my last company, after the QA manager was fired for being a cunt the QA department was restructured so each programming team had a couple of testers attached to it, with a handful of floating testers for the teams who weren't big enough for a dedicated resource or extra help in the main teams if they were busy.

I left not long after that (and was on one of the teams deemed too small for our own tester), but it seemed like a pretty decent setup from what I saw

If you go a single sprint without anything demonstrable, you're "breaking the rules" of Agile.

For a given definition of "demonstrable" that's applicable to your project. For some it can be tests passing, or even an ad-hoc tool using your framework (CLI or otherwise). "Demonstrable" does not always mean clicky-touchy GUI in the real application.

Also, if you cannot come up with anything demonstrable in any way in a month's time, an alarm bell should go off that you're drowning in architecture.

1) I have not seen a Waterfall process in decades. As soon as there is any type of "change order" it is not waterfall.

It can be "waterfall". Waterfall just means that changes are expensive, sometimes prohibitively, in terms of both implementation and red tape (and can come out just as useless) so users are taught to work around bugs. You can see this in major state/government institutions where you are basically fucked once you enter that field (they actually demand waterfall in most cases from you, I've seen that).

Also, Agile stands against Big Design Up-Front, but does not prohibit design up-front at all. When you have enough experience, you learn to know the differences between what you can tell right away, what you can tell with some thinking, what others can tell you, what no one can tell. You don't apply rules of thumb blindly.

How long a sprint is is not an universal metric. Whoever tries to impose that on you is a bloody fool.

At my soon-to-be-former company we have sprints that are three week long and are very happy with that. Your mileage may vary greatly.

Also, in my freelancing team of one, I employed two-day sprints: implement today, look back on it tomorrow, regroup and pick the next work item. Split if it's too large. It worked just right for some things, otherwise I would just get stuck in shuffling code with no real progress.

For a given definition of "demonstrable" that's applicable to your project. For some it can be tests passing, or even an ad-hoc tool using your framework (CLI or otherwise). "Demonstrable" does not always mean clicky-touchy GUI in the real application.

For the backend stuff I'm working on at the moment, I have my own basic CLI interface for testing it. It's completely separate from the actual GUI but it means I can test the functionality before the UI is written to call its properly

I don't even know why I'm arguing on this one TBH. My current workplace is AINO and I've never done a demo here. I'm just saying if you're demoing your own functionality you don't necessarily need it to be in the same UI as the customers will use

Not when the term was introduced. The concept of an "Engineering Change Order" was prohibited by the true waterfall contracts of the 1960's and early 1970's. Each stage was to be signed off, and once signed could not be altered. The only option was a cancelation of the project. The next stage could not be started (or in many cases even talked about) until signoff of the previous was complete.

Another classic case (a good friend of my father was involved in this one) was where there were typographical (Spelling, mainly) errors in the definitions of what the reports should contain (at the time, everything was hard copy). The code was faithfully developed to contain reports with misspelled headings and other errors that were in the spec. The software was delivered to the customer and accepted. Then a brand spanking new contract was sent out (RFQ) to create a new software package that eliminated these "limitations".

We use 2-week sprints, but carry over items more often than not. We do have a more stringent guideline that a backlog item that's in current sprint for more than 4 weeks needs a closer look in the weekly backlog item review (combined product owner demo + sprint status + upcoming items review)

I guess either your estimates don't improve over time, which raises question of what quality your retrospectives are, or you need to chunk more finely, or you need to lengthen your sprints. Sometimes you can't decompose further and the stories are still too big for two weeks. It's not like one size fits all.

Wait what, you always develop projects as a whole? You never break them down? No modular structure whatsoever? How do you tell a fucking module or a goddamn library works? You rush to other parts before you can even verify your APIs work?

Either there is a way you can tell, or the hell freezes before I'd even consider to pay you by the hour.