The not-so-flattering Agile analogy (not invented by me) is like trying to build a house one-room-at-a-time (where software is the "fine carpentry and dry wall" and data structures are the "foundation and roof"). "O, wow, I really like this kitchen" says the customer, "let's add a bedroom". Seems easy if you're doing fine carpentry and drywall. However, how do you do Agile foundation work and roofing? Brings to mind the parable told by the radical rabbi about the foolish man building his house on sand. The wise man built on a solid foundation.

It therefore always surprises me when Application programmers tell me that all 'Agile' techniques are directly applicable to database development.

Heh... what did you expect, Phil? Many application programmers and their managers have misinterpreted the Agile Manifesto as meaning there doesn't need to be any documentation or preparation and have also misinterpreted what Knuth said about pre-optimization being the root of all evil. With exceptions, of course, most of the application programmers that I've met and worked with personally all think the same thing... disk space is cheap, machines with more memory are coming, stored procedures are an evil to be avoided, and an RDBMS is just a place to store data.

As a very interesting (to me, anyway) sidebar, I'm currently working with 3 wonderful Application Developers that have seen the proverbial light and actually get it. Like the song says, "I'm in Heavennnnn!" I think that part of the reason why they get it is because they're the ones that have to clean up after their "Agile/No Pre-Optimization/Design-on-the-fly" predecessors.

--Jeff Moden"RBAR is pronounced "ree-bar" and is a "Modenism" for "Row-By-Agonizing-Row".

First step towards the paradigm shift of writing Set Based code: Stop thinking about what you want to do to a row... think, instead, of what you want to do to a column."

(play on words) "Just because you CAN do something in T-SQL, doesn't mean you SHOULDN'T."--22 Aug 2013

The Agile stuff ain't new at all. I worked in a CSC team in the Nineties practicing some techniques that were recognizably 'Agile'. The only difference was that we used Big sheets of paper stuck on the walls with Blu-tac rather than post-its and called the meeting 'shirtsleeve meetings'. I was in the Data Architecture team and we insisted on getting the data architecture planned out in its entirety before the coding started. It was the only part of the project that got finished, all beautifully mapped out in ERwin.

It therefore always surprises me when Application programmers tell me that all 'Agile' techniques are directly applicable to database development.

Heh... what did you expect, Phil? Many application programmers and their managers have misinterpreted the Agile Manifesto as meaning there doesn't need to be any documentation or preparation and have also misinterpreted what Knuth said about pre-optimization being the root of all evil. With exceptions, of course, most of the application programmers that I've met and worked with personally all think the same thing... disk space is cheap, machines with more memory are coming, stored procedures are an evil to be avoided, and an RDBMS is just a place to store data.

As a very interesting (to me, anyway) sidebar, I'm currently working with 3 wonderful Application Developers that have seen the proverbial light and actually get it. Like the song says, "I'm in Heavennnnn!" I think that part of the reason why they get it is because they're the ones that have to clean up after their "Agile/No Pre-Optimization/Design-on-the-fly" predecessors.

Amazing what happens when confronted clean up is involved.Kind of like when I ground boxes on the floor, and then started to weld.Every welder should have to grind their own welds.It tends to improve the process.

We do Agile dev in our shop and spend a decent amount of time planning DB changes up-front to avoid doing it over as we move forward. Sometimes we overplan, sometimes underplan, but it helps to at least attempt to get it right up-front. It helped once the team finally understood that you couldn't just drop tables and create new ones like you can with DLLs and EXEs. It took a while for that to sink in, but they understood it and it helped us handle DB migrations a little better in the future. There are still times we need to do a large data operation the night of a release, but on the whole knowing that helped us make better plans for the future. Finding the correct balance of "just enough" as opposed to "plan it as far out as we can" can still be challenging, but having the team backing up those decisions makes a huge difference.

I actually like the Agile plan when it can work. I've seen too many large DB Projects fail due to using waterfall (think Amazon Falls) methodology and then years of work to deliver the finished project. I'd much rather start small enough to show something and get feedback as we go. That helps us make course corrections and tackle more important needs as they come up instead of delaying them further.

ganotedp (1/7/2012)The not-so-flattering Agile analogy (not invented by me) is like trying to build a house one-room-at-a-time (where software is the "fine carpentry and dry wall" and data structures are the "foundation and roof"). "O, wow, I really like this kitchen" says the customer, "let's add a bedroom". Seems easy if you're doing fine carpentry and drywall. However, how do you do Agile foundation work and roofing? Brings to mind the parable told by the radical rabbi about the foolish man building his house on sand. The wise man built on a solid foundation.

Peter Schott (1/11/2012)I actually like the Agile plan when it can work. I've seen too many large DB Projects fail due to using waterfall (think Amazon Falls) methodology and then years of work to deliver the finished project. I'd much rather start small enough to show something and get feedback as we go. That helps us make course corrections and tackle more important needs as they come up instead of delaying them further.

I agree with this (having been wandering amongst a few agile teams of late) but with a caveat. The powers that be must understand that you WILL require entire sprints for re-design of the core module to integrate new work. It's not an optional choice. This will require downtime when the fixes are put into place.

With that caveat, I agree you can modularly build databases within the agile structure. However, I also agree with Gus. I've seen too many shops that got their prod powers removed and suddenly Agile is 'problematic'. It's all to often a Cowboy mentality.

- Craig Farrell

Never stop learning, even if it hurts. Ego bruises are practically mandatory as you learn unless you've never risked enough to make a mistake.