Monday, April 19, 2010

Questions I'm asking myself

But because Scrum works in short cycles and doesn't include any engineering practices, it's very easy for teams using Scrum to throw out design. Up-front design doesn't work when you're using short cycles, and Scrum doesn't provide a replacement. Without continuous, incremental design, Scrum teams quickly dig themselves a gigantic hole of technical debt. Two or three years later, I get a call--or one of my colleagues does. "Changes take too long and cost too much!" I hear. "Teach us about test-driven development, or pairing, or acceptance testing!" By that time, fixing the real problems requires paying back a lot of technical debt, and could take years. What frustrates me the most is that this situation is entirely avoidable. In a green-field environment, the solid agile engineering practices included in Extreme Programming pay for themselves within the first few months. Without XP's agile engineering practices, code quality and productivity asymptotically decreases over time. With them, productivity starts lower, but then it asymptotically increases. I can't prove it, but my sense is that the two curves cross at about the eight-week mark. Maybe sooner. Agile engineering practices aren't only important--they pay for themselves! Doing anything else is pure negligence... if you understand your options. Scrum is silent on the matter.

I think this is very indicative of the desire among managers especially to want to believe in a process or formula, and to fail to recognise the comparitively massive impact of the quality of the people in the team on project outcomes.

Basically, great developers tend to do good work, and bad developers tend to do poor work, and no process or methodology can fix that.

The real problem that successive methodologies have tried to solve is that the majority of professional software developers don't actually know how to write good code.

Scrum is just the latest competence substitute, and it understandably falls just as short as all its predecessors.

Invest in the people, their skills and in helping them to exercise discipline and reward good habits. Sadly, it's not a quick fix, which is why most organisations don't bother.

The missing ingredient from the Agile Manifesto is CRAFTSMANSHIP. It's like a whole bunch of musicians running around trying to look cool and talking about their "art" when the vast majority can't play a major scale. The problem is that teams adopt Agile before they've learned all the "hard skills" that the Agile founders like Kent Beck and Bob Martin take for granted, like OO design. But they talk about software development as if they've mastered it, when all they've really mastered is planning poker and how to run a retrospective. They still write poorly constructed, buggy and ummaintainable code. Because it takes years to learn how to write elegant, well-constructed and reliable code. And why waste years honing those skills when you can get Scrum Certified in a week?

Sadly, all these apprentices don't know what they don't know and simply will not accept that they have much to learn (as do we all!)

This is great timing, actually. I'm chairing a conference on Software Craftsmanship in London on Feb 26th. I hear more and more from people who are as frustrated as we all seem to be about this. Maybe our time has come? (Well, you never know...)

...they haven't paid enough attention to the internal quality of their software. If you make that mistake you'll soon find your productivity dragged down because it's much harder to add new features than you'd like. You've taken on a crippling TechnicalDebt and your scrum has gone weak at the knees

The scrum community needs to redouble its efforts to ensure that people understand the importance of strong technical practices. Certainly any kind of project review should include examining what kinds of technical practices are present. If you're involved or connected to such a project, make a fuss if the technical side is being neglected.

...it isn't methodologies that succeed or fail, it's teams that succeed or fail. Taking on a process can help a team raise it's game, but in the end it's the team that matters and carries the responsibility to do what works for them. I'm sure that the many Flaccid Scrum projects being run will harm Scrum's reputation, and probably the broader agile reputation as well.... Teams that fail will probably fail whatever methodology they mis-apply, teams that succeed will build their practices on good ideas and the scrum community's role is to spread these good ideas around widely.

http://agilesoftwaredevelopment.com/blog/jackmilunsky/state-agileScrum is a methodology and process that provides the mechanisms for teams to learn and adapt. Scrum however doesn't say much about the meaning of DONE and how to accomplish that. That's where XP comes in. XP has great practices around engineering discipline. It teaches us all about craftsmanship and producing quality work. I personally cannot see anyone practicing Scrum without at least some elements of the XP toolset. Be it pair programming, TDD, ruthless refactoring, emergent architectures etc.

We are faced with a global armada of agilists who know absolutely nothing of complexity theory and game theory. Yes, they know words like emergence and self-organization, because everybody's using them. But do they know where those words came from? And do any of them know what a phase transition is? Or a fitness landscape? Or an adaptive walk? Or an attractor?

Agility is about moving software projects to the area of complexity, right between order and chaos... If you don't adopt a properly balanced set of practices, including technical ones, trying to be agile can harm your organization by moving it to far, from the ordered to the chaotic regime. Some parts of the system (like quality) might go out of control.

OK, now it's time to sit back, relax, and enjoy the comments section of this post. Actually no, I will not relax. There's still plenty of stuff to do. Like, for example, figuring out how to introduce those bloody XP practices in our organization. Because I believe that some of them could actually be quite useful for us as well.

To blindly follow an approach in a mantric way (i.e. you will not practice agile without XP, you will not practice agile without XP, you will not practice agile without XP) is to simply ignore that agile itself will have to adapt and change as it adapts and changes the programming discipline. Adaptability is the key not only in agile development but in all programming. To approach it in a dogmatic way, whatever that dogma, is - in my view - to miss the whole point.

Boehm and Turner's assertion that Agile methods can only be applied successfully by senior developers is due for an update. When Agile methods were new, it surely required people who had enough experience to understand the potential benefits of adaptive development and to recognize and cope with the risks and obstacles. Once an organization has completed one or two initial projects using an elite team, it is appropriate to expand the capability for adaptive development by bringing in junior developers to pair with their experienced colleagues. The result is knowledge transfer in the context of real project delivery. In fact, this is just how we have built our internal Agile development group from its initial size of 6 people to its present size of 70. It's worth noting that a number of these have relatively little overall professional experience, yet every one of them is an effective Agile practitioner.

Can you separate the interdependent Agile development practices, introducing them one at a time in baby steps and not acrue Tech Debt?

About Me

Software Developer with 17 years experience as senior developer and tech lead in many large and small agile teams. I enjoy consulting with teams to implement improvements in development, testing, and devops practices leading to higher-quality software. I've experienced many of the pros and cons of Agile/Scrum/XP/DevOps and I'm always looking for continuous improvement in both team efficiency and personal skill. I believe the world needs more well-rounded developers, capable of seeing themselves in the bigger picture, able to quickly spot bottlenecks in the delivery pipeline - whether it be in Dev, QA, or Ops - and work with a sense of urgency to fix them with cutting-edge technical ability while using well-honed interpersonal skills to help improve the culture around them. Passionate about giving back to the community, I co-organise the DevOps Brisbane Meetup group and help run study groups for professional software developers on topics such as AWS Solutions Architect Certification, Continuous Delivery, Functional Programming, NoSQL & Distributed Systems, and enjoy inspiring IT professionals to sharpen their craft through professional development and group learning.