Scrum Will Die

SCRUM WILL DIE

Now, don’t get me wrong here. I like Scrum. I think it is a good thing. I think it changed the way many organizations think about software development, but I also think it is not so good.

I remember when Scrum was first gaining traction. I thought to myself, is Scrum a fad? I knew Agile was around to stay because the amorphous thing that is Agile is a bundle of good practices that really make sense, but I often wondered about Scrum. I wondered if Scrum was over hyped by consultants trying to make money coaching people and trainers trying to make money training people to be certified ScrumMasters. I especially questioned it when I noticed that the only requirement of becoming a ScrumMaster was to breathe the air of a ScrumMaster trainer for two days. (I know there is a trivial test now.)

The failings of Scrum

Scrum addresses many problems of a development team, but it also tends to create problems. After being in Scrum environments for several years and talking to many other people in organizations doing Scrum, I’ve identified what I think are the main failings of Scrum.

Planning

Planning is a total waste of time. It provides no benefit to the bottom line. The supposed benefit of planning is to be able to identify how much work a team can do in a sprint and use those metrics to plan releases and schedules.

The concept seems solid. It seems to make sense, but the problem is we are devoting one day of a two week sprint to an activity that basically gathers metrics. That is a 10% overhead if your planning session doesn’t go over a single day. Essentially, planning’s primary purpose is to chart velocity and commit to some amount of work. Let’s break some of this down.

Velocity

Velocity is the amount of work a team can get done on average based on how much work they have been able to get done in the past. What comes out of planning is based on estimates teams give to backlog items during planning.

Again, seems like a solid concept, but here are some of the problems. First of all, estimation is almost always wrong. The only way to get a real accurate estimation is to talk about the backlog item until you have squeezed all the requirements out of it up front. And if you do that, then your planning will probably take two days.

Even if you manage to get accurate estimates, if your team is not stable and your sprint length is not stable it won’t really matter, because you will be taking the average out of something that itself is not stable.

Commitment

One central unspoken theme of Scrum is commitment. The team commits to get a certain amount of work done in a sprint. The business commits to not change the work the team is working on. These commitments are critical, because without them time boxing fails, velocity metrics fail, and trust is lost in both directions.

So, what is wrong with commitments? They cannot be followed. Everyone means well, at least I hope they do. The problem is, like a fat kid in a candy store, they just can’t help themselves. The business doesn’t want to change priorities, but a critical issue comes up. The development team wants to commit to the sprint, but the development team can’t make more code get done faster simply by wanting to. They can add more hours to the sprint, but then they are skewing the velocity. The only way the development team can realistically commit to the sprint is to ‘pad’, and that is a very bad word. Don’t ever say ‘pad’ in a development shop.

Tasking

I’ve tried to fight it. I have closed my eyes and said “NO! NO! NO! There is a proper way to do tasks without copying and pasting,” but it’s simply not true. I have tried many different techniques, have asked many other Scrum teams, but everyone essentially copy and pastes the same set of tasks. The problem is, you don’t know the tasks for most backlog items until you have had some kind of design session to decide how you are going to implement the solution. I just cannot see a way around that simple plain fact. So, what we end up doing is making up templates and tasks that are generic.

Product Ownership

Scrum says there should be one product owner. I agree with this point. In theory it works great. In practice it ends up putting a whole product’s life into the hands of one person. Which person? Well, if you make that person someone high up in the organization, they won’t be the product owner for the team, but rather the boss. The team will just do whatever the product owner says and stop being autonomous and self-managing. If you choose someone lower in the organization, you end up having someone who cannot make decisions, and now you have all these high up business people, who are basically worthless, trying to find ways to be worthwhile (which usually results in them abusing the poor product owner.) I like to call this the “product renter.”

Scrum Meetings

These seem like a really good idea on the surface level. As a ScrumMaster, I tried to make these work correctly. 15 minute stand up meetings, you say what you did, what you will do, and what’s impeding you. The problem is, if you are working together doing pair programming, and “swarming” on backlogs, you already know what is going on and you are just repeating some rhetoric. If you are not working together on backlogs and are working in isolation, 15 minutes is not enough time. So what ends up happening is you either have a 1 hour long meeting where everyone talks about technical details of issues they are having, or you have a 15 minutes meeting where everyone repeats the same stuff everyone already knows.

What next then?

I don’t think Scrum is particularly bad. I actually like Scrum and think it can be a great process for a team that can help them to learn to use Agile techniques and vertically slice the system instead of trying to horizontally slice it. The best parts of Scrum are really Agile. Scrum is kind of forcing you to be Agile and do things like creating user stories, test driven development, having always releasable code, etc.

So, why even use a process at all, or why not make our own? Rules and process are important because they keep us from arguing about how things should be done, and they keep us from wasting time reinventing the wheel. The trick is in having your process lightweight enough that it is not burdensome, but constraining enough that you can enforce good patterns and principles without having to argue about and debate them each time.

So my advice is to learn what you can from Scrum, but don’t think it is the “be all end all.” Scrum is a fad, it cannot last ,at least not in the form it is now, because it has problems. But hold onto those Agile principles you learn from using Scrum; those principles will last much longer, if not indefinitely.

From my point of view, you’re completely wrong. I say this because what I read in this post shows me that you didn’t get to the heart of Scrum. You stick with what is written exemplary in books and not with what works for you.

Planning: planning is there in order for the developers to know what to build. That’s the main reason. Make your planning sessions shorter (estimating should be fast, don’t argue, does not have to be super-precise). Our panning meeting is about 3h for two week sprints.

Velocity: velocity is a very rough measurement. Therefore, you don’t have to squeeze out all requirements. Just say, this story is about the same amount as this from the last sprint, ket’s call it a 3. That’s good enough.

Committment: Yes, I agree, comittments are no good. But the team has to know how much work they are likely to complete in the next sprints. This simplifies looking ahead to get smoothly from one Sprint to the next.

Tasking: I really don’t get your point. Copy pasting? Don’t write tasks for things that are needed to get a story “done” (definition of done) like “write docu”, “test”. Write tasks that describe what the team has to build: “extend DB schema”, “add validations to XY”.

Product Ownership: Yes, this is the hardest part in Scrum. But if done right, provides immense advantages. If a company decides to switch to Scrum, the whole company has to change, not only the development. Otherwise “organzational gravity” kicks in and results in what you are describing.

Scrum Meetings: Do you hold retrospectives? If yes, why don’t you discuss, what you should change? If all team members know what all others do (although I thought that) there is still the part to coordinate the next 24h, who is pairing with whom, what the PO learned today from the customer, … If there is really nothing then skip it.

What next then? Kanban? Maybe. Just keep in mind that Scrum optimizes throughput (amount of work done in a time slot), Kanban optimizes cycle time (time needed to get a work item done). Make sure you know what you really want.

From my point of view, you’re completely wrong. I say this because what I read in this post shows me that you didn’t get to the heart of Scrum. You stick with what is written exemplary in books and not with what works for you.

I both agree and disagree with you, but I think you prove my point with this one statement. When you say that I should use what works for me. You are saying that Scrum will die. You may not realize that is what you are saying, but you are suggesting modifying a process, because it doesn’t work as it is. Scrum is designed to be a lean process, if you change it or reduce it, you have fundamentally broken or discarded it. There really is no way around it. Ultimately, you are either doing Scrum or something else that is kind of like Scrum. I am not saying the something else that is kind of like Scrum will die, I am saying Scrum will die. I am saying the something else kinda like Scrum will evolve and become the next thing. Perhaps it is Kanban… I do not know.

As for planning and velocity. I have tried many approaches, and I agree with you that making the planning sessions shorter and making rough estimates is the way to go. The problem is that at that level of granularity the cost vs benefit value is not there. There is no point taking up all that time to ultimately produce something that is so high level that it is not measurable. I believe this can be successful is followed as you describe, but that requires non-change. What I mean by this is that if you can keep the business priorities unchanging for the sprint, you can keep the sprint lengths equal, and you can keep the team members the same, then you can get a useful thing out of planning and velocity. The problem is, as soon as one of these factors moves, the whole measurement is no good.

Again, on the tasking, I agree with you, but it just doesn’t work. I have tried time and time again to make the tasks be the things that the team needs to build. And this is fine with totally new code and features, but when you are looking at any part of legacy code, you do not know ahead of time what the next task will be. At any given point, I can tell you what the next task is, but I can not tell you what the task after that will be, because it is dependent on what I discover from doing the next task.

In summary, you say all the right things. And I have been on your side of the fence saying all those same things to people and to organizations. And if companies and people were to actually do all those things right and follow Scrum to the letter, they would achieve much better results. There are two fundamental problems though:

1. It won’t happen. If you had even 50% of organizations doing Scrum right, it would live, but I doubt it is even 10%, because I haven’t found one company doing it right yet.
2. Even with “perfect Scrum” there is waste. It is a good way of doing things, but it is not the best way. Because it is a process and not a set of principles, it must be perfect or it will be replaced by something better. Principles like agile are around to stay, processes like Scrum will wax and wane.

Thanks for you comments though. I am glad to hear someone saying the same thing I keep saying, although it is weird to argue against the point I am usually making.

I totally agree that most teams claiming doing Scrum, don’t do it at all.
But this is not the problem of the process.

Furthermore, Scrum is called a framework because you have to adapt it to your needs, there is no way around this. If you get no value out of defining tasks, then don’t do it. Maybe try spiking first and then define tasks.
I agree with you that Scrum is not best for maintaining software. Kanban may be a better choice for this activity.

I just want to say that switching to another agile process without getting rid of the root problems won’t help much. If you (or probably more your management) don’t have an agile mindset, then all these methodologies will sooner or later hit a wall.

Yeah, we have been through the whole process of impediments and talking about really doing Scrum. To me that is not the way to solve the problem. I have been doing Scrum for over 2 years and I have never seen any organization that is successfully doing a true Scrum development model. Sure there are plenty of people with powerpoints claiming to do Scrum, but to be honest most of them are lying. When you take a look at what they are actually doing, they are not doing Scrum.

I am pushing towards Kanban, with the client I am currently consulting for, because that is how they seem to want to run the process. The business wants priorities to be able to shift, and to not have to be time boxed.

What we are currently doing is ending sprints and re-planning to compensate for changes. As a result our capacity is smaller.

You seem to dislike the parts of scrum which are trying to provide plans, commitments and metrics because they are imperfect by nature.

If you can get away without any of that, and just have your team work on a sprint-by-sprint basis, lucky you. You must have very trusting managers/clients with very generous budgets. I know I couldn’t get away without doing those things. At least Scrum provides some useful tools and language for producing plans and metrics, and is easier than most of the alternatives.

I think most Scrum advocates would agree with you that velocity, points-based estimates, prioritisation in the form of an ordered backlog etc are very coarse-grained tools. In fact, I think they’d claim that is a benefit. Plan enough to get something useful, and don’t try to substitute arbitrary precision just because getting high accuracy in planning is impossible.

I absolutely agree with you, though: following Scrum (or indeed any methodology) to the letter is silly. For me, Scrum is a toolbox that, along with other tools, I employ in designing a process for my teams. Scrum is a useful starting point because it’s so simple, but you need to build in additional controls or remove overheads where it makes sense. You also need to ensure you have feedback mechanisms so that you process can evolve over the scope of your project. Like software, we don’t always get the process right the first time.

It is true that I dislike the parts of scrum that are trying to provide plans and commitments and metrics, not because I dislike them those things, but because I don’t think they can actually really work unless you are following Scrum to the letter. In my organization I am one of the biggest advocates of “If we are doing Scrum, we are doing Scrum, not Scrum-but.” Yet, I still find that it is close to impossible to actually do Scrum. That is my point in why I think Scrum will die. I think the idea is good, but the practice is flawed.

You are right about coarse grained tools such as velocity and point based estimates. Yet, I have to ask, if this is the case, then how useful can they really be? In my mind they are now just taking up time which could be used to deliver the product.

I have to actually disagree with you on your last point though. If you are going to follow Scrum, follow Scrum exactly, no deviations. McDonalds is so successful, because they sell a process, not a burger. Good process eliminates debate and extraneous judgement calls. Ken Schwaber calls deviating from Scrum, “Scrum-but”, and he highly discourages it.

Also, I suspect he’s trying to stop people who want to be “Scrum in name only”. Unfortunately, “we’re agile” has sometimes become an excuse for programmers who don’t think they need any process at all. That usually goes very, very wrong. I’m sure Ken Schwaber is worried that “we’re doing Scrum” will become the same excuse. And yes, if you’re doing “Scrum-but”, you’re not doing Scrum, and you shouldn’t necessarily be able to use that label, which is trademarked and certified and all the rest of it. Perhaps that’s your point. 😉

Meanwhile, back in the real world, no one process is a silver bullet. Would you use the same process to build the life support system in a spacecraft as you would a “Web 2.0” social media service that you hope will be an overnight success? Of course not. Aspects like risk, budget, cost sensitivity, time-to-market sensitivity and many others mean that the two demand different processes.

Those are two extremes, but it’s not hard to see that these sorts of differences apply even to projects done within a given organisation. The challenge becomes that you can’t just train your managers on one process (sending them on that two-day course) and expect them to make good choices about how to adapt the process. If you have inexperienced managers, then telling them to follow a process to the letter is probably a good idea, and Scrum is not a bad one to follow. Training them on two or three company standards is probably an even better idea.

Once those managers have a few projects under their belts, I bet you they’ll start tweaking because they know that in the context where they work, those tweaks make sense. Forget about what anyone says about process X vs. process Y. Agile more than anything is about delivering value to the customer, not following some prescribed process at the expense of business value.

So back to Scrum: I find velocity, story-based estimates (the two go hand-in-hand) and per-iteration commitment useful tools. I tell my clients they’re about 20% wrong, but that this averages out. I also tell them the true cost if they want me to bring that error margin (both in estmates and tracking) down to, say, 5%. At that point, they normally agree that it’s better to move ahead with the highest priority items, especially since they’re normally a lot less sure they event want the lower-priority ones. That alone makes those tools useful to me. Highly imperfect, but highly useful.

By the way, one of my favourite books on the subject is Mike Cohn’s Agile Estimation and Planning. That’s not a Scrum book, really, but it covers the same ideas (story points, velocity, etc). I think it makes a very compelling case about why these techniques are useful.

You bring up some excellent points here. Thank you for making me think.

What you have made me realize, is what I am really addressing, which is that Scrum itself applied, preached and promoted to solve every software problem will die.

The points you bring up about different projects thriving better under difference processes is also very correct and an important statement. That is one of the reasons why I am leaning towards a lightweight framework, like Kanban + XP which talks about core values and a very light set of mandated process, then adding custom process and controls to fit the project.

One of the key problems right now is Scrum is being used as the silver bullet. Everyone is certified Scrum this and that. And people are lying saying it is working, when it is not working. The estimation point you bring up is also valid, but that 20% error margin is just usually not worth the trade-off of 10% of the sprint, and all the other time that goes to doing that kind of planning work (which may end being around 20% overhead.) I think this is a fundamental flaw in Scrum, which can not be easily resolved or dismissed. Again, I point to Kanban as a solution to this, although there are others, where the measurement is a natural part of the process not forced or subject to group think. Average time from start of queue to finish is accurate always, and requires no extra effort.

There is too much to be said on process and using hard fast rules to be said here. I want to address this in another post.

I will read the book you recommended. I am interested to take another look at planning and estimation.

Thanks again for all your comments, very much appreciated. You have some very good insights.

I like your post. Not that I agree with most of it – In fact, I disagree with quite a lot of what you’ve said.
But I do agree with the bottom line (or the title, in this case) : SCRUM will die.
However, it is not the death of Waterfall and other rigid or artefact-driven methodologies. These will die because they put little focus on people and a lot of focus on technical deliverables.

SCRUM will die and make way for an evolved framework, one that takes the agile values combined with the practicality and quick-start-liness of SCRUM, compared to other agile frameworks; Be it SCRUM-Ban or something else.

Note, that JIT, aka TPS, aka Toyota Production System, evolved between 1948 and 1975. It took 30 odd years for it to become what it is today, and its principles and values are being adopted in various other frameworks.

As you suggest in your piece, SCRUM per-se might die, but it will evolve into other agile software development frameworks.
Waterfall and the likes, however, are doomed to decline and ultimately disappear, much to the delight of enlightened business and technical people alike.

Great discussions (and perceptions, perhaps!) Here is mine.
My understanding of Scrum is that it is a project delivery methodology (okay, let me suppose we are in the context of software, so lets say software delivery methodology, though It was evolved at somewhere) rather than software “development” methodology. Software development has to be done in its own phases (requirement gathering to deployment) and not any other way. [I dont think we like to switch our focus of discussion to the phases of software development and their need]. Scrum sits over the SDLC as a delivery model. I think ALL that scrum talks of is the “delivery” model. Waterfall and like models focused mainly on the technical aspects of the project rather than “project management and delivery” and I think scrum is doing exactly the opposite, emphasizing ONLY on the “delivery” and never ever talks explicitly about the things that complements scrum. This is where the “assumptions and customizations bud”. Waterfall model did have an inherent delivery model.
I am not against scrum, It is good if everyone follows and to the letter.
As discussed by the author of this blog and many others who agree (or at least understand), Scrum is not a silver bullet. And scrum is not everything. Scrum is no different from iterative and incremental model. It does not state the change in the phases of development. But some “wasted time” is removed indirectly by being different in the fact that the shorter release cycles, regular communication, stake holders involvement is increased so the business “RISKS” are identified earlier etc etc. Scrum does SDLC inherently and underestimates technical aspects of it like the waterfall like models did for delivery. Scrum might die as per some of our understanding, or will evolve. We can’t predict the future but know the direction.

Scrum has benefits: faster delivery, evolved design artifacts, etc.
Scrum says follow simple process(es) and relies on “release early and release often principle which is good. It also says Don’t do anything that is not needed, may be design or a feature or detailed documentation when it is not used by anyone else; this is fantastic

My concern here is Scrum should not be an excuse. We should be using complementing tools/processes. And make the teams following Scrum aware of the fact that it is a delivery model and not a development model. Some other points are already discussed above by others in this post.

We are experiencing exactly the same problems at my company. Much might have to do with the company culture (or lack thereof). But I have the constant feeling that we are fighting to “conform” to Scrum, rather that it helping us be more productive.

Scrum is a highly viable software development methodology, and the teams implementing Scrum seldom experience failure. However, it would be wrong to assume that Scrum is a panacea for all sorts of problems and impediments surfacing during the development process. As a matter of fact, if the team implementing Scrum fails to deliver, Scrum is not to be blamed as a process by itself never goes wrong; instead, the shortcomings and flaws lie in the implementation of that methodology.

After I initially left a comment I appear
to have clicked the -Notify me when new comments are added- checkbox and now each time a comment is
added I get 4 emails with the same comment. Perhaps there is a
way you are able to remove me from that service? Kudos!

Provocative article (and very long discussion! I only took in parts). I’ve written extensively on the importance of understanding the underlying principles of Scrum, rather than focusing on the process. The principles are solid, and when an implementation rests on those—with people understanding them (!) then scrum will be successful. The essence of scrum of course is continual improvement, so to assume it is fixed, or for anyone to claim it is—even its inventer—is folly. How does one improve if one stays still?

By the way, I don’t see Kanban as an alternative to Scrum—neither does its inventor, from what I understand. Kanban in fact can be overlaid on top of any process, to help improve that process, or help it move into a better process, thus it can be overlaid on scrum. I’ve used it that way very successfully. I have also seen people use the “we’ll do kanban and not scrum now” approach as a cop out. Scrum raises issues, shows your organization’s dysfunctions (and your own). People don’t like that. They don’t want to look so they seek a way that they don’t have to. Enter “Kanban”. Be careful with that.

As a software developer, when I came across scrum i said this only works for cheating people. people who finish their work and do not notify management that they finished so they can play chess in the computer before they are assigned another task. for serious developers this is useless. After coming across a manager mocking people in front of the group for making mistakes and mocking people for spending 3 days on something already planned for 4 (four) days. then I will bet by 2016 this will be gone

One advantage of Scrum (and other agile methodologies) is that it makes an hopeless project to fail early and in an obvious way. In that sense, Scrum does not work on such projects. Unfortunately, a good portion of the projects fall into this category.

Sometimes we just cannot afford to allow certain project to fail (think about the budget debate in US Congress or the debates at the United Nation) no matter how bad they. Srum is not applicable in these cases.