Developing a SaaS product? Scrum is NOT for you.

As I had mentioned in a previous post, I believe that software-as-a-service vendors must adopt an agile development and release methodology to release software on a very frequent basis.

Interestingly, we at WebCollage have been recently hiring engineers to our Development Center in Tel Aviv (here’s a short promotion to our careers page). Perhaps not surprisingly, many of the people we interview have touched the Scrum development methodology in some way or form. Given the ubiquity of Scrum, it’s surprising how little discussion there is on why Scrum misses the mark when it comes to developing SaaS applications or web applications in general.

Like many software organizations, we had used Scrum in the past, and here’s why we feel it’s not the right tool for SaaS vendors.

Scrum in a Nutshell

In a nutshell, Scrum is a project management methodology which bundles several concepts and tools:
(i) iterative development (a sequence of iterations, each delivering value); (ii) a timeboxing approach (planning iterations with a fixed duration rather than varying their duration based on the deliverables); (iii) a planning, collaboration and role framework (more below); and (iv) reporting and tracking tools.

Iterative development and timeboxing are now somewhat prevalent outside of Scrum, so these days Scrum boils down to its planning and collaboration framework. In essence, Scrum advocates building teams of all-around developers. These developers conduct a short planning session at the beginning of each iteration with a “product owner”, and then self-manage the development and testing towards a committed goal at the end of the iteration. Inaccuracies in planning and other people issues are handled by the (all-around) software developers filling in for each other. Scrum further ensures that no new requests come in during an iteration, which helps align the committed scope with the committed timeline.

Is Scrum adequate for the construction industry?

There’s nothing in Scrum that makes it truly unique to software development. So, one may consider whether Scrum can be used, for example, in the construction industry. Based on the Scrum approach, a construction team would sit down at the beginning of the monthly iteration with, say, prospective apartment residents, and listen to how they plan to utilize the apartment (called stories in the Scrum vernacular). Then, they would commit to a certain scope (say: build one floor out of two). Then, they would work as a team to complete the elements required for that floor: flooring, windows, plumbing, carpentry, and so on.

If the Scrum approach is feasible, one may ask why industries other than software haven’t adopted similar methodologies. After all, they’ve been around for a while, and had plenty of time to create optimized practices.

Some of the reason a Scrum-like approach isn’t used in non-software domains relates to the the “soft” aspect of software. Specifically, it is much easier to change the first page of a web application than to change the first floor of a 20-story building.

But another fundamental reason—more specific to Scrum—is that it is unrealistic to expect a construction team to be truly cross-functional. The plumber is unlikely to be good at carpentry, and vice versa. (This isn’t universally true, but I doubt that Scrum fans would want a plumber to, say, install carpentry in their own apartment.) As a consequence, if all team members were to tackle a project together as a team, inter-dependencies (for example, windows cannot be installed until flooring is complete) would create huge inefficiencies because team members would have to wait fo other team members to complete. Human issues (sick leaves and such) would amplify such issues even more.

As importantly, dependence on external factors would further reduce the effectiveness of the team. For example, it’ll be quite hard to install windows until the actual windows arrive at the construction location; if the windows are late, progress cannot be made. And if ensuring that windows arrive on time is a core project task (versus a technicality that can be addressed by the so-called Scrum Master), external factors need to be considered as part of the planning.

Let’s put aside the construction industry, and move back to software-as-a-service (SaaS).

Why is scrum inadequate for SaaS?

In reality, the same two issues I outlined above apply to development of software-as-a-service applications, and—practically—to web applications in general.

Development of modern applications that face external customers is inherently made up of specialized team members. At a minimum, one would expect the development cycle to include some form of application design (some combination of user flow, schematic user interface, or actual graphical user interface and visual design), some form of writing (be it technical writing or friendly assistance write-up) and—I would want to hope—some form of user-oriented QA (at a minimum, testing for usability, friendliness and “smoothness” of the application; a task that cannot normally be carried out by the application developers).

What’s interesting in each of the above areas is that (i) it cannot always be carried out in parallel to core coding; and (ii) it cannot always be carried out by the same people. Minor variations in the application design and/or the visual design (e.g., a need for a new UI widget) can significantly affect scope and effort. So, the application design should really be nailed down ahead of the development iteration (as should the floor design ahead of an apartment construction). Customer-centric writing usually requires a different skill set than coders normally possess (though I’ve seen Scrum fans advocate using writers for QA tasks; I would again challenge such fans to apply this belief when they build their next house…). And, user-oriented QA can only commence once something is built.

While people have been proposing ways to adapt Scrum to address integration of specialized team members, these approaches often result in far-fetched variations. For example, some Scrum fans advocate conducting a separate pre-development Scrum for the application design team. If your company is successful doing this, I’d appreciate an invite. I have been curious to attend a daily standup (a scrum collaboration tool), which constitutes of exactly one person, as is often the case in application design. Other Scrum variations use the Scrum “brand”, but deviate so much from the original (simple and elegant) design of Scrum, that they truly formulate a different paradigm.

As importantly, a fundamental Scrum requirement is that development iterations are fixed in scope, and cannot be altered once started. In my mind, this misses the point when it comes to true iterative delivery of customer-facing applications. Suppose that you’ve just released Iteration N to your customer base and now started working on iteration N+1. Most likely, real input from customers would not be received immediately upon release. Instead, it would come in piecemeal throughout the subsequent few weeks. By the time the feedback is in, you’ve already started iteration N+1, and can only consider the requests, changes and feedback as part of the backlog for iteration N+2. So, in reality, you can only react to customer feedback (or, as I heard happens in other software companies (;-), resolve issues spotted by customers) in two iterations, which may take as much as two months. The diagram below tries to capture this limitation visually.

Shortening iteration durations helps alleviate part of this issue, but the fundamental issue is that Scrum rigidly offers no way to address time-sensitive market feedback.

Put differently, the SaaS reality is that external factors do weigh in during an iteration, and cannot simply be ignored or put aside for another iteration.

Is there a conclusion?

Well, for one thing, this note does not argue that Scrum is inadequate in general. Not only has Scrum played an important role in popularizing agile development, I believe it’s a viable tool for several classes of software applications (an obvious one being internal applications). However, Scrum is too far off the mark when it comes to modern customer-facing web applications, and other methodologies need to be called in.

In one of the next posts, I’ll try to outline some thoughts on how to better handle agile development and incremental releases in a SaaS (or: web application) environment. The key, though, is to acknowledge the real-life situation. When you consider the construction industry analogy I’ve used earlier on, it’s easy to spot that there’s a reason why not all construction team members work as part of a single construction project at the same time. Cross-dependencies and external factors do need to be factored in when devising one’s product release and product development methodology.

12 responses to “Developing a SaaS product? Scrum is NOT for you.”

All this might have been nice and all.
but to the fact that I’ve seen several Saas Developer using Scrum with great success.
It might not be inadequate but still it works.

Seriously though the limitations you present here are really far fetched:
1) In many organization marketing would love only a 1 month delay.
2) while not everyone in a team can do everything, pulling all the skills into a single team is feasible and the benefits for doing outweighs the costs to achieve that. Also (unlike construction) the skills you specified are much more “close” to one another. specifically I saw in most places that programmers take part in the technical writing, they definitely can help in doing QA and in most part are also pretty good designers.sure you would like to have an expert in each field handle the tough parts, but everyone can help.

Real question is: if you think scrum is not good for SaaS, what in your opinion is better?

Undoubtedly Scrum *can* work in SaaS companies. In fact, projects *can* succeed and in fact have been documented to succeed using most development methodologies (waterfall, Scrum, FDD, XP). What’s more, I’ve seen projects succeed with various ad-hoc methodologies too (chaos included ;-)). As things turn out, elements other than the development methodology play a much larger role in overall project success. Three that come to mind are clear market need, a clear articulate market proxy, and the quality of the staff; I am sure there are more. As for a development methodology, given my belief that continuous software delivery is critical for SaaS, I would pick Scrum over any non-agile approach in a heartbeat.

That said, I would beg to differ on the wisdom of using developers for non-development tasks: certainly not for artwork (design) and writing; and, to a lesser degree, not for high level application design and user-facing QA. Hence, the limitation in using Scrum as true methodology (versus for an ad-hoc project).

In tasks that are outside core programming, developers can sure help, but as a matter of real-life situation, I doubt that developers have played a noticeable role in the design or write-up at any of the known SaaS vendors (services like Basecamp, ZenDesk, FreshBooks, Box.net, Constant Contact, to name a few). There are simply too many elements that make these a separate profession. Ignoring this profession makes the end result sub-optimal where it really matters, in the customer’s eyes (considerations such as “voice”, “tone” and “style”; or, “brand”, “visual language”, “grids” and “proportions”, to name just a few, come to mind). While I’ve seen developers who can scale to carry out such tasks, this has been a singularity within real-life team and again does not truly meet the requirement for cross-functional teams.

As for your last question, I am definitely planning to cover some thoughts on methodologies that provide a better fit to the world of user-facing modern web applications. (Good or bad, there’s also a day job to deal with :-)).

In my experience organizations that try to respond to customer feedback outside of iteration boundaries risk falling into madness. Whatever the cadence (once a month, every two weeks), iterations are a good balance between “cowboy release management” and “wait til next year”. The key is to guide the customer with a good roadmap that adjusts to feedback without losing its spine. Pure reactiveness doesn’t benefit the customer or the vendor.

It’s a fair point, and I would certainly agree that the main drawback of Scrum is its reliance on know-it-all team members, rather than its insistence on fixed duration and scope for an iteration.

That said, even in non-SaaS environments, team members are approached with time-sensitive tasks that fall outside the scope of their Scrum commitment. A few examples that come to mind: helping peers, responding to customer issues (level N support), researching a potential future direction, interviewing, etc. In a SaaS environment, the interrupts are bigger because as developers are (and should be!) also called in to help with production issues (at ones that involve the application), as well as (in reality) with ad-hoc requests (e.g., a time-sensitive marketing campaign relying on certain pieces of data that require programming to extract) or questions (if a new customer did THIS and then THAT, would that REALLY work).

Realistically, some production issues simply cannot wait for several iterations as they are too critical. To me, it seems that accepting interruptions breaks the “holy scope, holy timeline” paradigm introduced by Scrum and calls for a different approach. But, perhaps there’s a way to address such issues within the Scrum approach?

I Believe in Agile & Scrum, the reason i believe in it is not because i think it is good, I’ve heard it is good or heard it is good. The reason i believe in it is that i have seen, in real life in a variety of environments, cultures and technologies that it can help. A lot.
All that said, i would like to address a few of the things you have written which i find fundamentally wrong:
1. “Scrum advocates building teams of all-around developers” – Scrum does not even try to define how to reach the goal of building an increment of software at the end of each iteration, it simply states that you should do it. How? This is a question left to be answered by the organization adopting Scrum. Indeed having Generalizing specialists helps in some cases, but it definitely does not cover all of the situations.
2.” one may ask why industries other than software haven’t adopted similar methodologies” – Again a wrong assumption, a lot of industries have adopted this approach, including hospitals, education and retailers. There are also a lot of who have not, since scrum was tailored for software, and while it fits some other domains, it is not suitable for all.
3. “dependence on external factors would further reduce the effectiveness of the team” – IMO this argument is valid in almost any scenario and any domain, i don’t see the added value in this points.
4. “Why is scrum inadequate for SaaS?” – Reading through this paragraph i could replace the word SaaS with every type of application that has a user interface and is updated frequently, which i believe is a very big precentage of the s/w written these days. So your claim is that Scrum is not fit for software, and that is a legitimate opinion, however when you claim that one, You will have a lot of real life data to “deal with”.
5. It seems that there are some fundamental error regarding what is a “Cross functional” team and the expectations from adopting Scrum. I highly recommend reading the Scrum Guide published by Scrum.org http://www.scrum.org/storage/scrumguides/Scrum%20Guide%20-%202011.pdf

To summarize (and sorry for the long comment): IMO Scrum is fit to SaaS, it just does not even remotely tries to solve all of the inherent problems that are a part of any s/w project, this is up to the organization to experiment and learn, Inspect and Adapt.
Expecting any process to solve the project’s problems – That is the real problem.

Undoubtedly agile is a must for SaaS companies, not an option. See my earlier post on this topic in case this one came across as not promoting agile in principle.

>> “Scrum does not even try to define how to reach the goal of building an increment of software at the end of each iteration”

Scrum doesn’t define the types of teams, but it does strongly suggest that a team must be self-contained to accomplish its goals. If some of the tasks require certain skill sets, then either these must exist within the team (which creates imbalance in the case of non-development tasks) or they must be outsourced (which creates a “broken” Scrum environment). Either way, if Scrum is Intrinsically awkward or clumsy in certain constellations (as I argued above) (even if one is to say that Scrum is indifferent to the situation), I would certainly view this as a Scrum limitaiton, and a severe one in this case.

>> “a lot of industries have adopted this approach, including hospitals, education and retailers”

My point was a little bit different. Many industries adopted Scrum as a *software development* practice. But I am unaware of hospitals with daily standup meetings of doctors around handling patients (and, personally, I would probably stay away from a hospital organized in sprints of cross-functional team members, where any member of the hospital can complete any type of surgery ;-)). Or, a school teachers’ daily standup meeting around next education tasks (again, not sure this is the school I would pick for my kids). I’d love to hear more, but I am not aware of Scrum-like concepts used to advance projects that are not around building software.

>> “dependence on external factors would further reduce the effectiveness of the team argument is valid in almost any scenario and any domain”

External factors are always a challenge. But, when you dissect Scrum, the key driver is *really* the team’s *commitment* to a *closed set of goals*. The (valid) assumption is that when a team commits to a set of goals that they are in control over, they will do above and beyond to achieve them. Hence, Scrum is truly fragile when this one assumption (“closed set of goals”) is broken, which happens when external factors come in. Indeed, having external factors is ubiquitous, but the problem it raises is unique (or at least amplified) with Scrum’s dogamtism around keeping both timeline and content of an iteration unchanged (see also below).

>> “Reading through this paragraph i could replace the word SaaS with every type of application that has a user interface and is updated frequently”

Indeed, Scrum has (advantages and) disavantages for all types of applications. However, its weaknesses are larger with SaaS (and indeed other end user-facing web applications) environments. End user facing web applications inherently require more visual design, writing (and for that matter, usre flow planning and testing) than (say) internally facing applications. These are areas that fall outside the core programming team. Further, end user facing web applications are (or should be) *released to the market* in frequent iterations (not just developed in internally-facing iterations, like some other software) hence there’s an inherent need to address “feedback” (issues, change requests, enhancement requests), which is again a weak area of Scrum. This is especially true in a SaaS environment because business customers will not always take “no” for another. And lastly, SaaS companies (unlike on-premise software) constantly deal with needs that relate to them providing a service, not just selling software: production issues, unexpected operational requests, data analysis and other non-core-programming tasks (even Nth-level support, for that matter). All those issues indeed apply in some capacity to numerous organizations, but their *dominance* in SaaS companies makes Scrum much less useful, and in certain ways, harmful.

>> “there are some fundamental error regarding what is a ‘Cross functional’ team and the expectations”

I am happy to learn more; if you can point out the specific errors you believe are in the post, I am happy to address those. On the surface, the docuent seems pretty consistent with the book I mentioned in the post body.

—

To summarize: I think that Scrum fans (and non-fans) should acknowledge that — like any other practice or methodology — Scrum has advantages and disadvantages, and consequently “sweet spots” where it shines, and (well) “sour spots” where it is less of a good fit. The idea that one methodology, let alone one as simplistic as Scrum, would be a panacea for all projects is implausible.

My points above essentially boil down to the assertion that the less programming-centric the projects are (i.e., require more than coders), the less Scrum is helpful (or aims to be useful, for that matter); and, the noisier the environment is (i.e., unexpected/unplanned tasks that truly need to be addressed), the less Scrum is helpful again (or aims to be useful, for that matter, unless one considered cancelled sprints a desirable practice). I don’t believe that the inceptors of Scrum would disagree with the above statements. And my viewpoint is that in such a constellation, which is, I believe, typical to SaaS companies, there are simply better approaches within the agile framework. I will do my best to cover those thoughts shortly :-)

We’ve used Scrum for many projects, including SaaS development, and non-technical implementations, and we’ve had great success. To make the argument that Scrum is not adequate for SaaS, and use the given examples, shows that either the author is not really well-versed in Scrum, or that the Scrum process implemented within their company exposed some of the flaws within the organization. Scrum works if the everyone involved understands their responsibilities, and that specially includes the product owner.

Thanks for the comment. I think the argument is not materially different than the one previously posted. I think the logic you suggested is something along the lines of:
1. We (H Lee and team) use Scrum adequately in many/all projects
2. We (H Lee and team) are successful in many/all projects
3. Therefore, Scrum results in or ensures success in many/all projects
4. Therefore, suggesting that Scrum has drawbacks or isn’t a fit for all (e.g., this post) is senseless
5. Therefore, anyone suggesting that Scrum has drawbacks (e.g., the author) is also senseless
Personally, I’m not sure I find the above argument convincing (for one thing, if one replaces “Scrum” above with “Religion X” or “Cult Y”, one comes across as a missionary ;-)), but this may be a matter of personal taste.

Eilon, you hit a nail on the head: Unfortunately, people often have cultic devotion to methodologies, and such zealots often accuse anyone who says the cult may not be all things to all people to be a heretic. For some reason I don’t profess to understand, Scrum seems to be more cultic for many.

I am also a certified Scrum Master, and know from experience that Scrum, like many methodologies, is not one size fit all. There is no substitute for thinking out of the box for your specific environment, being realistic about skill sets, time, quality, and all tasks that must be accomplished (not just the subset that fits nicely in the cultic methodology box), and architecting the most expedient process based on an established methodology, a hybrid of established methodologies, or rolling your own if this best meets needs of your projects, company and customers.

I think that’s where people go wrong with Scrum though roryroybal; Scrum is not a methodology but merely a framework for managing a process.

It’s actually a great framework to have in mind and use as an idea to develop, alongside other methodology, framework and processes, what each company should invariably try to do: develop their own, best-suited Agile Way of Working.