The traditional way to show the hierarchy of a traditional organization is with the CEO at the top and the workgroups/teams at the bottom. The Toyota way (as far as I know) is to reverse it, with the CEO at the bottom. I don’t think either shows the diagram as I’d really like, although the reversed model feels much better to me.

The reason is, the CEO role is a dual role of support and showing direction. In the support role, the role is certainly “at the bottom”, providing support for everyone else in their jobs. In the showing direction role, it is at the tip of the organization.

I have often wondered how I could show that duality effectively, both at the same time. Tonight, as I was going to sleep, an image came to me. I had to get up and do it (and then blog it) so that I don’t forget it.

Here’s the image:

It’s pretty rough picture since I had to do it quickly (still gotta get to bed before dawn breaks). But imagine that like a triangular arrowhead pointed towards better future. Read the diagram on the top and the back.

Certainly, that diagram is useful only in the context of more hierarchical organisations. Organizations like Futurice and Spotify are better described with other approaches.

In those times when I have had more time to work with a team to get them started, I have used the following general outline and found it working. By working I mean, it has struck a good balance between effort invested and value delivered. Investing more time could make things even faster, but not proportionally. Investing less time would reduce value and impact disproportionately. You know.

The Plan for Team’s First Sprints

* Pre-game *

1) Kick-off meeting on first day, to get started (this is more
facilitation than training), including

What are the things we don’t know? How critical are they to know before we start Sprint 1?

Longer term goals

Teach someone (like the SM from the first project) to be able to coach new teams (instead of external coach). Then start building a community of SM’s who know the process and can run the process for their teams.

Interested team members start helping other teams in adopting good engineering practices and team behaviors.

Management learns how to support these Agile teams, and barriers for effective operation are removed as they are found.

To me, there are two things in Agile from which everything else follows. Before moving on, what are those two things in your opinion?

Getting things DONE

The first thing is the potentially shippable product increment, delivered frequently. In Scrum this means that at least at the end of every Sprint, the technical quality of the product meets the criteria of shipping the product (but it doesn’t have to be shipped, for whatever reason, such as not having a shippable feature state). In Kanban flow, this state is reached at the end of every developed item.

This state means, for example:

the product has been properly tested and there are no known bugs

the product has been integrated with its environment

code is clean and refactored

documentation is up to date

If we don’t have that state, it should be the most important thing the team works on achieving. It is generally much more important than adding any new capability into the product.

Because, if we don’t have that state, we don’t have 80% of what makes Agile really work. The team is really doing glorified waterfall. “Almost Agile”, but still not Agile. We also don’t know where the project really is, since there is an undefined amount of work remaining in the features created so far (which is exactly the key problem with waterfall based project management). Essentially, we fool ourselves.

For achieving this in software context, Extreme Programming is a necessary framework (or something very very much like it). Scrum and Kanban are silent on this, but it has always been an assumption that smart software teams are aware how critical it is and use it appropriately. For those who have not experienced XP for real, it is unfortunately too easy to dismiss the importance (because they just don’t understand how much it really changes the name of the game).

Relentless improvement

The second thing is retrospectives – the continuous process of evaluating the way of working (including achieving & improving the technical quality of the product).

Unfortunately, this is often the first thing teams omit in a hurry or when getting less excited about their work. This is a symptom of “work harder” belief gone wrong. Yes, there are moments when working hard (i.e. focusing on getting stuff done) is very important, but it should never replace “working smarter”. And dropping retrospection is exactly that.

All examples of really masterful people or teams exhibit retrospection. They spend time to understand the system and focus on finding behaviors that enable them to get more done at their normal effort. They look at the world with new eyes that allow them to rephrase their problem, thus opening opportunities to new innovative solutions.

Masterful teams understand the importance of potentially shippable outcomes, and continually retrospect any faults that escape their process. They want to understand how to further “error-proof” their work. Of course, it’s a work that will never completely finish or achieve perfection, given that things tend to change and some surprises always happen.

And it is not just the process they retrospect, they also look at other elements of their value delivery. How can we better understand our users real problems? Are we able to come up with innovative solutions? Are we creating a successful business? Do we have the right skills or enough variety in our team? Do we appreciate and support each other enough? Are we communicating effectively in our meetings and daily operation? Are we seeing the reality, or an illusion of some kind? Are our challenges big enough to maintain the drive we have?

And everything else follows

If we follow these two elements, we can distill all other practices and frameworks we need.

We probably need to plan what we seek to achieve in some way (e.g. Sprint Planning and release planning). And we also want to replan daily for surprises (e.g. Daily Scrum) and inspect that outcome with stakeholders (e.g. Sprint Review). And so on. We don’t have to choose Scrum as our framework – we can also choose some other metaphor, like Kanban – but the process does emerge from these needs.

We can also refine our general approach. Is an iterative and incremental approach right for all the things we do? Do we need to assign singular responsibility for certain parts (like prioritizing things to do)?

As a consequence, a masterful Agile team will have a great process, but that process may be unlike anyone elses’ process. And that’s the point – they have not copied their process, but evolved it through countless small improvements and experiments. That’s the heart of Agile.

I am working on an agile project and I don’t know when to incorporate prototyping.

The project is a private web application with a number of complex workflows. We have developed high level user stories, but we would like to validate some of the user stories by testing them with a clickable prototype.

I can approach this a few ways, and I am a little confused. Maybe your advice could help me.

Option 1: I treat the prototype as part of the user story, as part of the iterative process. I build it into the sprint process and validate it during one of my sprints and iterate from there.

Option 2: I conduct the prototype work separately before I incorporate it into a sprint and make it part of the discovery / validation process at the beginning of the project. This is how we handled the prototyping for the example I shared. The design/ux team developed and validated the prototype prior to putting the user story into the sprint.

Option 3: A better solution from Petri? :)

To me, both approaches mentioned are valid approaches (what is an invalid approach? One that never works? Anyway…).

I would use option 1 in these two scenarios:

The amount of uncertainty is manageable by the team without undue disturbance to other agreed items.

All, or most, of team’s work is stories like this. Thus, it is just the natural form of stories for the type or stage of the process, probably an early explorative stage of the product development.

There is great haste to moving the item forward, and it is better to commit to uncertain work (and face the consequences) than to spend time learning first.

I would use option 2 when either:

There is significant uncertainty about the thing to be done, and the team would not feel comfortable committing to the work as a Sprint story.

There are some constraints, like an approval from third party, and that stakeholder is not reliably available for normal sprint work. For example, the appearance has to be approved by Marketing.

Many product programs end up using “two teams” – the development team and the research “team” – that share some of the members. Imagine this as two overlapping circles (which are inside a larger circle – the product). One is focused on developing well-understood stories into Sprint releases, the other is trying to understand what is worth doing. in Scrum terms, the former would be the Development Team, and the latter would be like a Product Owner office (consisting of the PO, some external stakeholders, and some of the team members on a part-time basis).

Both “teams” would have their own flows. The Development Team probably uses something like Scrum, with agreed batches (but they could also use Kanban-like flow), given that the work this team undertakes is probably well understood. The research team is likely to use a Kanban-like flow, given that the work is highly uncertain and there probably are more “balls in the air” at any given time (i.e. there is research work going on in many epics/stories at a given time, and it is not necessarily very clear how long each item will take).

Most Development Team members spend 5-10% of their time in the research team, but some members, like UX designers or domain experts, may spend as much as 50% of their time there. This depends on a given Sprint and will fluctuate as needed. The people must manage their participation in both flows in a responsible way :).

And there is a option 3. It’s not better, just a slightly different approach.

This approach is called “spikes”, “research stories”, or a few other names. It’s kinda like a combination of the two above. Take a research item, write it as a story, and put it into a Sprint. The outcome is a learning outcome, not a product increment. The trick is, these items are not estimated, but timeboxed. In the Sprint Planning, the Team and the PO agree how much time is spent on that item, max (i.e. less time can be spent if a solution is found in that time). When the timebox expires, the team ceases working on that item and reports whatever they learnt in that time in the Sprint Review.

I would use this approach when:

The research items are reasonably few in number (like 0-2 per Sprint)

There is no “PO office” to handle research

Or if there is, but the item is heavily related to the “How”, i.e. technical domain)

The PO or the Team want to learn first (in a low-cost way) either about the What or the How, before being able to refine the story further and commit to actual development work.

The work can be time-boxed and easily fits inside a Sprint (i.e. there is more development work than spikes in a Sprint).

I have noticed that all systems have some natural capability for
productivity, value delivery and quality. As the people in the system
gain experience in the system, their performance will start reaching
that systemic speed. Just like with a ship, this happens quite easily,
but when the “hull speed”* is reached, the amount of effort / power to
go faster dramatically increases, up to the point that a certain speed
seems unsurmountable regardless of power expended. In systems, we can
perceive this e.g. in overtime, which does not yield real benefit
since the extra effort translates to more mistakes and other negative
factors that detract from real progress.

I continually observe that also in the ball point game, where people
psyche themselves to try harder, but they still get the same result.

Also, waterfall has a certain hull speed. No amount of pressure will
make it deliver stuff faster.

Organisations, as they are composed, also have a hull speed.

The only way to go faster than the hull speed is to change the system
into one that has a higher hull speed. E.g. starting to use Scrum. But
any given set of practices also has a hull speed. And the only way to
go faster (after the learning period) is to change the practices to
ones with higher hull speed.

Thus, we can say that any action that does not affect the system’s
structure and fundamental behavior in some way (and merely is an act
to “shape up”, “increase discipline”, or “try harder”) is very
unlikely to produce any lasting effort. Any benefits gained from such
activities either produce negative side effects that ultimately cancel
any positives, or disappear over time as the system returns to its
hull speed.

TL;DR: Work smarter, not harder.

* I know that hull speed is technically a little different from that,
but a layman’s approach is good enough for this concept :).

While I have many stories of release planning, I like to share this one.

Getting started… a little late

I was coaching this game team/organization. The original plan for the next game called for about 6 months of design and development. However, due to other pressures, a lot of things were prioritized over this project and when the first people were able to start the work, only three months were remaining. There was hope that using Agile could help them speed up the development (through prioritization, backlog refinement and faster development), but it was still known that it will be tough.

At the beginning of the work, the game concept was available. A backlog had been built collaboratively with the team. For the first two sprints, only half of the team was available, and they started on the most critical stories. During this time, the backlog was radically refined and prioritized (i.e. features slashed), up to the point that PO started to be concerned that “there was no fun left”.

The Wall

At the beginning of the third Sprint, the rest of the team joined in, and on the third day of the sprint a release planning meeting was held. At this point, the team estimated the size of the items they had already completed (in story points) and used them as a scale for remaining items in the backlog. Based on the stories completed, the estimated total capacity of the team (for the rest of the planned sprints until release) was about 350 points (give and take a bunch). The size of the remaining backlog was estimated at 1300 points.

The lead dev’s estimate of their chances of success went from “optimistic 20%” to “definite 0%”. The sprint was terminated. Everybody scrambled to figure out a new approach with even a slight hope of success. The CEO could not find a feasible solution by extending the release date or otherwise renegotiating the contracts with the publisher. The team looked at their work and tried to figure out simpler ways of reaching the goal.

The New Plan

Luckily, the team’s previous game had a lot of similar elements as this new one (which was kinda considered “2.0” in many respects). However, it was considered severely lacking in technical quality (though it did work). Thus the team had planned to rewrite the core architecture to allow for future extensibility. Upon closer reflection, the team felt that it was possible to return to the previous game and simply modify and extend it to enable the new game design. A new plan was created that would allow them a shot at successful delivery, assuming that they could meet the weekly goals successfully.

Prior to this planning event, the idea had been to practice and implement Scrum in this project. It was decided that there was simply no time to practice, but they decided to continue with the team level elements that they had learned already. Also the new plan did resemble a Scrum flow with its weekly goals and they did review the plan weekly. They also tested a lot each week to ensure it was working sufficiently well. They also ran a beta test alongside the development during the last month.

During the remaining two months and a bit, they actually were able to create a version of the game they were able to ship to publisher and game reviewers. And they also took Steam to use, allowing them to refine/polish the game post-gold since the platform allowed for a forced update upon installation. While it wasn’t the game originally envisioned, it was good enough. And, the team actually did without a similar crunch as they previous releases (but it was still tough 2 months, just much less pressure in the last month and some). So they attributed a lot of the success and reduced stress to the Scrum coaching.

Aftermath

Irony of the story? After the release of the game they decided that they didn’t want to make games like that anymore. So any refactoring of the architecture for future would’ve been a wasted effort. Realizing that they were running into a wall ended up saving huge amounts of effort for them (against an essentially fixed payout for game delivery).

And they did continue using Agile ideas in their future games. They’ve been happy with the increased team collaboration and have continued with the new culture. And the lessons about user stories game them a very good foundation to create increasingly growing game versions instead of “everything at the end”.

If that planning had been any later, it would’ve been a disaster. Any earlier and we might’ve not been able to see it as well. And without the coaching it would’ve been a huge mess for everyone. While this is speculation on my part, the coaching maybe saved the company.

My adventure in San Francisco is coming to a close. While 12 months here isn’t anywhere enough, I couldn’t resist the opportunity to join Futurice as an Organisational ScrumMaster. Don’t take me wrong, I love my job at CollabNet, training people in Scrum and Agile. And I’ve thoroughly enjoyed the opportunity to see the west coast of US close up. But I’m ready for the next challenge.

I pretty much explained the role in this post on CollabNet’s blog site. Though I’m replacing any notion of Team with the whole organisation. CEO Tuomas Syrjänen has been trailblazing this work in his role, and admittedly, being ranked as #1 place to work in Europe is pretty good recognition for a job well done. With me in my role, we can together put more focus and face the future challenges with more support for cultural growth.

With that in mind, one of the first things I did was to start organizing Lean Startup and Management 3.0 courses :). They are now happening in mid-November. If you want to join the courses, check out this blog for more information.

I will still continue running CSM & CSPO courses and small-scale coaching, but again on eastern side of the Atlantic. If interested, you can still contact me at petri.heiramo@gmail.com.

It is surprising how many IT customers (some of which are actually expected to know a lot about software development) decline to pay for good development practices. Be it preventive QA, TDD, test automation or whatever, many customers assume those are additional costs to them (i.e. by not doing them, they would get their software faster and cheaper – and then they demand up-front planning!).

What if you went to a surgery and only wanted to pay for the room and the surgeon. Afer all, it’s the surgeon who makes the cut and fixes you. How would that go? What else is happening in a surgery theater? There are other medical professionals ensuring that whatever the surgeon needs is available (or is quickly retrieved). Some monitor patient’s vital signs and some control the anesthesia. What about the cleaning staff who take care that the hospital is clean? Sure, it’s entirely possible to make surgeries unnecessarily expensive, but there is certain level which has to be maintained (or else the patient pays the difference). I don’t even know, really, and I haven’t been watching those TV series to be better educated. And I don’t have to; I expect them to know what they are doing.

On the other hand, why do we software developers so often ask customer permission for those things? I don’t ask the staff at a restaurant on how they create my food (and instruct them on not using certain types of knives or grill/oven/whatever they choose to use). I stick strictly to my needs and desires (“and the steak – well done, please”), and trust them to know what they are doing. And if the outcome, or it’s cost-effectiveness compared to other restaurants, isn’t to my liking, I probably choose to use some other restaurant next time.

We have to keep in mind that most of our customers don’t have the capability of making an educated decision. They want outcomes that work (they do, even if it really doesn’t sound like that often – and they do complain if the outcomes don’t work), so maybe we should leave them at that. Let them define what they want, we define the how.

Only when they act responsibly, we allow them peek inside. Then again, if they see working outcomes effectively delivered, they are not that interested in what goes inside. After all, it’s the outcomes that make the difference.

Do you instruct the chef in a restaurant?

PS. Funny enough, when it comes to restaurants, there is a kind of expectation that more expensive is better. And often there is a correlation, though I feel that the value/cost ratio comes down. And we choose the place based on the quality (or value ratio) we want. With software, seems that customers want the cheapest. And then complain when it doesn’t taste good.

In this blog post, I’ll share a bit of a conversation I was having with a participant of my CSM course, who had his team ask provoking questions. I’ll share the questions and my responses. Remember that this is a conversation piece, so I can’t guarantee it’s more than opinions :).

There are no single answers, but I’ll try to give some possible perspectives.

> How does Agile make me a better developer?

Well, “Agile” doesn’t :). But I guess that’s twisting words :/. I guess by “Agile” they mean the combination of all three layers – mindset, control framework, and practices & tools. So let’s take Agile as that.

Still, I feel the answer is “no”. It is they themselves who make them better developers, by learning a new consistent framework they can choose to use when they feel it delivers higher value. Yes, there are many people who swear by Agile approach, because it let’s them deliver much higher quality code, deliver it faster, spend much less time debugging, and having fun working with a close-knit group of colleagues towards a meaningful shared goal. Yes, that is available to almost anyone who wants to continually improve themselves. No, it doesn’t come for free – they have to put themselves to learning it.

Also, as a point of view, as the number of people who can write virtually error-free code on first pass and deliver highly maintainable code effectively, that skill level turns from competitive advantage to pre-requisite in the next, say, ten years. Having only traditional skillset, conversely, turns from still competitive (because employers don’t still know much better) to competitive disadvantage. I mean, I would not hire a person who didn’t know how to deliver Agile code, or would not be willing to put themselves to learning it.

> How does this process make me happier in my Job?

The “process” doesn’t, again :). Badly done Agile will make their lives miserable, because it will take the old framework, and add expectations of doing faster and more on top of it.

If the environment is set up to support Agile, that is, there are managers who allow (and support) the team to self-organized, participate actively in removing impediments, assign sufficient authority to a PO who can take ownership of the direction, give the SM sufficient time and support to focus on the role, etc., then the team can focus on learning to do proper Agile at technical level and to learn working together as a cross-functional team.

There usually is a period where it gets worse before it starts getting better. XP requires focus and commitment, and practice. TDD will be difficult at first, especially if there is legacy code which is not supportive of code level testing. Pair programming will feel awkward until skill & knowledge levels start to level a bit. Continuous integration and test automation will require learning new skills and establishing new tools, until they start to show benefit by allowing faster feedback on errors.

I recommend the team, assuming they want to go for XP and Agile, to set themselves a two month period where they focus primarily on the process and learning to do it properly. Delivering features is secondary (though not irrelevant). Speed will come naturally. Make sure that your management understands the team’s need for learning new ways and grant them the time. If all goes well, after two months they will be where they would’ve been without XP, but now with a faster process, better collaboration, more shared knowledge, and, likely, happier people. No guarantees, but that’s what I often see and hear. :)

But if Agile is supported by organization (or benevolently “allowed” and not actively prevented), it does increase the amount of autonomy and mastery in the work. If you have a good PO, it will also increase the purpose of the work. And those are the secret sauce of motivation.

> What Defines “Productive” for a Developer in the Agile way of thinking?

Shortly, the team has a greater capacity to deliver value to PO and customers. This may be numbers (in terms of features), technical quality, or feature value. Ideally, it is a combination of all three.

Also, we should get rid of the idea of individual productivity and look at the team level. Traditional approach is very poor at leveraging collaborative effort – the power of teams. For people who have not been in a real team, they don’t know what they are missing from their professional life. “Aren’t you curious?”

End of the snippety snip part.

As the reader of this post, what elements would add (or modify) to my above comments?

I sometimes struggle between relative and absolute quality, and how they affect the way we work in businesses and the success we gain. That is, I really would like to think we should strive for better outcomes as a kind of value on its own (and merit from that as business success), but I so much more see “being better than competition” as “sufficient” for most people (and frankly, it is good enough for many companies to stay in business).

In practice, the questions boils down to “we’re already better than our competition, why over-invest?” The way I hear that is “let’s cash in on our advantage now, until we lose it”. I also hear “let’s gain in short term, and figure out long-term competitiveness later”. And I can see the logic in that. But it’s the same logic that has effectively crippled so many companies and destroyed their long term profitability.

I recall reading somewhere that the founder of Ikea, Ingvar Kamprad, once said that “the worst thing that could happen to Ikea would be to go public.” He was referring to the tendency of publicly traded companies to focus on short term outcomes and shareholder value over any other business priorities. While I’m no fan of Ikea, I can appreciate the success they have built (and yea, some few of the items they have designed have appealed also to me). And I can appreciate the mindset.

So is focusing on relative quality subscribing to short term thinking (and, as I believe, to sacrificing long term success)? The idealist in me says yes; the pragmatist accepts it as a short term goal as long as it is recognized for what it is.

So what would absolute quality look like in, say, subcontracting business? How would a company striving for absolute quality approach their goal differently from a company that is satisfied being just better than direct competition?

I’ve tried to give this some thought. I’ve so far identified the following dimensions:

Seeking to continually improve their development practices, to deliver error free solutions on the first go and remove all types of waste in the way they develop

Seeking to build their people to be self-driven, capable of taking initiative, and having satisfying professional growth

Looking inside to compare themselves to their past selves, instead of looking out to compare to others

Curiously, the first three loosely match the three characteristics of Radical Management – delighting customers, deep job satisfaction, and relentless improvement. The last one relates to one of the key characteristics of Performing teams – dedication to not accept sub-par performance from oneselves or team mates.

I’ll try to explore each of those dimensions to share what I feel about them. I’ve added some concrete ideas at the end of each dimension.

Excelling in delivering customer value to me is to seek the real value the customer _really_ needs, rather than seeking to deliver what the customer believes they want. It is not saying “well, you don’t know what you want, let me show you” but recognition that we (that is, both the customer and us) don’t have the possibility of defining “maximal” value up front. Only through successive iteration and revision of our understanding we are capable of finding what is really the most valuable bit, and then focusing on delivering that as effectively as it can be done. It is also recognizing that we may often have to challenged establish “truths” to discover new avenues for innovative solutions. The company doing the above is challenging established practices in IT subcontracting industry, and will have to build demand (and appreciation) for their approach. They will not be asked for it, they have to find the way to do it first within the constraints of current business practices.

Implement “Money for Nothing, Change for Free” in all customer contracts, and help customers to actually take advantage of it, even if they didn’t ask for it originally.

Seek to understand what the customer really wants by demonstrating progress continually and asking for feedback, then feed this back to customer’s decision-making processes for action

Teach customers of the benefits of iterative and incremental approaches, what is the value to _them_.

Continually seeking to improve the development practices involves all levels of their operation, starting from the technical practices, to project leadership, organizational management, and internal operation. Internally, they should be challenging themselves continually, seeking impediments to delivery at all levels, and relentlessly seeking to eradicate them. And to understand that this all is normal operation of the organization, and not an “improvement initiative”.

Deploy full-time ScrumMasters to all teams and at different levels of the organization

Use “technical excellence programs” to keep people learning better ways all the time, e.g. through internal coaching

Use metrics that reveal waste and bad quality, and feed the results back directly to teams and individuals

Encourage people to learn outside their typical domain

Build careers around technical excellence, so that we don’t lose more great developers to ranks of bad managers

I believe that people excel only when given sufficient authority to do a great job. There are examples of organizations that have done exactly that, and their people have delivered outstanding outcomes. A necessary condition is access to information. To really drive self-organization into the organization, information must be made easily accessible (and pushed) to everyone in the organization. Doesn’t matter who. And this is a challenge, for sure. Not just overcoming the culture of “information on a need-to-know basis”, but also providing that information in a way that is digestible and creating the skills to using that info for everyone. Only then can we start expecting people to make good system-wide decisions about their work and take real ownership of it.

People are trusted and external post-decision authorization is kept to a minimum

Teams know their P&L, but also that of their neighboring teams and groups at different levels

Progress is made clearly visible to everyone interested, through demos and clear communication approaches

People are held accountable for their decisions and must personally justify them when challenged – people are taught how to evaluate the value of their idea before committing and also recommended to verify those with colleagues

The desire to look within rather than compare to outside is important to maintain momentum. Any company doing that will realize that improvement is never-ending and that there are always ways to get better. But if we compare to outside parties, it’s easy to get satisfied with the results and become complacent. That doesn’t naturally mean that we forget the outside world, but only that our primary competitor is ourselves.

Maintain metrics showing improvement in operation, but try to make sure the metric is open-ended and doesn’t have a “cap”, if possible.

Measure the amount of improvements in the organization and have warning levels when they go too low

To summarize, if I look around at many of the successful companies (e.g. Toyota, Apple…), I see these ideas, at least in many ways, being deployed. Excellence cannot be achieved by being satisfied with good. We should celebrate when we progress, but never satisfy ourselves with it.