There are perhaps as many ways to run a retrospective as there are teams to conduct them. I want to describe my favorite way, especially because it's an approach that has stood the test of time, having worked for years with many, many teams.

The Start, Stop and Continue Retrospective

I like to conduct a sprint retrospective by asking team members what they would start, stop and continue doing. This type of meeting becomes known as a “start, stop and continue” meeting.

The start items are things a team member thinks the team should add to its process. Some examples would be:

Showing the software to customers early

Specifying acceptance tests early and with customers

Doing code inspections

Being on time for daily standups

Finishing one story before starting the next

Items on the stop list are things that someone on the team thinks are inefficient or are wasting time. The team should stop doing these. Examples from past retrospectives include:

The continue list contains items the team would like to continue to emphasize but that are not yet habits. So any of the start or stop items above could go onto the continue list and stay there for a few sprints.

Eventually--once the item became a habit--it would be removed from the continue list. Otherwise, the continue would become tremendously long.

Ask for Items in Different Ways

A ScrumMaster can ask team members for items in different ways. The easiest is just to say, “Yell them out,” and team members are free to intersperse start items with stops and continues. This is my default mode.

But, it can get repetitious sprint after sprint. So, I’ll mix things up and sometimes I’ll go around the room asking each person to give me one item, perhaps making two passes around the room before opening it up for additional items.

Other times, I’ll want to emphasize a specific type of item--often the stops. So I’ll ask all team members to yell out nothing but things to stop doing. Or, I’ll combine approaches and go person-by-person around the room asking each to identify one thing to stop in the team’s current process.

There are plenty of ways to mix up the idea generation in a start-stop-continue retrospective so that it will take a long time before it gets boring or repetitious.

Vote

After enough ideas have been generated, have team members vote for the most important item or items. It’s often obvious when it’s time to do this because the creativity has died down and new ideas are not coming very quickly.

The ScrumMaster can have each team member vote for the one, most important idea or can use any typical multi-voting approach. For example, give each team member three votes to allocate as they wish (including all three votes to the same items).

I like multi-voting in a retrospective. The nature of most retrospective items is that many do not really take time to do it. Many are more behavioral. Consider being on time for daily standups from the examples above. That doesn’t take any time. In fact, perhaps it saves time.

Multi-voting would allow a team to choose to work on that behavior and perhaps another couple of items. Generally, I’d pick no more than three. Even if they don’t take any (or much) time, choosing too many items does detract from the importance of those selected.

In addition to voting for new items to pursue, discuss whether items on the continue list have been achieved, are no longer important or should be otherwise removed from the list.

The Next Retrospective

In the next retrospective, I suggest the ScrumMaster bring the list of ideas generated at the previous retrospective--both the ideas chosen to be worked on and those not. These can help jump-start discussion for the next retrospective.

I tend to write them on a large sheet of paper and tape it to the wall without any fanfare or discussion. The items are just there if the team needs them or wants to refer to them. I then facilitate a new start, stop, continue discussion.

Benefits of Start, Stop and Continue

I find that conducting retrospectives this way is fast, easy, non-threatening and it works. A start, stop and continue meeting is very action-oriented. No time is spent focused on feelings. We don’t ask team members how they felt during a sprint; were they happy or sad, warm or fuzzy.

Each item generated will lead directly to a change in behavior. The team will start doing something, or they will stop doing something, or they will continue doing something until it becomes a habit.

Yes, I’m prepared for many people to leave comments saying it’s important to work through people's’ feelings first. Or that we won’t know how to act until we’ve first dealt with how people feel. Go ahead. In some cases that may be true. But in plenty of other cases, we can identify what to do (“we need to start testing sooner”) directly.

And that’s the strength of a start, stop, continue approach to sprint retrospectives.

What Do You Think?

What do you think? How do you like to run retrospectives? And specifically, is there anything you’d like to start, stop or continue about your own retrospectives?

Would you like to include comments?

Tagged:

About the Author

As the founder of Mountain Goat Software, Mike Cohn specializes in helping companies adopt and improve their use of agile processes and techniques to build extremely high-performance teams. He is the author of User Stories Applied for Agile Software Development, Agile Estimating and Planning, and Succeeding with Agile. Mike is a founding member of the Agile Alliance and Scrum Alliance. He is also the founder of FrontRowAgile.com, an online agile training website. He can be reached at [email protected] or connect with Mike on Google+.

I’m glad you and I agree on inviting the product owner. I suggest putting no more stock in the Scrum Guide than in any other book on Scrum. In particular, Ken Rubin’s “Essential Scrum” is the best introductory book on Scrum.

Posted by Mike Cohn on 2016-12-02 12:04:22

I was going through your weekly tip on "The Product Owner Should Attend the Retrospective" and that reminded me of this blog. Even in Scrum Guide it talks about Scrum Team and not Dev team when it comes to sprint retro. I invite PO and I find its beneficial for both PO and team. They get to understand each other and many improvement ideas come up.

Posted by Yuvraj on 2016-12-02 00:37:00

You’re welcome. It’s an unfortunate thing to do but sometimes necessary.

Posted by Mike Cohn on 2016-10-22 10:23:34

Thank you Mike, Excellent work around with "Post Sprint Debrief". Yes, I agree with you that I need to cancel the official retro. Because I even put the note in the invite "Limited participation from Scrum Team and Agile office" but that even did not help. Last option is cancel official retro. Totally agree with you and thank you.

Posted by 2pkothari on 2016-10-22 10:19:25

I’d tell the outsiders they can’t attend. Tell them you will notify them (with the team’s approval) of issues needing their attention. If they refuse, organize a team lunch every last day of the sprint and do the retrospective that way with local team. Do the same with the remote team whenever you can visit them. Occasionally get the whole team on the phone for a “Post Sprint Debrief”. Then as needed run the official retro with nothing discussed or tell those outsiders you’re flat out canceling retros and do those “Post Sprint Debriefs” instead.

Posted by Mike Cohn on 2016-10-22 10:03:30

I know scrum team needs right environment to conduct the retro due to sensitivity on the topics. I have other Project Lead role and other stakeholders attending this meeting who are not close to the Scrum Team. Major part of my team is offshore and team is not participating in the retro being completely silent but tech leads from onsite team are being silent too. One reason i think is due to the presence of other leadership folks. What would be your guidance to resolve this issue?

Posted by 2pkothari on 2016-10-22 09:02:11

Thank you for your reply!

Posted by André Mauritz on 2016-10-19 05:30:12

I understand, André. I fully understand that many people want to start a retrospective with a discussion of how we felt during a sprint. I do understand the value. But my background is as a pretty hardcore programmer. And I don’t think I’m the only programmer who doesn’t want to do that. (There are plenty of other software development jobs that can feel the same. I just relate to it as a programmer.) If I go into a meeting of any sort and someone says, “Let’s talk about feelings,” I’m done. I’m going to disengage from whatever follows. I know that isn’t optimally and may not even be healthy. But I also know I’m not the only one who feels that way.

This is very different from not valuing or respecting the feelings of others. As for “people want to talk about their feelings.” We must have met very different people. I largely disagree with that premise. Some people do, some don’t. I wouldn’t guess as to percentages but I can’t even say which side of 50/50 it would be.

Thanks for your comment. I do appreciate it.

Posted by Mike Cohn on 2016-10-18 10:14:48

Hey, Mike!

Great article - as usual, thank you! I'm also a big fan of focusing on fast outcome (or even more: impact), especially in all kinds of Scrum meetings. I do like the idea of talking with the team about their feelings in Retrospective for other reasons. But people want to talk about their feelings. I like addressing peoples feelings actively in Retrospectives to help team members open up and know that they can - and that their feelings are respected and valued by other team members. For me it's not part of my process to get action items faster - but to have dedicated time to talk about peoples insides and strengthen the bond between team. And also see, what really is important them.

Regards,André

Posted by André Mauritz on 2016-10-18 09:33:50

I think it’s fine to bring up big issues any time. I wouldn’t bother with small issues mid-sprint though. There’s a value to looking at a set of improvement opportunities all at once so the team can better select which they want to address.

Posted by Mike Cohn on 2016-05-03 17:26:43

Currently working with a new team member who conducted a mid-sprint retrospective - on a specific item (release) - comments? opinions?

Posted by Laurie Duncan on 2016-05-03 15:20:14

Very interesting, Ben. I’ll have to try that some time.

Posted by Mike Cohn on 2016-04-15 08:58:37

I use a similar retrospective exercise with teams. I've called it Start Stop Spice-up or SSS. Just continue is often not enough, I challenge teams to looks for ways to get even more value from things that they are doing well. And to think about ways to ensure that they will keep on doing them.

This is the basic retro I often use when I start with teams along with the what went well, what do we want to improve, my only real difference is the vote, I like to give 3 votes but you can only spend a maximum of 3 on any single issue, I noticed it stops someone holding back and trying to load a story with all 3 of their votes because its really important to them and not so important to others, with a max of 2 per issue it tends to show the issue the team as a whole think is important, of course they could put all 3 up in singles as well

Posted by Tom on 2016-04-08 07:30:07

I like those phases but don’t necessarily have an activity for things like set the stage. I keep retrospectives much shorter than advocated in that book.

Posted by Mike Cohn on 2016-04-01 14:45:10

Do you take the approach suggested in the book "Agile Retrospectives" of splitting the meeting in 5 phases (set stage, gather data, generate insights, decide what to do, close retrospective) with an activity for each of the phases?

Posted by kkrac on 2016-04-01 14:04:53

Great idea. In fact, we're already on the same page. In the long run, we plan on adding a performance/metrics layer on top of Retrium to track your progress over time. Measuring action items against other team health indicators would be a great addition to that plan. I'd love to chat more with you about this. Want to email me? david __ at __ retrium __ dot __ com. Looking forward to it

Posted by David Horowitz on 2016-03-22 07:19:28

I really do like the Start-Stop-Continue approach, because it visualizes progress and makes it transparent and can even be used to be communicated to management or stakeholders.I have seen a really, really great example of that in @mdubakov 's article https://www.targetprocess.com/...What he does is track the Start-Stop-Continue items across years and combine them with performance metrics such as the most simple one: user registrations or even sales.It would be awesome if Retrium could do exactly that - provide me a sprint time-line with started/stopped/continued methods that I can use to combine it with some other indicators for "team health", performance or whatever you might call it.

Posted by danielkun on 2016-03-22 05:16:42

Hi Tina,

Anything with lightweight screen sharing should work fine. So even just a Google Doc would be enough.

Posted by Mike Cohn on 2016-03-15 18:08:21

Hi Mike, what is your best recommendation for running retros with remote people? I've tried the stop/start/continue where we write on the board and then send a picture to the remote person and also tried to just do an GlobalMeet meeting where I just take notes in a Word doc.

Posted by Tina Martin on 2016-03-15 16:53:47

Thanks, Sarah. I’ll check it out.

Posted by Mike Cohn on 2016-03-14 09:49:09

I really enjoy using retromat (http://plans-for-retrospective... for different retrospective ideas. My favorite has been #tweetmysprint (#97) - it got the entire team excited and was a new way to generate ideas from the same start/stop/continue and what we did well/what we can improve, etc. that we have been doing.

#tweetmysprint (#97)

Produce the team's twitter timeline for the iteration

Source: Thomas Guest

Ask participants to write 3 or more tweets on sticky notes about the iteration they've just completed. Tweets could be on the iteration as a whole, on individual stories, a rant, or shameless self-promotion - as long as they are brief. Hash tags, emoticons, attached pictures, @usernames are all welcome. Allow ten minutes to write the tweets, then arrange them in a timeline and discuss themes, trends etc. Now invite participants to favorite, retweet and write replies to the tweets, again following up with discussion.

I know that what theory says is sometimes difficult to achieve and in this case it is much easier to just go straight to find a solution to the problem. The average team member doesn't like to express his/her feelings and feels much more comfortable when just talking about those problem in an indirect way. Because of this and for the sake of time and productivity it is normally better to skip the feelings and face directly the solutions. But what I like the most about scrum is that it takes people into account and introduce psychology in the team. In my opinion scrum should be focused on improving team members also in a psychological manner and skipping talking about feelings is easier but doesn't help. If you go to the psychiatrist you will must talk about feelings, so why not in scrum?

Posted by Imanol Bernabeu on 2016-03-04 02:45:25

I occasionally pass around a box and have each person contribute a card with a suggestion written on it. I ask people to write “nothing” even if they have nothing to suggest (which shouldn’t happen often). This prevents it from being obvious who made the the one suggestion if only one person contributed something.

Generally, though, I really want to coach a team to being sufficiently comfortable with each other that they can say these things to one another. I’m an absolute introvert, so I can relate to the difficulty of that. But it’s a better place for a team to get to.

Posted by Mike Cohn on 2016-02-28 12:10:39

Sounds like a nice idea, I am gonna try this out with my team in the next retrospective.

I realized, that letting the team write down their comments before starting the discussion is better, because then also more introverted team members have a voice, as their item will be picked up and discussed together. Before we did that, only the more extroverted people were discussing, while the introverted ones didn´t say a word during the whole retrospective.

This is where I think the "Team Room" concept is a great idea - having all the things the team needs in one space, keeping it constantly visible to them and having no need to go off to other meeting rooms for certain ceremonies. As with many things in Scrum, you're trying to surface the information and learn from it.

@Özmen - the team I inherited when I joined the company used the Confluence pages that are designed for writing up retrospectives and they were really proficient at doing it...but then they'd forget about it and never actually do anything with the actions they had created.

We now just have big ass post-its with the action from the retro (we try and just choose one thing and make sure it gets done, rather than multiple things - this also helps us get a bit more cross over within the team, devs and testers working in each others areas to help achieve their action.

Using Confluence isn't a bad way to record the actions, but you need to visualise them another way too - I've had other teams in the past where by we created tickets in our sprint backlog for the retro actions we were going to tackle and we linked them back to the confluence page, this made the reviewing of previous actions really easy..so there are pro's and con's. The main thing is, if it's stored somewhere out of site and in the digital realm then it's much easier to forget about it - do you ever experience any of these shortcomings?

Posted by mothergoose85 on 2016-02-19 08:18:28

Good suggestions, James. I like biscuits so I'll attend!

Posted by Mike Cohn on 2016-02-15 08:20:17

*hard even :)

Posted by James Buckley Barrett on 2016-02-15 07:47:27

Retro’s are head to keep interesting. I find biscuits help :). Also I'm planning on asking a different member of team to act as host - allow someone else to bring new ideas and ways to the retro. Start/Stop/Continue is also the simplest method for new teams.

Posted by James Buckley Barrett on 2016-02-15 07:46:19

Thank you Mike! Appreciate it!

Posted by Andriy on 2016-02-03 20:24:26

Thanks, Andriy. I definitely would not let a lot of old items accumulate. But if you want to bring the list of items from the prior retrospective in to seed the discussion, go ahead. I often do. But, I just hang it on the wall. If items are important enough that we want to do them, I have team members yell them out again.

The way to think about it is this: You are generating a list of the candidate items to improve. You’re only going to do a handful of them. This is not like a product backlog where it’s everything you’ll ever want to get better at. So don’t bother adding an item that you know is valuable but you aren’t going to do for 6 months.

Posted by Mike Cohn on 2016-02-03 20:13:46

Hi Mike,Great post, thank you! I was wondering what approach do you take on "Start doing" lists from the previous sprints. You mentioned that you have list from previous sprint on the board so team can see it, how about previous sprints? What happens to me is that we would come up with the list of items (around 5-8 let's say), then we would vote on 1-2 items to focus for the following sprint. Problem is that this way we are not keeping up with the "Start Doing" list and it keeps accumulating. Should that list be maintained (contain all the items from the previous sprints) or it's better to keep last 1-2 sprints of items and then drop the oldest ones?Thank you!

Posted by Andriy on 2016-02-03 12:46:52

Thanks, Hanna. I agree with much of what you say. But I’d be careful about having one person (presumably a ScrumMaster?) who decides about things.

Posted by Mike Cohn on 2016-01-30 07:54:05

Thanks for the article, interesting point of view. 'Yes, I’m prepared for many people to leave comments saying it’s important to work through people's’ feelings first.' - I would turn things round and say: feeling are important but they're still just feelings. They may come from the fact that someone began the day too late, that had to hurry up, that's angry with someone. And one may take all these as a base for general disagreement and what may come after it... https://pl.pinterest.com/pin/5... And 'stop' doesn't evolve into 'continue'. That's why there's always one person, meeting leader that decides about things in the end. The one who should notice the feelings but often be above them.

Posted by Hanna on 2016-01-30 05:03:21

I’ve never tried “What tripped us up” exactly worded like that. I may try that wording. Thanks, Arjay.

Posted by Mike Cohn on 2016-01-28 14:55:57

I like the way this method flows and coaxes team members to take ownership of their own improvement. I'll add it to the rotation for "shaking things up." The core method that I use with my teams is to track in a shared document (google docs is one option) three groups of retrospection: A) What went well? B) What tripped us up? and C) What are we going to act on in our next sprint? Every member of the team can enter items in the sheet. As we go through A, we dissect each one and list any behavior or action that we want to incorporate as a team standard or continue to do in the C area. We then go through "what tripped us up? (B) and dissect the items to determine if there are any actions we can take to prevent them or behaviors we can change; we list those in C. Finally we review the list in C and decide, as a group, what 2-3 items we will take on and incorporate during our next sprint. From your post above, I think I'll modify the process by incorporating the multi-voting as well as listing those "actionables" with the most votes on a piece of butcher paper in the team area.

Posted by Arjay Hinek on 2016-01-28 12:54:01

Thanks for this article, in our company mostly use sprint development model, which very effective.

Posted by softwaretestingbooks on 2016-01-28 12:48:45

Thanks Mike, hope you like it. Let me know what you think.

Posted by Martin Wickman on 2016-01-28 01:36:10

I’ll have to check out your book, too, Martin. It looks good.

Posted by Mike Cohn on 2016-01-27 18:56:27

The less/more/stop/start activities works well in the beginning. Especially with new teams. But... it gets boring pretty quick. After a few retros, people get into a rut and it just doesn't give you anything fresh. The thing is, you need to change stuff around (activities at least) to keep your team on their toes. I second Mike's book recommendation, but you may also be interested in my book, released some week ago, http://amzn.com/B01AK3NHBA, about running agile retrospectives.

We have been using the Start, Stop and Continue format of Retrospectives over a year now and what I have noticed, is that members, continuously thank each other for what they did during the Sprint in the 'Continue' section and often do not have anything to say or add to, at all, in the 'Start' or 'Stop' sections. It seems that these Retrospectives may have become boring, as this has been continuing for almost a year now.

As there are other Formats to run the Retrospectives, is there a second best or recommended format that you would suggest? Or could you please suggest on how the current format (Start, Continue, Stop) can be made fun and interesting to the Team?

Posted by Temsila on 2016-01-27 14:23:04

Hi Matt—

I like having the team own the process and I do the same with Post-its. I know this is somewhat my personal problem but I cannot stand talking about what made me happy or sad. I don’t care. All I care is what we’re going to do about it. I know the theory that we’re supposed to think about what makes me happy or sad first and then then think of actions, but I’ve always found it a completely needless step. I just prefer to jump right to the actions of start, stop and continue. But, maybe you’re right that with some teams new to Scrum, this can be the right way to start.

Posted by Mike Cohn on 2016-01-27 08:36:47

Thanks for sharing this, Özmen. I’ll admit that many times I forget or don’t bother to actively review the previous items. But I always like having them on the wall so we at least see them.

Posted by Mike Cohn on 2016-01-27 08:24:50

I’m not familiar with that and don’t have time to comment on review other sites or products. There are many ways to run retrospectives. This post was merely describing my preferred approach as a simple way to do them and get good results. My preferred source for alternatives is still “Agile Retrospectives” by Derby and Larson.

Posted by Mike Cohn on 2016-01-27 08:23:03

With teams new to Scrum, I like using a close variant to the start|stop|continue method where the three headings are:

* Things that made us happy (happy face)* Things that made us sad (sad face)* Ideas for next time (lightbulb)

Eventually you have to shake it up a little unless you like settling into a steady state of "really enjoyed the teamwork" and "felt like we got a lot done" coming up every single retrospective, but to get teams into the retrospective spirit I really like it as a format. It reinforces the feeling of owning the process - the idea that good productive development is rewarding, but blockers and enforced crunches make us unhappy.

The other thing I like is to put a big pile of Post-It notes in the middle of the table and let the team spend a few minutes writing down their ideas in each of the happy|sad|lightbulb categories. As each person puts their ideas on the board you can group them up, see common themes and also outliers. I still run voting afterward - sometimes one single outlier will be the most popular item to address!

Finally, I like to encourage teams to agree their retrospective plan, print it, and put it on the wall near their work area, ideally on their sprint board. That way it can serve as a reminder of what needs to be done to make things better. We then use this to start off the next retrospective, pretty much the same as you're doing.

Posted by Matt Kimber on 2016-01-27 03:33:34

Hi Mike,After agile transformation, we have facilitated our retrospectives the way our tools suggested. We have created retrospective documents automatically, it has suggested us to write down "What should we have done better?" and "What did we do well?". Those questions are great but there were two main problems with our retrospectives:1. Someone, often Scrum Master, sitting in front of a PC, writing what others tell. Sometimes one of team members did not talk during entire retrospective or one of us dominates the whole conversation. That was passive and not a collaborative way to let the team to improve each other.2. "What should we have done better?" question is great but does not drive people to think deeper and we did not have enough items we expected.

So, we started to use "Start|Stop|Continue" method. We draw three lines and expect people to fill those lines with post-its they wrote on. This is time-boxed to 10 minutes. After all, everybody told their ideas standing next to board loudly, so everybody can hear clearly. After declaration of ideas, we group ideas into groups such as "teamwork", "code quality", "customer collaboration" etc. Afterwards, everybody puts down their three votes to any of groups. Than, it's time to talk about action items for top-three voted groups.

This way, everybody has a voice, no one dominates and we got closer think as a team.

We are skeptical about starting with previous retrospective items. Because sometimes, people are too much concentrated previous items and can not be creative. But it's a good idea to remember how much we achieve since last retrospective.

Thank you for writing a step-by-step guide for this method and the reasons why it works.

Posted by Özmen Adıbelli on 2016-01-27 01:56:46

Hey Mike, have you heard of this website: http://plans-for-retrospective...What do you think about something like that instead of just talking about start, stop and continue?

Posted by Roz on 2016-01-27 00:43:51

Great! Thanks

Posted by Mike Cohn on 2016-01-26 21:29:58

You’re welcome, Brian. I’m glad you find it helpful.

Posted by Mike Cohn on 2016-01-26 21:08:17

Thanks! I haven’t seen that movie yet. It just moved up on my “To See” list.

Posted by Mike Cohn on 2016-01-26 20:35:55

Very wise words! I relate what your saying to a quote or line in the movie Iron Lady/Margaret Thatcher, which I nearly always swear by: "don't ask me how I feel, ask me what I 'think'. If we do that then we can always act in a better way and naturally everyone will feel better because of it.

Posted by Christopher Pola on 2016-01-26 15:19:51

@Mike Cohn Thanks for the continuing useful information, it is very valuable to me as a newly appointed Product Owner just trying to get my footing in this Agile environment.

Posted by Brian E. Jones on 2016-01-26 15:18:52

Just in time! Really needed a new idea, and this is something worth trying. Thank you!

Posted by Cheryl Magnuson on 2016-01-26 13:37:56

We have been using Start, Stop, Continue as part of our 'Big Room' planning efforts. We pass out post it's for the team and have them put them on the flip chart. In the past we have taken away two items to work on in the release, but have not re-used the board we created. I will keep a copy of the board this time and break it out next to see how relevant it is in the next release.

Thanks for the tips Mike they are greatly appreciated!

Posted by MrDoug Inc on 2016-01-26 13:31:41

Thanks

Posted by Mike Cohn on 2016-01-26 12:55:30

Thanks Mike Cohn! Love the blog and your advice. Very helpful for all of us.

Posted by David Horowitz on 2016-01-26 12:51:44

Thanks, David. I just talked about Retrium in CSM class today.

Posted by Mike Cohn on 2016-01-26 12:43:28

Thanks for this Mike! Couldn't agree more on the importance of using a facilitation technique to make your retrospectives engaging and effective. In fact we wrote an article on this (https://retrium.com/resources/... that helps describe why you should be varying your technique from sprint to sprint.

In that spirit, I'd also highlight a few other techniques that we've found useful: Mad Sad Glad (if you want your team to focus on its emotional state), 4Ls (Liked, Learned, Lacked, & Longed For), and Lean Coffee (a structure-less, but facilitated technique).

All of these are available in one-click on Retrium (http://www.retrium.com), which can be very useful if you have a distributed team. (Full disclosure: I'm a co-founder of Retrium, but it really is the best way of running a distributed retro!) Hope that helps!