Featured in AI, ML & Data Engineering

In this article, author shows how to use big data query and processing language U-SQL on Azure Data Lake Analytics platform. U-SQL combines the concepts and constructs both of SQL and C#. It combines the simplicity and declarative nature of SQL with the programmatic power of C# including rich types and expressions.

Featured in Culture & Methods

The book Agile Leadership in Practice - Applying Management 3.0 by Dominik Maximini is an experience report of the agile transformation journey of NovaTec. Maximini shares his experiences from applying principles and practices from Management 3.0, success stories, failure stories, and learnings from experiments.

Featured in DevOps

Yuri Shkuro presents a methodology that uses data mining to learn the typical behavior of the system from massive amounts of distributed traces, compares it with pathological behavior during outages, and uses complexity reduction and intuitive visualizations to guide the user towards actionable insights about the root cause of the outages.

Mark Levison on the Magic and Science of Teams

Bio

Mark is a Certified Scrum Trainer and Agile Coach with Agile Pain Relief Consulting. He has over twenty years experience in the IT industry, working as a Developer, Manager, Technical Lead, Architect and Consultant. He also publishes a blog: Notes from a Tool User. When not in front of a computer, Mark spends his time with his wife and two daughters.

About the conference

Every day, Agile practitioners and thought leaders are defining and refining new and existing practices that bring Agile’s values and principles to our world of work, play and service. Agile2014 reinforces our understanding of proven methods and illuminates some of the exciting new innovations that represent the future of Agile.

I have the pleasure and the joy of walking into organizations and helping them figure out how to be effective in an agile way and very often that involves telling people that what they are doing at the moment isn’t working well for them and to help them figure out what would work and that often involves “moving the cheese”.

There is always going to be magic involved in this, but too often what happens is we have people who have a success with one team and then they have a success with another team, and they think they’ve discovered a magical formula on what it takes to build high performance teams. And, absolutely there is some magic involved, but it’s very important to understand some of the fundamental and basic science that’s out there now, understand what’s going on. So, in the past few years we’ve got an intersection. There’s been years of research in basic studies of teams in organizations, we’ve got some behavioral psychology, now we’ve got neuroscience starting to kick in and very recently we’ve had people gathering large data sets. So we’ve got Larry Maccherone over at Rally gathering data, we’ve got Ben Waber and Alex “Sandy” Pentland, both formerly MIT, deploying badges on people and measuring people’s behavior. So, we have lots of different ways of gathering data now and it’s starting to stabilize and tell us some interesting things. I think what’s important is to actually discuss what we know from science so that we understand where to put our time and our energy and we save wasting time on putting the energy into the wrong stuff.

It turns out one of the most important things you can do, and this shouldn’t be shocking for most people who have encountered agile before, you need two things. You need dedicated teams - so team members who are only working on one team. What’s often misunderstood about that, is that when you are working on only one team, it doesn’t mean your team can’t work on multiple products. Obviously when your team works on multiple products in the same sprint or the same cycle there is going to be an consequent loss in efficiency and effectiveness, but if that helps the organization move forward, so be it. So that’s one thing, have people dedicated to one team. Larry’s data suggests that that’s actually worth a doubling in performance right there, just by itself. In terms of magic, that is almost magical if I can double your performance simply by dedicating your staff to one team, even your specialists.

The second one that comes up that is honored sadly in the breach more often than it is in the obeyance, is stable teams. A stable team is one where the people are actually on the team, and on the team full time for a reasonable period of time. So I gather from Larry’s data again that team membership changes often people roll on and off every three to four months. Unfortunately the evidence we have suggests that you would actually like to keep your teams together for years, usually multiples of years, in fact until what we call stagnation occurs, so we’re talking three or four years in a lot of cases before you see stagnation. So that’s what we are not seeing honored in obeyance in a lot of organizations today. There is a little bit more evidence behind that, a little bit of explanation that might help people from the world of sports. I don’t remember the exact dates because I am not a sports fanatic, but what I think there was a period of time when the Miami Heat laid off all their staff, which is not something unheard of in the software world, sadly, and they kept only one gentleman around, Chris Bosh. And having kept Mr. Bosh around, they hired some other stars in to support him.

The next year they didn’t do very well at the start. They were a new team, not unlike a new scrum or agile team, so surprise, surprise, their performance was fairly bad. Slowly but surely over the course of the year their performance climbed. By the time the playoffs hit, they come up against the number one and the number two teams and they beat them both. So you would think at that stage that there are going to walk in the finals and it’s going to be a walkover, they should destroy the Dallas Maverick, the numbers are clearly in their favor. Yet at the end of that season, the Dallas Mavericks win it in the finals. The next year - with largely the same group of people, so now an intact stable team, they’ve been together for one year - they go on, they win the NBA finals. So you would think if anybody has the advantage over us in the software world, it would be a sports team, with all the coaching, all the support, all the individual skills development, all the stuff that might make the magic happen on its own, and even they needed time to gel. Time to know what you can rely on. Time to know that you didn’t have to look back to see a pass coming. So that gives you an example of why stable teams are so important.

Shane: This is stuff that we’ve known about in management science for a while, see Tuckman’s pattern for team formations, for instance.

Right. And even though we’ve known about it for years and years, we’re still seeing that teams don’t do it. So, what I’m liking at the moment is that we are getting more and more data that’s really just hammering it home. The advantage of both Larry’s data and also the stuff from the MIT people - they call it sociometric data for the badges they use - the advantage of their stuff is that it really hammers home with cold hard numbers of data gathered either from physical devices in the sociometric stuff, or from the behavior of people using the Rally tool, it gives people cold, hard data on what the outcomes were for these groups of people. It’s much harder to argue against cold, hard data than it is arguing about “fluffy stuff” as too many people complain it is.

Shane: Another topic you mentioned is cohesion.

Cohesion is understanding the bonds between people; I’ve got a couple of examples for us here. In the first example, I have a team that is very cohesive. Most people in the team have relationships with most other people in the teams; there are only two people in this picture that don’t have relationships with other team members. I actually draw these diagrams for a lot of teams - I’ll talk in a few minutes if it helps in how we might draw such diagrams. This team is not so cohesive. In this team the only person for whom all information travels is the controller or the task assigner or perhaps the manager in the center. It’s not that the manager is a bad person, but they are the only person that all other team members talk to, so in fact if you look at this team, you don’t have one team, you have three distinct sub-networks. That’s not a team, that’s a loose working group. As Waber commented so ably in the book People Analytics, when you have a group of people working on something that is nominally a product who are not collaborating, you don’t actually have a group of people, you have a flock. They’re nice, they wave at each other, they get along well when they see each other in the hallway, then they sit back down and work in their silos again. That’s what we are trying to avoid, we are trying to take this network and make it more like this network, that’s our goal.

First of all, you actually have to measure cohesion, you have to understand what it represents. So, cohesion is just the bonds between human beings. You can go all fancy and measure, you can bring in the sociometric people, and they’ll get your people to sign waivers and attach badges to them and then the badges have Wi-Fi receivers which tell where you are in the building and all sorts of other sensors to tell where you are standing at any moment in time and who you’re talking to. And once we get over the creepiness aspect of it, and it is really quite creepy if you think about it, it gives them a tool to measure who is talking to who. I do something a little less sophisticated. I didn’t even realize why I was doing it a number of years ago when I started. All I do is ask the team’s permission to record some data, and I show quite upfront what I’m going to do, and I share with them the results of what I learn from it. I record on any given day who talks to who and who doesn’t talk to who. When you are with a colocated team it’s dead simple: sit somewhere in the center of the space, observe. When you are with a non-colocated team, watch the IM interaction, the messaging system or whatever you use to chat.

Now, appreciate that email doesn’t actually count towards interactions, email may be necessary, email may be required, but email doesn’t actually count towards the quality of your interactions. So that’s how you measure the cohesion, then you sit down and you draw some trivial graph. You can either draw the very lightweight version I drew a few minutes ago, or if you want to get really fancy and sophisticated, this person talks to people only every couple of days, they have effectively no real relationship. This person talks to somebody else once or twice a day, it’s a very light weight relationship. This person is in constant communication with one of their peers, they have a very strong relationship. So you can draw anything, you can draw as simple or as fancy as you want, all I ask is that you are open with your team about what you are doing, so they know what’s going on. Don’t worry about the Hawthrone effect - a lot of people worry what will happen if I observe the team, will they change their behavior. Here’s the neat thing, for the first couple of hours, yes, they change, and then they forget you’re doing it, and then you get real data. So, now we can measure our cohesion. Now we have to figure out what we want to do with it.

Well, so what? The first thing we have to appreciate is that we need cohesion. Cohesive teams are what get more work done. Individuals working in silos on their own aren’t where the magic is, so we need to value the cohesion. And once we appreciate that building those relationships between people is actually what matters, now we have to start doing it. One of the interesting things that comes out of Waber’s research is that a lot of what matters among your team members is not the discussion at their desks, and it’s not the discussion around the formal work that we are doing - that’s incredibly important and that does contribute towards cohesion - but as big a contributor towards the cohesion is actually developing social relationships with your peers. Talking about their vacations, talking about their kids, talking about their families, talking about what’s going well in their lives and maybe also what’s not going so well.

So it turns out it’s incredibly important to value a couple of things that we often value in the agile space, but don’t really understand why. You need to care about where the water cooler is, or the coffee or whatever your preferred soft drink or beverage is, and it also helps to create long lunch tables and create social environments for lunch. So you have to think very carefully where is the water cooler - is it near the team so it’s easy for them to go get water or soft drinks or whatever it is that they want, or is it far away in which case they only go once or twice a day, that’s the first thing. You want to create an opportunity for people to have frequent interactions with said water cooler. The other funny thing you have to be careful of is where you place it, is it too close to the group?

I was at a client a few months back and I wasn’t in a position to make a comment, but I was at a client a few months back and I saw a sign hanging in the kitchen area and the sign said “please be quiet.” “Please be quiet, don’t disturb the people working here,” failing to recognize that the people having the conversations at the water cooler were in fact conducting quite valuable work. First because they are conducting your formal work, the work you acknowledge, but second because they were also building and maintaining those social bonds that make it possible to continue working together. The other thing that we care about there is the lunch tables. The lunch tables turn out to be interesting because one of the risks with a cohesive team is that it may lack diversity of thought. If you are too used to getting all your great ideas from the people next to you, you are at high risk that they may be your only source of ideas. Classic examples of that, as a Canadian, the one that makes me cry is RIM, or Blackberry, who have injured themselves because they focused for a long time on all devices need keyboards, all devices have to have better batteries, people care about faster and faster hardware. Unfortunately, there was a shift in the market a few years ago. For better or for worse people have decided that touchscreens matter, people have decided that the software matters as much as the hardware; it’s not that the hardware didn’t matter anymore, but the software mattered now too. For a long time Blackberry chose to close its eyes to that, made a series of decisions that say “no, we know what we are doing”. They lacked diversity of thought.

Now, this particular point wouldn’t have solved their problem, but long lunch tables invite people to sit with your team when you are having lunch, from outside of your team, allowing you to bring new ideas from still within the organization but outside your team. So have long lunch tables and even better, invite outsiders into your building. Not necessarily outside consultants, just friends from outside to come and chat with you from time to time to make sure you are getting new ideas into the building on a regular basis.

Shane: So, creating those extended social networks.

Right, try to build networks outside of the organization. So build cohesion within the organization but make sure you are building ideas from outside the organization as well.

That becomes a great deal harder. When you are thinking about this and when you are measuring this, a bit of data that comes from Pentland and Waber is quite interesting: distance actually matters a great deal. The closer people work together in where they sat, the more often they talk to each other face to face, they still talk over email, but they also use face to face conversation. The further apart you are, the less you talk face to face, the more you rely on email. The surprising discovery for me was that if you have somebody sitting one floor apart from the rest of the team, it’s almost as bad as if they were in a different city. When we do have teams in different cities, you can become highly productive, you merely follow along the road, it’s not that you can’t get there, but the road it’s going to take to get there is going to be a lot longer. You have to set up the moral equivalent of the water cooler. Perhaps that’s an IM channel where you encourage people to actually talk about their vacations and their social life. You encourage people to actually acknowledge every time there are stepping out for an hour or for an appointment or something else or to go check out the kids’ recital.

When they come back it’s actually important for their peers to query them and find out how was the kids’ recital, how was your appointment, is there something you need to share, do you need to share anything, so that your peers have an understanding of what’s going on in your world. One of the benefits we didn’t touch on before for the cohesive networks is when something goes wrong in a cohesive network, we can heal the wounds because people are aware that you are down, that you need support, maybe you’re having a bad day and you need support just to make it through a bad day personally. Maybe for some other reason you are going through a bad period of time and the team need to offer you the opportunity to take the time off. When you are distributed or dispersed you are not going to gain those social signals as easily so you have to invest much more in working on those social signals. You probably also have to invest in people who have a better typing speed than I do. I’m afraid the IM generation has passed me by, I struggle to type fast enough to keep up with IM, it’s probably going to be a massive learning for me the day I actually have to deal with that.

Shane: Distances become really large when we start talking time zones. How do we tackle this, and it’s not uncommon to have people that have 10 or 12 hours off set.

There are no great patterns for that. That distance will harm you, there is no getting around that. Now, if it is going to harm you, there are some patterns we’re aware of that minimize the damage and the pain. The first and the simplest pattern is create teams in each location. So create a proper scrum or whatever flavor of team you are creating in location A, often North America, and create a proper scrum team in India which is all too often where our wonderful work gets outsourced to. Create proper teams in both locations. And I don’t mean a dev team and a test team, I mean teams capable of getting to done in the standard agile sense of the word, capable of delivering software independently. It probably also helps - although I don’t have any formal evidence on this, this is now more intuitive evidence - it also helps to a degree to use practices like acceptance test driven development, to make sure you have a common understanding of the story or the feature you have agreed to build. The people in each location will still define their own acceptance tests, but that way the acceptance tests can be reviewed by people on the corresponding location to make sure they understand what we are trying to build. The other thing that clearly does matter, and there is some literature to back this up, you need to fly people to be together at the start of time to build those relationships and you need to fly them together from time to time thereafter. The pattern I am recommending to my own clients now is probably two weeks at the beginning of the project to really step back and get the time to build the relationships and then probably a week every x number of months, probably six, seven, eight months, you probably need another week or so of time together. And this isn’t just flying a project manager from one location to another, let’s be clear, that doesn’t build relationships, you need the whole team and you need them there so they can see each other.

Once you’ve done that you’ve created a wellspring of trust. I imagine trust is a lake and we’ve created this lake of trust. Unfortunately day to day work, when somebody misses a small commitment for somebody else, a little bit of trust is drained out of that lake, so we have to make real efforts on a regular basis to ask ourselves how are we restoring that lake. The idea behind the additional week from time to time, every six, seven, eight months, that’s an additional time to refill that lake, all the stuff that’s been drained out. We also know that avoiding email and going direct to video conferences where you can, where time works helps, and patterns that I’ve heard a few people talk about at this conference help too. Have somebody who is appointed to take on the pain of being available on off hours, to participate in the daily scrum of your team in the different time zone. And vice versa, somebody in the foreign time zone pays attention to say the North America time zone and is available in off hours. As long as you are not exactly 12 hours apart, that can work well. Even 12 hours it can work well, but you are going to have to interrupt your evening and your family dinner, to be available for somebody in India, to be available for your North American team, and vice versa. None of these patterns are perfect, all of them are going to slow you down, all will cost money.

Shane: Mark, thank you very much for talking the time to talk to InfoQ today, it’s been really good to catch up.

Fantastic. Thank you so much for your time, Shane.

Certified Scrum Trainer Mark Levison offers insight into the neuroscience of teams with five proven Agile methods to create teams that sizzle not fizzle. Get the FREE download here.

Community comments

Miami Heat

Your message is awaiting moderation. Thank you for participating in the discussion.

It was the 2011 season where they lost against the Dallas Mavericks.However, the 2012 season only had like 30 games... vs the usual 60 or so, because of the lock out... maybe thats why they were able to win in 2012!