Agile Working for Analytics Teams Needs a Cu​lture Change

Today I’m pleased to share a guest post by Paul Laughlin. This post originally appeared on Paul’s site on February 21, 2019.

The term Agile working is being used within more and more businesses. Although loosely defined, it generally refers to a more flexible and pacey way of working. In this series, I share what this means for data, analytics & insight teams who need to work this way.

Those businesses who have invested in formal training will likely be following one of the five most-popular methodologies. Although sounding very professional, in reality the application of Agile to non-IT teams is still in its infancy.

The top-five Agile development methodologies are generally agreed to be:

Scrum

Kanban

Extreme Programming (XP)

Lean Development

Crystal

For more details on the technical pros and cons of each method, a useful overview has been published by Xpandit.

Agile working methodologies were originally developed for use when delivering IT projects. They were a response to slow, bureaucratic projects that all too often failed to deliver using the traditional “waterfall” methodology. With 8 out of 10 IT projects failing to deliver, one can see the need for change.

Agile working in practice
As with most innovations, they have a mixed track record. Agile software development has certainly delivered some significant improvements. The pace of delivery and visibility to business users have improved as a result. However, some large complex projects still benefit from greater consideration and planning when following traditional PRINCE-type methods.

My interest in this topic is the impact Agile working is having on customer insight, analytics, and data science teams. I’m finding many of my clients are working hard to adapt their way of working to these new methods.

Part of their challenge is that adaptation of these IT development methods to business processes is still a “work in progress.” Despite the confidence and eloquence of a growing number of Agile Coaches and Scrum Masters, best practice for business teams is still not proven.

Data and Analytics leaders can feel under pressure to learn a whole raft of new terms and practices. To name only a few these include:

Visible backlog of prioritised work

Tickets for units of work needed

Public boards to track progress

Sprint meetings with bidding to deliver units of work

Standup meetings to discuss progress

Post-Sprint reviews to learn from what worked & what didn’t

Beyond those shifting sands, the other problem I have recognized is that succeeding with Agile working requires a culture change, not just process change. Those teams who succeed have mastered how to collaborate better. In this two-part series, I will share some themes I have seen amongst those teams who achieve this.

Agile working in hearts and minds
The first theme I noticed is a culture that embeds four principles. Each is a new way of working compared to the traditional behaviour seen by data or analytics teams seeking to “cover their bums” when working with business. Together they represent a winning over of hearts and minds to the benefits of a more-collaborative way of working as a business.

Principle 1: Individuals and Interactions
The first principle is a valuing of individuals and interactions over processes and tools. Rather than hiding behind formal steps or documents, Agile working means human interaction. That is the person who is delivering a particular unit of work talking directly to the internal customer who needs it. Together this encourages personal accountability and early transparency.

Such conversations are aligned with the dialogue encouraged in our post on Socratic Questioning. Talking early and often can avoid misconceptions or wasted work. Compared to many traditional projects, it can be a revelation just knowing who to talk to. However, analysts may need to be encouraged to be this visible and supported if things go wrong before you will see sustained progress.

Principle 2: Working (but imperfect) output
The second principle is to prioritise delivering working (but imperfect) output sooner rather than later. This is counter to the traditional approach of using QA check and documentation to ensure output is “up to scratch” before others can see it.

I’ve posted previously on the numerous benefits when analysts embrace imperfection. This is one of those examples. As long as both the business user and the analyst recognise that the output is expected to be imperfect, it helps to see it sooner. Early feedback can help address misunderstandings and bring to life priorities. It can be surprising how much more effective this is than relying on documentation to clarify requirements.

One word of warning: this is not a panacea. Quality and diligence still matter. I have seen cases of analysts delivering slap-dash work under the guise of this principle. If this happens, leaders need to recognise their responsibility to give “in-the-moment feedback.”

Principle 3: Collaborate with your customers
Here I mean both real (end) customers as well as internal stakeholders. Principle 3 is all about collaborating to co-create what is needed. All too often in the past, project leaders have resorted to formal contracts to protect them from unreasonable or ever-changing customer expectations.

This principle turns that conflict on its head. What if both your customers and your insight team felt equally invested in your project? I’ve shared a series on how to run an insight generation workshop. It can be a powerful exercise to invite your customers into your business to innovate with you.

Principle 4: Respond to change when (not if) it happens
The last principle relates to the reality that things change. As William Buist wrote when responding to the challenge of GDPR, external change is a reality for businesses. One they should expect.

In a similar vein, it would be hilarious (if it were not so tragic) how surprised project managers are when things change. Name me the last project you worked on when nothing changed from the original plan? You see my point.

In response to that reality, this principle encourages breaking away from a rigid plan. Expecting change and having an agreed (more flexible) way of embracing change. Utilising the more-regular communication with stakeholders, about smaller units of work, to empower better responses. The ethos should be to agree on chang
es quickly, so as to be able to see the impact and minimise cost or time wasted.

This isn’t a panacea either, and this time business users can exploit this flexibility if left unchallenged. That is one reason why Agile working also requires strong leadership and empowerment of all team members. Any business that still expects analysts to be “order takers” from leaders wanting evidence is not ready for Agile working.

Are you Agile working this way?
I hope those thoughts help any data, analytics, or insight leader who is transitioning to Agile working. Have you seen these principles apply in your business? Do you have any other insights into how to get the best out of Agile working? Please share your thoughts in the comments below.

Next in this series, I will turn to drivers of success. Beyond those culture principles, what drivers distinguish teams who succeed with Agile working from others?

Paul Laughlin, Chief Blogger at CustomerInsightLeader.com, has over 20 years experience of leading teams to generate profit from analysing data. Over the last 12 years he’s created, lead and improved customer insight teams across Lloyds, TSB, Halifax and Scottish Widows. He’s delivered incremental profit of over £10m pa and improved customers’ experiences.