I’d appreciate your advice on working for the GDS and different ministries. I have recently applied for senior positions (Deputy Director and Head of Product) at GDS and another UK government department.

On both occasions I was told that while I had the skills, my previous products had not been used by the millions of users, rather the hundreds of thousands, and therefore I was not considered the right person for the role.

This seems unusual given that building for 6 figure as oppose to 7 figure usage would not make that much difference in my opinion.

Could you recommend anything I could do to improve my suitability for next time a role like that comes available?

Many thanks,

D

Read on for my reply.

Hi D,

I can’t comment on the specific reasons why you’d have been rejected for the roles you applied for as I wasn’t part of that particular process.

However, I did a fair amount of interviewing both at MOJ Digital and GDS while I was head of product at each, so a bit of feedback based on my reading of your background from your LinkedIn profile and comments.

First off, I would suggest that you’re aiming for too senior positions given your background and relevant experience. Deputy Director is one of the most senior and politically-charged positions in a government department or agency, so you would really need several years of board-level experience as a director under your belt before being considered seriously.

Secondly, as an interviewer, I would suggest that there is, in fact, a significant difference in building for millions (and tens of millions) versus hundreds of thousands of users, and it would indicate a lack of experience to me when making such an assertion.

You have to remember that in the public sector, you don’t get to pick and choose the ‘best’ users to work with, you are obliged to ensure your products and services work for everyone, regardless of their ability or background. This makes creating a single service to meet very varied user needs particularly challenging.

There are also the more technical challenges of ensuring the service is robust, performs well and is secure at scale – which often means engineering for a significant factor of load above what would be considered normal usage. This could be because of unexpected spikes in interest from the public (the register to vote service before the last general election was a good example), or malicious DDoS attacks.

Either way, it’s not necessarily as straightforward as you seem to believe. My advice to you would be to apply for either senior or lead product manager roles. The remit and complexity of these roles within GDS would probably fit better with your experience. That’s speaking as someone who wrote the job specifications at GDS for both :-)

I need a product manager with specific experience in machine learning and AI, can you help?

Yours,

C

Hi C,

If you’re looking for a product manager to join your team, I’d like to offer some advice:

Good product managers are agnostic of technologies (and markets); they’re more bothered about what user/customer problem they’re trying to solve. Only then do they determine (with their team) what technology will solve that problem most effectively.

If the product you’re building incorporates machine learning and AI, then it shouldn’t make any difference whether your product manager has direct experience in those technologies or not. They may be fashionable at the moment, but in reality they’re already commodity tool sets to solve a particular tech problem, like a development framework or relational database.

After all, teams can start using both AI and machine learning from AWS at relatively low cost without having to develop their own.

If the product you’re building is itself a new machine learning or AI component, then fair enough, I can understand why you’d want a product manager with that specific experience.

The problem you have by constraining your search in that manner is that there are thousands of great product managers out there, but relatively few have the tech experience you’ll be looking for (as is the case whenever specific tech or market experience is required). More likely you’ll find some mediocre product managers, but with the relevant experience.

Most product managers come to a new product, market and set of technologies every time they change job. I work on a freelance basis and see different technologies used pretty much every single time, many of which I’ve never encountered before – but it doesn’t stop me doing my job well as a product manager.

All I’m really suggesting is that you consider looking for a good product manager first, and if you can find one with relevant tech experience, even better, but the relevant experience shouldn’t be a deal-breaker.

I’ll not lie: my eyebrow involuntarily rose an inch or two. I wasn’t aware such a product development methodology had reached the legal profession.

Office musical chairs

She clarified: ‘agile’ is the new business buzzword for ‘hot-desking’, which itself was the updated term for ‘office musical chairs’. In other words, there are more people than desks at her office, so each morning everyone needs to grab the nearest available desk, while HR hope that several people are not coming in that day.

I. Will. Get. A. Desk. Today.

Organisations have been roundly abusing language for years, seemingly to project an air of intelligence and authority, sometimes to deliberately obfuscate unpalatable news. For nearly 25 years, Lucy Kellaway has been writing a column in the Financial Times that points out the worst offenders. Among the gems she observes: Uber describing itself as being in a “reputational deficit” – its name is mud.

The abuse of Agile

“Agile” is another term that has been misappropriated and abused by organisations to such an extent that its original meaning is all but forgotten.

Agile means “able to move quickly and easily”, whether applied to physical or intellectual endeavours. When used to describe the Agile Manifesto, the intent was to describe a new way of developing software that was similarly nimble. That ability to ship software rapidly and easily was a direct riposte to the document- and process-heavy death march that software development had become.

Later, people (probably user researchers and product managers) realised that it wasn’t enough to simply be able to change the direction of software development nimbly, it was probably also a good idea to jump in the right direction. Having hard evidence in the form of user research and analytics helped to highlight the right direction to move in. Gathering that information as frequently as possible then helped to keep those course corrections small.

However, organisations immediately began to confuse this sense of agile with the processes, frameworks and methodologies that had also labelled themselves as “agile” by the consultancies and training companies who wanted the agile dollar.

Every organisation I’ve worked with for the last decade has claimed to be agile. About one in ten actually understood what that meant. About half as many again actually worked with agility.

The acid test

Whenever someone proudly tells me that their organisation is agile, I simply ask them when they last spoke to a real user of their products, and when they last changed direction in response to what they’d learned from their users. Any less recently than “in the last week” (“in the last 24 hours” would be better) and they’re probably fooling themselves.

“Agile” has become meaningless in software development because it’s used to describe more organisations that patently are not working with agility, than those that are.

Working with agility

Simply put: working with agility means you can change direction quickly and easily in response to new things you’ve learnt, and that you learn from your users as frequently as possible.

Everything else is irrelevant. Including whether you have a desk at work or not.

Agile with rigour is apparently a thing and, from what I could determine when I encountered it first-hand, is complete and utter bilge water. ↩

I’ve been happily looking after my product (buying and selling cars online) for the last couple of years. I still think product management is a complete mystery to me, but my boss thinks I’m doing fine. In fact, at the end of this week she will be announcing internally that I will be product managing a second product, which will put me across the entire post-login customer journey!

I am freaking out about how on earth I’m going to do this. Any words of advice?

Yours,

K

Dear K,

Advice, advice… yes – here’s a couple of thoughts:

The difficult bit’s never the actual product (even if you have several of them). Instead it’s making sure you know enough about your customers to be able to decide what you should be doing with your products to meet their needs, and about your own business to make sure you’re aligned with the corporate goals.

Balancing the needs of these two groups of people is the trickier bit, but get that right and you’ll definitely be on the right track.

It’s good that your products sit nicely together. When you go out and listen to and learn from your customers (ideally with as many of your product development team as you can manage), everything you learn will inform both products, so you’ll kill two birds with one stone.

It’s even more important now to observe your customers trying to sell and buy cars (and whatever else your company can help them with). You want as many opportunities as possible to think “hmm – that’s interesting, why do they do that?”. As a rule of thumb, you should be spending a minimum of two hours every six weeks face-to-face with customers.

It’s all about the people, then about the products.

Time management will be important – learn to delegate (or just stop doing) stuff that you don’t personally need to do. Working on user stories with your team, rather than trying to write them all yourself, is a must. As a product manager, your objective is not to be the busiest person, but the most effective, so be ruthless in picking what to spend your valuable time on.

Getting sucked into too many meetings? Get people to have their discussion without you, then recommend to you (with evidence) their proposed course of action for your decision. If you can get to the point where your team can resolve their own problems without your input (and in a way that you’d approve of), then so much the better.

If you have a company calendar, block out ‘no meetings’ time, and keep this sacred. Only your boss should be able to override this, though if you can encourage her not to, that would be good :-)

Lastly, remember you absolutely can do this – they wouldn’t have given you the second product if they didn’t trust you. Make ’em proud!

I guarantee that a little bit down the line, when you’re managing a team which in turn is managing dozens of products, you’ll look back and wonder what all the fuss was about.

Early on in my blog, I wrote one of my most popular posts – 4 key ways to spot a successful product manager – about measuring the performance of product managers. The problem is that a lot – and I mean a huge amount – has changed in product management, and my own approach, since I wrote it.

I found myself describing to Martin Eriksson at his recent book launch some work I did at the UK’s Ministry of Justice on measuring product manager performance. So here’s an update to my original article from a real-life case study.

A bit of context

Let me put this in context for you. When I wrote the original post about how to spot a good product manager, my experience up until that point had mostly been in the kind of companies that regarded product management as an evolution of project management – very process-driven. Gantt charts at dawn!

There, the product manager was meant to follow a waterfall process and attempt to make the user interface look like it hadn’t been designed by a database developer. (Even though it had.)

As a result my initial take on measuring how well a product manager is doing was against that backdrop.

Since then, I’ve learned a helluva lot from the dozens of different organisations I’ve worked with in the private and public sectors, many of which are embracing a more up-to-date approach to product delivery (i.e. user-centric, evidence-led, multi-disciplinary teams).

A terrible idea

Credit: BBC

In the UK’s Ministry of Justice, I was Head of Product for their digital team, during which time I was responsible for 12 product managers of varying degrees of experience. Each product manager worked with her or his own multi-disciplinary delivery team (product management, research, design, development and delivery management).

Misguidedly, the senior management team was clinging on to their attempt to apply a command and control structure to a capable team of autonomous and intelligent people. Not the right approach at all. They had asked me to “stack rank” my product managers (assess their performance and put them in order from best to worst). There’s rarely any other reason for doing this other than a desire to reduce headcount by culling the worst performers.1

It didn’t appear to occur to them – though I pointed it out repeatedly – that this was frankly a terrible idea.

Hurray for diversity

For starters, I knew that each person on my team was already a strong performer. Some of them had been in product management far longer that I had. Others were relatively new product managers, but even considering that were doing just fine, thanks.

Even if it was possible to assess all 12 product managers on a relative, linear scale normalised for their relative levels of experience, my team was perfectly capable of reading between the lines – they’d object so much to the stack rank exercise that they’d probably leave on principle. And I wouldn’t blame them for doing so, because who doesn’t love being devalued and treated as a commodity, right?

Each member of my team had different strengths and weaknesses, different backgrounds and levels of experience, different skill sets and different approaches. Sure, they were all product managers, but they each did product management differently. This was exactly what I needed them to do, because they were all solving different user problems.

Between compliance and disobedience

It was becoming clear that I couldn’t refuse to assess their performance outright. This would lead to my swift removal, then someone less protective of my team than me would simply pick an arbitrary metric, rank my team and fire a few of them. I could serve my team better by treading a fine line between compliance and overt disobedience. I would do some kind of performance assessment, but I’d structure it so that a stack rank was bloody hard to do.

This is what I came up with.

Start with the team

First of all, I sat down with my team and talked through what was going on. I then surveyed them to establish what they felt were the most important soft (emotional) and hard (technical) skills a product manager needed. We used the top ten or so as the dimensions for their assessment. I also asked them how conversant they were with skills from other disciplines they frequently would interact with, such as research, design, content and development.

Click on image to enlarge

Then I got them to assess themselves individually. They gave themselves a score out of 5 based on how skilled they felt they were in each area. I then plotted the results for each individual on a polar chart, and compared that to the team average.

Immediately it was clear to see where the team felt it was stronger and weaker. After seeing the results, several of my team took the initiative to start filling in the gaps: one person buddied up with another product manager who was more experienced in Kanban, another put themselves forward to do more public speaking, and so on.

Peer feedback

But the really interesting part was yet to come. I asked each of the delivery teams to rate (anonymously) their product manager on the same sets of skills. A product manager needs to be effective in their interactions with their multi-disciplinary team, so I was interested in their team’s subjective perceptions of their product manager. Were they good communicators? Did they inspire?

Click on image to enlarge

This exercise again yielded some interesting insights. In some areas the product manager’s self-assessment and their delivery team’s assessment were in agreement. In others, the product manager had rated themselves weaker in an area, while the team thought they were particularly strong – and vice versa. These findings were also worth exploring further.

The distance between us

Lastly, I asked senior management to rate the product managers in the same way I’d asked the delivery teams to. With a couple of notable exceptions, it was clear that senior managers had next to no idea how to answer the questions – they simply weren’t close enough to the product managers to make any kind of assessment.

While this highlighted a failing of the senior management team in what was declaring itself to be a ‘flat organisation’ (yeah, right), it also was a point for improvement in the product managers. Given how reliant they were on having senior management’s support to keep things moving apace, clearly they needed to invest more time into stakeholder management.

There is no single, absolute measure of performance – people have different strengths and weaknesses (and that’s okay)

You don’t want a homogeneous team – user problems are not all identical so you want a mixture of personalities, experience and abilities on your team to tackle them

Never underestimate the power of a pretty graph – it’s never a waste to spend a bit more time on making your stats easier to consume and tell their story more easily

Examples

Click the image for the live demo

I used Highcharts to render the charts. Each product manager had their own page – for their eyes only. Senior management only got to see the aggregated averages in case they got any big ideas.

I’ve got a live demo of the full report page available if you fancy taking a look. Needless to say, for this article the individual’s data has been anonymised (there was no product manager called Petra) and slightly randomised.

Forming a multidisciplinary team – ensuring that the product delivery team has the right skill sets needed throughout to make the product a reality

Writing epics and user stories – not just a task for product managers, as everyone on the team should be writing epics and user stories, but ensuring the stories written are focused on the user outcome (achieving real-world goals that matter), not the output (features and widgets), and driven by hypothesis

Prioritising epics and user stories – ensuring you and your team are working on the most urgent and important things, bearing in mind that priority and urgency will continually change with the circumstances and as you learn more about the problems you’re trying to solve

Using a user story and bug tracking tool – whether it’s a spreadsheet, a bunch of Post-It notes on a wall, or a more specialised online tool, making sure that it conveys information clearly, accurately and helpfully to the team and anyone else who’s looking (hint: transparency is a good thing)

Participating in Scrum / Kanban / Lean Startup / other delivery methodology of choice – you should know when and how to use different delivery methodologies depending on how well you understand the problem and its solution (some methodologies are better suited for discovering answers quickly when uncertainty is high, others for efficient, repeatable output when uncertainty is low)

Setting SMART objectives / KPIs / success criteria – whether it’s a personal objective or a broad product goal, everything you do should be testable and measurable, otherwise how else will you know you’ve succeeded or finished?

Analysing and interpreting data and metrics – no escaping it, you need to be fluent with extracting information from data in an honest and unbiased manner

Planning how to migrate users from an old thing to a new thing – you’ll have smaller user transitions between major product iterations and bigger transitions when you retire a product in favour of a replacement

Launching a product – even if you practise continual deployment, every now and again you’ll need to launch a new thing and make a bit of noise

Retiring a product – knowing when it makes practical sense to decommission an old product, and actually doing so

Public speaking – whether it’s one-to-one, a team daily scrum, a stakeholder meeting or a public address, you need to know how to take the needs of your audience into account and communicate effectively

Promoting your product – not just advertising, but evangelising to anyone who’ll listen why your product matters so much to the people who use it

What do you think of the list? Are there any product management skills you think are missing? Or any you think shouldn’t be there at all?

I bet you’ve found yourself in this situation. You’re trying to get your head around the main user problem your new product is going to solve. The thing is, for every question you try to answer, several more questions arise.

On a recent product I was helping with, we were trying to make it easier for academics to exchange information with each other. “Simple,” everyone thought to begin with, “just use Dropbox.” And for some users sharing certain types of information, Dropbox was probably the best answer.

But as we delved deeper into our user research, we realised that there was much more to it. Some users needed to exchange snapshots of very large data sets – a sequenced genome, for example – so synchronizing the files to and from Dropbox would take far too long to be useful when needed.

Similarly, some data were particularly sensitive and needed to remain not just on UK servers, but within servers owned by the university. Dropbox wasn’t the answer here either.

It was like our user problem was a fractal – the more closely we examined it, the more complicated it became.

When this happens to you, it can feel overwhelming and increasingly difficult to gauge whether you’re making progress any more. Couple this with a bit of pressure from senior management to deliver something tangible and it’s extremely tempting to lose heart with research and build something based only on what you’ve found out so far.

To stop too soon is a mistake.

You’ve not yet reached the tipping point where you know enough about the problem to set about trying to solve it.

Discovery is where you set out to understand the problem, who has it, the causes of the problem, and why solving the problem matters. You can think of the first part of discovery as being divergent – each question you ask uncovers several more questions to answer. For some, this can be the disheartening bit, as they realise how complex the problem is, and how little they actually know about it.

For others this first, divergent part of discovery is extremely rewarding: every question answered reveals another piece of the puzzle and increases the team’s understanding of the problem. This knowledge helps you to decide whether it’s feasible to solve the entire problem as originally intended, or whether you need to break it down and focus in on a specific aspect to begin with – perhaps a subset of the users, or one part of what you now realise is a connected set of problems.

Then there comes the tipping point. You’ll know you’ve reached this point when you start to see trends emerging. By now, you’ll have spoken to enough users, performed enough desk research, and run enough experiments to see certain patterns. At this point, your tests of your understanding of the problem will start to come back with much greater success.

In my Dropbox project, this was the point at which we could confidently play back to our users what we’d learned about them and their file sharing needs. Now our user interviews were less about asking what they did and why, and more about checking with them what we already understood.

After the tipping point, your understanding of the problem solidifies until you have a pretty robust definition of the problem, who has it, what causes it, and why solving it matters. (But you’ll never know everything.)

Then you embark upon figuring out the solution to the problem you now understand. And just like before, you and your team start with (at best) hunches of what might solve the problem. You have to diverge again before you reach the second tipping point and start to home in on the right solution.

Some organisations, like the UK’s Government Digital Service and other digital services, call this part of the process alpha.

Again, it’s tempting to jump on the first plausible solution that appears. It’s unlikely that you will hit the best possible solution first, so keep your solution experiments, cheap, short and throwaway. You are not building the product for real yet.

When to build for real

Only when you’ve tried out several possible solutions, established what bits work (which you keep) and what bits don’t (which you discard), can you decide what you’re going to build for real – that is: to scale; with the right choices of technology for long-term support; and secure and robust enough for a real-life user base.

The build should then be relatively quick because you and your team already know what you’re going to build, and are confident that it’s going to solve the problem.

This is an interview I did a little while ago with a user experience author living on the US East Coast. She was interested in moving into freelance product management.

We cover:

how to move into product management;

differences between working in the private and public sectors;

KPIs and financial modelling; and

the pleasures and pains of being a product manager.

If you’re interested in asking a question, you can do so in the comments below.

SA: You started with Classics and ended up in product management… I wonder what led you there?

JB: I kind of fell into it, if I’m honest. I’m a self-taught techie. I ended up doing several different jobs at a startup I joined after university.

SA: Yes, I noticed you did some development work after university… impressive with a degree in classics :)

JB: Thanks! So how can I help you out ?

SA: So I’d like to learn more about your product management experience. I’m interested in that field, but it seems like that they are looking only for full-time employees, not freelancers or consultants. Also when I check job adverts, they expect several years of experience, even though my skills could be transferable.

JB: That’s certainly generally the case. On the first point about freelancing, most organisations are looking for someone full-time, but the way I typically work is to fill in when there’s a gap.

This might be while they’re recruiting (if someone’s left abruptly), as it typically takes about 3 months to find someone to hire, or it might be a startup that doesn’t yet need a full-time product manager because they’re not big enough yet (and the founders are not busy enough running the company yet).

So if you can find companies at the right time, you can open their eyes to the option of having a product manager to fill the gap, or to set things up for their eventual product manager hire.

The experience point is a bit circular, isn’t it? You need experience to get a job, but you need a job to get experience…

SA: I have 15 years of experience in various fields. I started as a graphic designer, then moved into web design. I also did some user experience, QA work and also do accessibility, so I can understand well what team members would need if I was a product manager.

JB: What have you done on the technical side of things (i.e. working with developers, experience with different technologies)?

SA: Coding mostly. I know the ins and outs of that side. Client and server-side, security, QA etc. but i am done with it :) I prefer to work with developers instead. I’m more interested in strategy.

JB: Okay, good – so you’ve got hands-on experience there also. How about on the business side of things, have you ever dealt with internal stakeholders directly, managed a team etc.?

SA: When pursuing a business degree, yes, and working on projects in school, but not in work experience. I have experience sharing my ideas during work meetings and offering solutions.

JB: So potentially an area to focus on there. You’re aiming to achieve balance in three areas – the customer/user, the business, and technology.

SA: Are product managers expected to design or code? I’m okay with wireframes and personas but not more than that. I’ve done enough of design and coding and am tired of it :)

JB: It depends. You should rarely be expected to code as a product manager, and in an ideal world, there would be a dedicated designer who would create the front-end look and feel, based on decent user research.

In practice, organisations rarely are enlightened enough to have dedicated specialists, so the product manager unfortunately becomes a bit of a catch-all role. Plus, with your background, an organisation might think they can kill two birds with one stone by hiring you to do both product management and user experience (run a mile from this).

I find user experience a bit of a confusing role title, to be honest. It’s far broader than the simple visual UI design. Think of it as several separate functions that sometimes can be done by the same person – as long as they’re able to keep their natural biases out of it.

Unbiased user research is one main bit, of which outputs include user personas, etc. Then designing the solutions to discovered user problems is the other main bit. The yin and yang, if you like. But there’s plenty more to it than that, I’m no UX expert!

SA: What about organisations expecting the product manager to do the job of quality assurance (QA) tester and project manager?

JB: Run a mile – they clearly have no clue what a product manager does. As a product manager you might be fulfilling the role of user researcher if there is nobody else doing that, but you should really avoid getting roped in to do the design as well. It pulls you too much into the day-to-day stuff and ties you to your desk, stopping you from going out and meeting users.

If an organisation is still thinking in terms of QA testing and old-school project management (Gantt charts etc. etc.), there’s a good chance they don’t really get product management either. And if they think product manager = project manager – again run a mile :-)

I was told that it also depends on a size of organisation? I talked to one PM who does QA because they don’t have QA managers.

JB: These days, I’d argue that testing should be part and parcel of the development process, rather than lobbing something over the wall to a dedicated team testers – everyone on the development team should be writing their own tests for their code and peer reviewing their work these days.

Product management confuses organisations because it’s evolved in a couple of major ways over the last 20 years.

Generally speaking, if an organisation is thinking in terms of project management, Waterfall, huge lists of requirements specified up-front without anyone ever speaking to a real user, then product management is essentially an administrative role, shuffling large requirements documents about.

If however the organisation is enlightened enough to realise that building something that users need comes from actually talking to users, then the chances are they’re probably open to working in a more agile, evidence-led way, which does away with all that unnecessary up-front documentation.

Why bother planning for 6 months from now, if things are still so fluid and new that what you learn next week from your users might change your product’s direction completely?

SA: I feel it’s such a common sense to talk to users and find it hard to understand when businesses don’t.

JB: Yup. Just like UX, product managers are champions of the user. Half the battle is getting an organisation to let you speak to (potential) users.

Coming back to the freelance thing, it’s arguably slightly easier to get this point across as a freelancer, because the organisation feels that it has to listen because it’s paying you and it’s a short-term engagement.

SA: Is it because they are more likely to listen to an outside person than in-house staff?

JB: Annoyingly for the in-house people, yes (in my experience at least). It doesn’t mean that you can’t change an organisation from the inside, but you have to be tenacious.

SA: Okay. Which organisations are more likely to understand about product management and UX? Do you still have problems with clients who don’t?

JB: It’s difficult to generalise – there are notable exceptions in each industry/market, plus I’m not as familiar with the East Coast scene as the UK.

Here, the banks, media publishers and big multi-nationals tend to be behind the curve, if for no other reason that they’re like oil tankers and it takes them ages to change direction. One place where you might be surprised by would be the US Government.

SA: Is it because they are larger organisations? Do larger organisations tend to be behind? because of bureaucracy?

JB: Larger yes, bureaucracy yes. Though actually US Government in process of changing. Take a look at the US Digital Service and 18F. They’ll probably be very welcoming of a person with your talents, and you can probably work on a freelance basis.

If their model is similar to the UK Government approach (which they initially modelled themselves on).

SA: Honestly I prefer private sector :)

Oh. I didn’t know that Dana Chisnell worked for the US digital service.

JB: There you go – you might be surprised.

SA: No wonder why their website looks modern :) That’s interesting… I’ve always thought of any government as outdated and bureaucratic :)

Like for example US government uses section 508 for accessibility which is way outdated. For some reasons they don’t follow WCAG.

JB: Yup – that’s how many people think of them, that’s why I’m suggesting you’d be surprised if you took a look. If it’s anything like the UK Government Digital Service, it will give you an opportunity to build out your experience for your freelance work.

SA: I still prefer private sector but I will think about it. One of reasons is because I don’t like going through lots of paperwork – government agencies usually do that.

With private companies they ask for less paperwork. I worked for a local public university and I filled out piles of paperwork plus fingerprints. It doesn’t usually happen in private organisations.

JB: I can’t speak for the hiring process – in the UK part of the reason there were so many freelancers involved was because it was easier / less paperwork to bring them in (at least at the outset). Whether this translates to US, I don’t know I’m afraid.

SA: That’s interesting. I will keep that in mind. Which parts of product management are organisations looking for for freelancing work? Product backlog? Testing? User research?

JB: It always depends – often it’s the whole deal, but with a quick start, i.e. you need to get up to speed in days rather than weeks or months. More often than not, you’re going to be joining an existing team working on an existing product. The brief will be “make it better / more profitable / gain more users”.

So I think the real skill is to identify the problems with the product, with the team, with the organisation, and set about clearing those obstacles as efficiently as possible to allow the product a chance of success.

It could mean doing better user research, or it could be about refocusing the team onto stuff that matters. Or it could be about convincing senior management to do something differently.

Essentially as product manager, you take responsibility for all aspects of the product, even if you’re not directly in charge of each. So if the sales team is performing badly with your product, it’s on you to figure out how to influence and help them to be better.

The solution might be multi-faceted – improved product, better engagement with sales team, better messaging, better training – whatever – but you’ll need to turn your hand to lots of different things to get the job done as product manager, ideally, with the help of capable and motivated specialists in each discipline (but this is often a luxury).

SA: What about KPIs and financial modelling? Does a product manager do it regularly or do they work with marketers/finance people?

JB: That certainly is a part of it, either in a formal sense or on a more day-to-day basis.

SA: I notice some job descriptions requiring experience with financing or even having a MBA.

JB: As a product manager, you should have an idea of whether your product is on track. This could be through monitoring of KPIs (some of which, but not all are financial measures).

SA: What if a product manager is great with ideas and words, but has difficulties with numbers? For me personally I care more about how satisfied users are than how many of them use a product.

JB: This is an area where I’d argue a product manager needs to be credible – not in the sense of complex financial modelling (if the underlying data for a model is based on guesswork, the model will be bogus), but in the sense of being able to back up your decisions with data and evidence.

SA: Helping a business with bottom line is important, but I feel that if their product is not user friendly then it may affect their bottom line.

JB: This is where you need balance between needs of users and needs of business.

SA: I was told that financial modelling is based mostly on guesswork as you cannot predict what will happen?

JB: Sure, an unusable product will be likely to do badly financially, but you also need to have confidence that you can understand the financials for your product so you can have a sensible discussion with sales and finance when you need to.

SA: I may not know history of the business to understand their financial status or predict how much they may profit. Especially if i do freelance work. Do you do a lot of KPI and financial modelling?

JB: KPIs yes, financial modelling I try to avoid. Predicting the future is inherently a fool’s game, however if you’re building a product or feature based on an understanding of the currently baseline, you can hypothesise whether the improvement will change that baseline.

That baseline could be customer satisfaction, conversion rate, speed of completion of task – whatever.

SA: That’s why I was asking what organisations basically look for in temporary projects :)

JB: KPIs should be the kind of metrics you keep track of every day to indicate whether you’re on the right track (ideally automated). KPIs prompt the questions to investigate, not provide the answers.

So to give you an example – I was recently head of product over at the Ministry of Justice in the UK. Each team would aim to spend between 6-12 months taking an idea from initial user research to live release.

SA: And it worked?

JB: More often than not, yes. Appropriate KPIs would come out of the initial user research and the ongoing research – they were the measures of how we knew the product was helping users to achieve their goal more easily and efficiently. These metrics were on big TV screens and automated so they were shown in real-time.

SA: What do you use for metrics?

JB: Sometimes we had to approximate the baseline for certain measures because no data had been collected before.

For example: improving the registration process for a particular service would make life easier and quicker for the user, but the efficiencies should also result in cost savings on the business side because of less manual processing, no need to take calls etc.

To prove this hypothesis, we needed data on how much a transaction costed to begin with. We did this by researching the original user journey and identifying the bottlenecks and costly parts of the process (i.e. when people get involved to do something manually).

This also guided us to figure out where the improvements could be made to the overall service through a digital product to provide automation.

By building the same measurements into the product from the outset, we could demonstrate which bits of the process were faster, and therefore cheaper (less staff time processing, fewer mistakes etc.).

So the kind of things we’d measure would be completion rate, time taken for transactions (both user and staff), rejection rate (whether the user achieved their goal first time or had to go back and do it again).

You had to be good enough with numbers to be able to figure out the baseline, decide the KPIs, measure them all the time and take action to fix them if they weren’t improving as expected.

Financial metrics are but one aspect, but these are usually on a more granular level to traditional finance measures, i.e. in an e-commerce situation average basket value, average spend per user per month etc.

SA: Makes sense about metrics.

I need to go soon. Before ending the conversation I would like to know what made you interested in product management, what are your favourite parts and which parts you feel frustrated with? Sounds like metrics are your favourite part? :)

JB: I got interested in product management because I fundamentally wanted to create better products, and I’d realised that the best way to do that was by focusing on user needs. I also enjoy the variety of jumping between different disciplines, if not doing so myself, then working with talented people to do so.

I’m as passionate about usability testing, accessibility, different software architectures, good product copy and so on, and could talk about those for ages also – that’s kind of what being a product manager is about. They’re all important!

SA: Yes, everything is important :) What about frustrations?

JB: Usual stuff – not getting the chance to speak to users, organisations thinking they know best for users without any evidence whatsoever, being stuck in process that the organisation is afraid or unwilling to change.

SA: So nothing specific to product management?

JB: You’re always fighting fires; and you’re the shield for the team and the product so you need to have a tough skin and be forever calm and collected, and the smartest, most knowledgeable person in the room at all times :-)

SA: Smartest? :) Everyone has their own smarts and talents – you cannot be smart at everything ;)

JB: If being smart = having done the research and having the knowledge, then yes. Or at least knowing who to ask ;-)

SA: Okay :) How did you hear about General Assembly?

JB: I met Matt Cynamon just after General Assembly started up in the UK, then became their instructor early on for their 10-week product management course. I also wrote their book on product management.

SA: Yes, I saw your book and looking forward to reading it :)

JB: Most kind :-)

SA: Why is the General Assembly logo on your book?

JB: They paid me to write it, but it’s based on my own experiences and learning.

SA: Oh okay. Makes sense. I did wonder why the logo. So why not self-publish? :) You have more control that way – that’s what I did with my book.

JB: I didn’t realise I was any good at writing books until I’d done at least one…! Call it an experiment.

SA: I was not sure myself either and just took a shot :)

JB: Best way.

SA: Okay, I gotta go. Pleasure chatting with you and thanks for talking to me. I look forward to reading your book and can I ask you questions later if I have any?

JB: Feel free – drop me a line if you fancy a chat. Also I’d be very grateful for a review on Amazon if you bought the book there!

SA: Sure – I’d be glad to write a review :) Have a nice weekend and keep in touch.

When it comes to the ecommerce checkout process, what’s one thing that retailers are doing wrong? What’s one thing they’re doing right?

From a product manager’s perspective, there’s no one thing everyone gets wrong – it depends entirely on the circumstances.

This is why context and understanding of user needs are so important. Figure out what people need – their goals, frustrations, distractions… everything – then figure out how well your ecommerce capability meets those needs. Improve, then rinse and repeat.

And don’t forget Marty Cagan’s observation that “people don’t know what they need until after they’ve seen it”.