How NOT to make your product a feature factory

An interview with John Cutler, senior product manager at Zendesk. John has written extensively on different product management topics. In one of his most widely read Hacker Noon posts, John discusses what he calls Feature Factories. In this post we explore how to avoid the feature factory approach to product management.

How would you describe a feature factory?

A feature factory is a product or company that measures output over outcomes. It’s a storypoint machine where the company only cares about is how much is being shipped. The company celebrates the work done and it feels like a conveyor belt going in one direction. No one is thinking about the question: is any of this really working!. No one is thinking if any of this complexity is adding business value.

There is a fundamental difference between knowledge work and the manufacturing industry. Output cannot be measured in the same way. You have to get that experience right for the customers and focus on the right outcomes.

Some engineers can be motivated by a decently challenging technical problem. There are aspects of building a product which is challenging regardless of whether it has impact. For product people, if there isn’t anything compelling about the product you can’t do your job.You just check out or leave so the impact matters.

What do you do to start focusing on outcomes and not output?

We have to start by measuring product differently.Leadership can measure every other department pretty easily in the company like operations and sales by thinking about the bottomline of the business. With disciplines like product and UX it’s much harder.

The next thing is to look at the problem and figure out how to create outcomes in that area. One thing I always say is if the product is not doing their job they are outsourcing product strategy to the rest of the company. Put the pieces of the puzzle together in a way which people can relate to and understand. Most people get that that product can be destroyed by unvalidated complexity. You have to figure out how to create a system of value delivery.

Avoiding focusing on output also means that it’s not that you are not working enough or working hard enough. It’s that you are working more thoughtfully.

Let’s go through 10 ways in which we can do this:

1. Measure the results and learn from them:

Companies spend a huge amount of time making sure they are tracking every action of the user. The main problem I’ve seen is not lack of access of data but they don’t measure if any of the features actually work.

To start measuring the impact of releases it’s also important for a company to have a learning mindset. You need trust in your organization to measure impact because there will be failure along the way and some releases may not work as intended. The learning aspect of measurement is also different than core level KPIs — it’s how can we measure things to get better than just coming up with KPIs and OKRs. Differentiate measuring outcomes from measuring people and teams.

It’s also important to establish health metrics for the product to make sure user experience is maintained at all times as well.

2. Storytelling and rituals make difference

One small way to create a product mindset around outcomes is to create small rituals around reflection. I did this in the form of precap design studio where when you kick off an initiative you give a presentation with blank numbers. And we would reflect on the before and after regularly once we ship it.

Having a ritual of reflection is thinking about the before and after. By doing this you get the team focused on the artifact. Every two weeks do a retrospective on the metrics. Doing a little over time is better than doing a whole lot at once and over committing.

Another simple tactic is emailing out your metrics dashboard to the organization or sharing the retrospective meeting notes. These little things have an ability to become viral in an organization. Small rituals help build the habits.

People usually over commit in terms of the methodology. Small companies invest massive resources to try and do some form of predictive analytics which takes at least an entire day to even report on. Start with what’s sustainable and what you can keep up.

3. Avoid shuffling of teams frequently and the project culture:

Some companies get the product and engineering people in a room quarterly, and have engineers self select the initiatives they want to work with.

Sometimes shaking off teams can be advantageous but in this case the company gets obsessed with 150 things they have to do in the quarter and starts playing team tetris.

Product teams should be focused around a mission. The release of a feature is just the start of it and teams should get the satisfaction of seeing it through. The moment you start calling features “projects” then it signals that when it’s released that’s the end, when in most products it’s just the beginning.

4. Pay attention to how you celebrate product

Make sure you celebrate outcomes not just outputs. Some companies have demos where once in a week or two weeks the product team demos all the new features, which is the signal of being part of a feature factory.

There is a huge impact of how you celebrate as a company. You can talk about different things like we talked to so many customers and what we’ve learned, what happened with the feature we shipped and how the customers are reacting to it. Little words matter.

Support and marketing have the need to know what is being shipped — usually these shipping success theatres start when someone says “I don’t know what’s coming out”. That’s a different problem than knowing if the stuff you ship is working. Be good to other teams and relay features but don’t solve that problem by celebrating shipping only as a company.

5. Iterate immediately after you ship to achieve better outcomes

Some PMs tell engineers that improvements will come “later” and move on the moment the feature is shipped. Pulling a team to work immediately on something else and getting back to it after 6 months doesn’t work in practice.

Companies also don’t acknowledge what the opportunity cost is. Sometimes teams take weeks to understand what the other team has built to iterate on it. One way to convince your company (assuming your dev team costs $50k per week) is to ask management if they’ll be willing to spend $150k to come back to the item later. The best teams usually iterate on it to reach a better outcome.

6. Get into the discipline of removing complexity and features which don’t work

One good exercise is to match customer value streams to areas of complexity. There will be areas of the product that have a large set of features but don’t map to a lot of complexity. Killing features require multiple dialogues in organizations.

Once you start mapping your features to customer usage you can start digging into why and what you need to kill. In some cases you will see features being used for the way it’s not intended .

7. Make sure PMs do retrospectives regularly

Products are a series of decisions. Humans are terrible at talking about decisions retrospectively.

Usually product management meeting are about what we ship and status updates on ships that have sailed, they tend to be very reactive and there isn’t really any time for reflection.

No PM says they made a wrong decision a few months ago. It’s important for product people in the company to get together and discuss how the craft can be better as a whole and reflect on the decisions they have taken.

8. Get in the habit of iterating on features to achieve the right outcomes

In feature factories product roadmaps have no room for iteration and it’s all about shipping new features.

The trick to get the company to think around iterating is to craft stories everyone can relate to around adoption. For example, you can say we have 1250 customers right now who meet the category for which a feature is valuable and right now 100 of them use it, so we want to focus on getting adoption up.

You should be able to make assumptions for your people who will be adopters. Say you are getting 500 beta testers on a new feature every week, one thing you can also say is that we will keep iterating with successive cohorts till we we get a high satisfaction score from most of them. Tie it to metrics and customers and tell a good story.

Sometimes the people who get nervous with this approach are your engineering management who are highly incentivized by a feature factory and their measure of success is being responsive to product needs and keeping churn on the team low. They don’t want to be blamed for being slow in any way. Usually what happens in that case is that engineers resign to the feature factory model and say we did our part and it was a poor product decision. This just drives more output and not good outcomes which are perverse incentives for engineers as well. Make sure you align the iterations with good incentives for engineers as well. Incentives are a difficult topic, but if people are incentivized for the fast completion of projects you will get a different culture than if teams are incentivized and rewarded for outcomes and learning.

9. Don’t just focus on reactive prioritization

All product managers focus on prioritization — in many cases it’s the wrong type, it’s reactive prioritization all the time. PMs are always stressed out with so many input channels and are trying to balance all of them to ship things and getting sign-offs from the company. Much of prioritization is about making educated guesses and making sure all stakeholders are happy (even if there is very little data backing it up). These guesses are important and it’s good have a good set of heuristics but don’t stop there.

Have conversations about the cost of delay — what will people pay to have a feature out today and not delay it. Have the conversation about whether is really worth it to us. Are we shipping value every sprint and by value it’s not just shipping, it’s learning and risk reduction as well. That’s more important to focus on. Force different types of conversation in the organization.

What people are missing is talking about if the product delivers continuous value and continuous learning. You want to bet on your horses when they are moving not before it and spending too much time on the reactive prioritization part.

10. Avoid handoffs and tackle bottlenecks

In a lot of companies it’s all about being more efficient and product management teams are outsourcing to other teams to get work done efficiently. So people start focusing on local maxima and not global maxima and managers end up playing team tetris. Customer feedback is a great example of a hand-off gone wrong. Your team ships a feature and you wait patiently for support to tell you when things have gone wrong.

The culture of handoffs also means the culture of working around people. Instead of tackling bottlenecks you start working around them.

The key to solve it is to visualize the system and show what steps things go through before it gets to the right people who can really act on it. Periodic reflection helps as well. To deliver 3x value from your product you have to make big shifts in the way you operate and chasing local maxima won’t do it.

Finally John, what’s your product management process?

I don’t use scrum and don’t do story points. It’s important to ask oneself why are we hiring the tools we are using and then use them. Scrum in a toxic and non trusting environment just becomes more toxic. If scrum is a good template for you go ahead and use it. But if people say they want to use scrum to just measure velocity or holding engineers accountable you want to ask why.

For example, we are good at writing stories and decomposing problems in general we take 3–5 days for every story so you can easily calculate velocity, we don’t need scrum for that.

We use JIRA but it has it’s limitations, it’s built on the philosophy that there is only one owner so you start to develop a shadow economy if engineers need help from each other. Know the limitations of the tools you are using. Kanban is an amazing visualization tool which I like.

The more important question to focus on is what’s the outcome and are we delivering continuous value every sprint which is what we try to do.

A call to action is a marketing term that refers to a prompt that invokes a response leading to a sale. When referring to a call to action (CTA) in the digital design world we usually mean the interactive element that leads to the next step in the experience - something that needs to be clicked or tapped.

User testing refers to a technique used in the design process to evaluate a product, feature or prototype with real users. There are several reasons why you might want to undergo usability testing, the most common is that it allows the design team to identify friction in a user experience they are designing, so that it can be addressed before being built or deployed.

WYSIWYG (pronounced WIZ-ee-wig) is an acronym for "What You See Is What You Get". It helps identify an an interface that allows user input resulting in an output that is rendered in a similar way. For example; a word processor application interface might resemble a piece of paper,so when printed the user can see how the output will appear.

A content management system (CMS) is an tool that allows a website editor/administrator to manage the content that is displayed. Websites are made of HTML and CSS to create pages. Pages can be hard-coded but would require technical development skills to make changes. A CMS usually allows a person without coding knowledge to amend existing and add new content to a website using a WYSIWYG interface.

Responsive web design refers to a web page that dynamically adapts its layout to fit the size and orientation of the device on which it is viewed. A responsive design allows for a more optimised user experience across desktop and laptop computers as well as smartphones and tablets of varying sizes.

User stories allow the functionality of a product or service to be expressed as written descriptions of an experience as seen from the users perspective. The writing of user stories creates a list of design and development tasks to complete in order to create any required functionality.

A user interface (UI) is a conduit between human and computer interaction - the space where a user will interact with a computer or machine to complete tasks. The purpose of a UI is to enable a user to effectively control a computer or machine they are interacting with, and for feedback to be received in order to communicate effective completion of tasks.

A persona in UX Design is the characterisation of a user who represents a segment of your target audience. On a project you might create any number of personas to be representative of a range of user needs and desires. The solutions you design must answer these needs in order to deliver value to your target audience.

A great, reliable, inexpensive method for discovering patterns in how users would expect to find content or functionality. Card sorting is used to test the taxonomy of data with a group of subjects, usually to help inform the creation of the information architecture, user flow, or menu structure on a project.

A technique used to generate ideas around a specific topic. Often done in groups, but can be done individuals. The process usually involves writing down all ideas around a topic onto paper, a whiteboard or stickies often implying some kind of association.

An MVP is a product that has the minimum set of features to prove the most essential hypothesis for a product. Businesses building a new product can create a Minimum Viable Product to prove that an idea is viable and warrants further investment. A further benefit being that the next stage of development can be informed by feedback obtained from testing that MVP.

A sitemap is a diagrammatic representation of a hierarchical system. It usually depicts the parent-sibling relationship between pages in a website, showing how sub pages might be arranged underneath their parent groupings. This arrangement forms a map of the site.

A user journey represents a sequence of events or experiences a user might encounter while using a product or service. A user journey can be mapped or designed to show the steps and choices presented as interactions, and the resulting actions.

A prototype is draft representation built to test ideas for layout, behaviour and flow in a system. Prototypes are an indispensable tool for resolving a large number of potential issues in a concept or business before too many resources are deployed to put a design into production.

A Wireframe is a visual schematic that conveys a basic level of communication, structure and behaviour during the design of a system. Wireframes are low-fidelity designs that bypass including a detailed user interface or visual design, conveying just enough to get across the core idea.

To say something is usable is a qualitative statement about how easy that thing is to use. Usability is an assessment of how learnable a system is and how easy a user finds it to use. The usability of a system or product is a key factor in determining whether the user experience is a good one.

Information architecture is the design and organisation of content, pages and data into a structure that aids users understanding of a system. A more organised system enables users to more easily find the information they require and complete the intended tasks.

A general term that covers all aspects of a user's participation while engaging with something that has been designed. Usually when talking about User Experience in the digital design field it refers to the interactions, reactions, emotions and perceptions while using an app, service, website or product.

Got a problem to solve with your service?

We'd love to hear about it. We can add value to any business that has a digital product/offering.