I’ve been thinking a lot about what are important things to measure for a desktop software products and SaaS projects and here are my thoughts:

You can gather operational metrics and leading indicators that can impact your decisions as a product team. I think there is value in collecting both. Its important to think about what are the key metrics you want to focus on and if there are any key leading indicators that can predict changes in these metrics. For example: Reduce active use is a leading indicator for reduced revenue in the future. Lack of customer acquisition is a leading indicator of reduced trial downloads and eventually trial conversions, etc.

Generally it helps to have uninterrupted time and a piece of paper to jot down what’s important to your business. For example: Is your goal to grow revenue? If yes, here are the hypothesis that you might want to test:

Marketing is not effective – not enough new customers coming in?

Customers are coming in but existing customers are leaving faster?

So, I’ve tried to categorise metrics that I try to gather for any software project I work on:

Customer acquisition Funnel

This is different for every channel through which you sell your product. The Direct Channel is simplest to understand and the metrics are listed below

Visits to the landing page – say “myproduct.com”

Landing page to trial download or “sign up” conversion

Trial to full product conversion

These to me are the most important metrics that drive business goals and metrics. Everything else is making sure you provide sustained value to keep people once you’ve got them. Keeping existing customers is always easier than getting new ones. If these numbers are healthy and the business is tanking then you start focussing on customer retention.

Customer Retention Metrics

Active use: Usage by version (number of app launches by version per day)

Such a spreadsheet can also answer my Customer acquisition requirements.

Customer Profile Metrics

Platform – Mac/Win/iOS

A Questionnaire during app install that can ask the user to share more about themselves

Feature usage metrics

Most used features – be careful about when this event is logged and how its logged

Are new features being used, and how will you agree that a feature is being used a lot?

Look at new feature usage/session or /day post release

Is the addition of new features impacting customer retention or customer acquisition metrics?

This is really important – especially for established products. You may find that nothing you do impacts the business metrics. If you do then why develop features. Work on a new product, service or something else.

While any experienced developer can instrument your app so that you can log usage data on your servers its important to get the user to opt into usage tracking. Here are some companies that make it easy to instrument app usage and help startups get a better understanding of their customer

http://www.deskmetrics.com

http://www.trackerbird.com/

Omniture from Adobe.com

Google Analytics since its free

You can also try to implement a custom logging solution built by your engineering team

Every now and then engineering is going to come back to you and say that something’s not possible for will take too much time to implement. Its generally good to hold your ground and try to motivate the team to push the envelope or look at the problem differently. And – to always ask for alternatives.

I’ve noticed that, at times, when I push back, splendid things happen. The engineering team gets a chance to shine and prove themselves wrong. Trust me. Development teams enjoy solving problems and outdoing themselves. This is what makes it really fun to work with a really talented and motivated teams. Its easy to agree to a cut down feature set or to say yes to an “ok” experience but it never challenges the teams to out think themselves.

So.. when there is no time pressure, I ask the teams to think about the problem some more and come back with alternatives. Push back and state why the solving the problem a certain way is beneficial to the user. State the importance of the “right” solution in terms of the user experience rather than your opinion. Smart people can always distinguish between facts and opinions.

While I’m fairly technical I could not have created responsive websites for http://www.delhishoppingtour.com/ or http://www.vkelectric.com/ without wordpress. I did this in just a few hours of my free time and have had great success with these sites. And, this blog is also on wordpress.org. I dont think I will pay anyone to create websites for me unless they were really complicated or required some insane level of customisation or security or high level of design.

This thought was echoe’d recently by Chris Hardie in his “The end of website development” post and has been discussed here on hacker news. The comments are enlightening as they show a crop of developers clinging on to hope of grabbing a larger share of a dwindling market while others agree that what wordpress, squarespace, etc provide is more than just good enough. And if you get a Pro membership, you might be able to bill the clients the same as you would for a custom site and get it done using wordpress in no time.

The comments also highlight how the web is becoming less and less transparent and open be being gamed by companies with perverse incentives. I’m sure you’ve found yourself on a dynamically generated web page that contains a lot of crap around your search string. I know I have.

Considering that NPR’s This American Life has about 200,000 listeners, I assume Planet Money would have about 100,000. Of this 20% or 20,000 people paid $590,807 for the T-shirt. This is a contribution of about $30/subscriber. This is pretty staggering and demonstrates both the power of an engaged audience and of KickStarter, which is owned by Amazon.com

One of the biggest issues in working out of India for a US based company is late night and early morning meetings. Really … India is as far as you can travel from the US without traveling back.

So either you are up early or you are working through your evening 8:30PM-11:30PM trying to sort through issues with the US counterparts. These meetings are essential for the project and for your professional growth. As a Product Manager, you should be seen as the face of the product and hence you need be make yourself available for all roadmap review meetings etc with the business unit heads as needed.

In my many years of working with the US company, I’ve been able to manage my work life balance by using the following guiding principles:

Give up one night a week for meetings instead of short spans of time on every other night

Consolidate meetings back to back on this night

Don’t be shy to ask the US team for an evening meeting

Skew evening meetings based on where most attendees are so that you can maximise attendance

Say no to any unnecessary or agenda less meetings

Its a well known fact that issue resolution and relationship building is faster over the phone, in person that over email. Changes are high that if you stick to working via email you will not receive information via informal channels in the organisation. These channels are always faster than the formal ones.

I’ve read so many motivational blogs and books that encourage you to quit your job and be everything you think you can be. Here are some examples of these works:

John Acuff’s Quitter Conference: http://www.jonacuff.com/blog/events/

Four Hour Work Week: http://www.fourhourworkweek.com/blog/

Personally I’ve found a lot of the ideas in the 4HWW fantastic and original (dreamline and the dreamline worksheet) but some of them don’t work if you already live in a low cost economy like India and one is just wrong. This is when Tim Ferriss says that everyone hates their job. I don’t.

Here are the reasons I stay at my job:

I can significantly impact the life of millions of users of my product as they use my product 8hrs a day or more. I can’t have that scope in my own small company

I can compete globally on a level playing field with the best in the industry yet work out of India

I can be very creative and work with a very capable team. I may never be able to build this in my company

I make a great living without any of the risks that come with starting a business or owning an established business

But I want to temper this enthusiasm too. Here are the issues I wrestle with when I think about quitting:

I’m not building a personal legacy yet. This blog is my first attempt at building a presence independent of my employer

I have a lot of support in a large company, will I be able to succeed on my own? Am I becoming too soft working for a large company, is the challenge gone?

I dont think the company can give me the level of ownership I want in the long run. This is mostly because of structural issues. So maybe I should quit

As you can see I keep wrestling with this question. And I dont have the answer right now. Perhaps I feel that time to quit is not now and that I will intuitively know when it is time to pursue something new. I’m coming up on 10 years at the same company now. I’ve been lucky to play different roles in this company every 3 years and that made these 10 years really wonderful.

As a product manager based in India, its great when your product gets centre stage at a worldwide conference in the US. This happened to me last week and it made all the struggles over the last 8 months worth it. But the real heroes are the engineers on the product team. They put in the real work to realise, shape and rationalise management’s vision. It was gratifying to send pictures and emails to the team letting them know that their work mattered and is being showcased at the highest level.

It’s only after shipping that the real analytical work begins as we try to answer the following questions:

Are people using the new features?

If yes, how much are they using it?

What amount of use makes an investment in a feature worthwhile?

Are these features bringing new users to the product? – If this was a goal for the release.

Are the new features usable?

I try to get question #5 answered via prerelease testing but all the other questions are truly answered only after the product reaches the customers hands. I know that #4 was not a goal for this release.

Q3 is always tricky. Sometimes you invest way more resources into a feature that you anticipated at the beginning of the cycle. This is just the engineering truth. You discover usability issues late or the engineering team discovers workflow issues that we did not think of before hand.

Sometimes you find a single bug or a performance issue that prevents you from shipping a feature that you worked on for 4-8 weeks. This is still not bad considering that only a few releases ago we would have wasted many more months of work since we were were following a waterfall approach. Every time I run into such situations I feel proud that we develop software incrementally and send it out to a set of users for testing every month. This validates our assumptions earlier and allows us to discover issues earlier. And, that is worth the pain the process changes bring with them.