software

Every thought leader after Deming has extolled the virtue of measuring whatever we need to improve. I recently read this article – it suggests seven metrics to measure a recruiter’s performance. Many more articles and suggestions for building performance scorecards are published. We need some simple metrics that could be quickly ascertained without investing in specialized software etc. In this article we are exploring two such metrics to measure software recruiters’ performance and ways of improving the same. By definition a software recruiter specializes in hiring software professionals.

In this post we will focus on the recruitment process including recruiters, hiring managers, other members of the interview panels, recruitment consultants and agencies and candidates. Combined effect of their individual behaviors results in the inefficiencies of the recruitment process.

Here are some typical characteristics of the software job-seekers’ market. Candidates often claim a lot more in their resumes than their real “hands-on experience”. Recruiters – particularly those who are experts at Boolean search rely a lot on what is claimed in the resume and base their search on keywords and extrapolate an individual’s capabilities based on the companies worked for and the schools attended. The best way to separate substance from hype is by having a short telephonic conversation. Just a few questions would have the candidate himself telling where his or her real strengths are and what should be ignored.

At this stage let us introduce two metrics to measure the efficiency of a source such as a recruiter or an agency providing candidates.

Recall of a source measures its reliability or spread of coverage of the the total population of suitable candidates. This is tough to measure as we don’t know the “total population” of suitable candidates who are currently looking for a change. As a proxy we can replace the “total population” with “total known number” by adding number of suitable candidates sourced from all sources including employee referrals, direct applicants, agencies and recruiters.

Precision of a source measures how many suitable candidates were sourced as a percentage of the total number of candidates sourced. This shows what percentage of the sourced candidates were useful and what percentage of the sourcing effort resulted in “waste”. This is measured easily by taking a ratio of candidates who are found worthy of second interview over total number of resumes coming from the source.

Candidates sourced but not found suitable are called false positives – our effort on interviewing these candidates is wasted and needs to be minimized. Similarly candidates who were suitable but were not sourced are called false negatives– indicating lower reliability of the source in terms of its ability to find suitable resources.

The main reason for false positives is due to the fact that many recruiters and agencies are singularly focused on improving recall. Their intent is to improve the probability of finding a match by sourcing as many resumes as possible. This “spray and pray “approach results in a lot of wasted effort in interviewing false positives.

On the contrary if a recruiter applies a filter and reduces the total number sourced by having a preliminary telephonic round , it will reduce false negatives and improve precision. An upside of this approach results in a better deal for the hiring managers who have less interviewing but better results.

Majority of hiring managers believe that recruiters can’t really do any technical screening. Recruiters do “keyword” based search – not really going deeper to find out if the candidate really has the relevant technical skills. This results in a communication gap between the recruiters and the hiring managers. Hiring managers don’t think that feedback any more detailed than “Technically Unsuitable” would be understood by the recruiters.

We believe that recruiters can be trained to do preliminary technical screening. Some amount of guidance in the form of technical questions that weed out obviously unsuitable candidates can improve the recruiters’ ability to judge.

If we have more meaningful feedback coming more frequently; it will improve the precision and reduce wasted effort and interviewing fatigue. Smaller batch sizes would help get early feedback resulting in corrective action of improved technical filtering. Baby steps of small batches each one improving precision in an iterative way seems like the way we should hire technical talent.

Main Takeaway- Continuous Experimentation Well Beyond The Startup Stage

Contrary to the generally held belief that lean startup principles advice experiments in early stages of a startup; many speakers at the conference showed how they are experimenting continuously at all stages of their ventures.

Eric Ries in said that “Product market fit and experimentation is not a one time activity. It’s a continuous flow of activities. There are no discrete big jumps! Think of these steps in continuous flow that lend themselves to go back if an experiment fails”

Hiten Shah of Kissmetrics reiterated that a meaningful metric leads to a hypothesis and then to an experiment to validate it. Startups should always be A/B testing. Empirically 1 out of 5 tests succeed. Strive to win 1.67 out of 5.

A/B testing can help not only at different stages of a startup; but also for various activities including website traffic, app installs, welcome emails, Web/Mobile onboarding, E-mail digests, Triggered notifications, dormant/churned users.

Des Traynor also said that having continuous feedback is more valuable than one time event driven feedback.

Experiments helped even established brands like Rally, Google and Vox Media to validate hypotheses at later stages of their product lifecycle

Rally launched a dummy brand waffle.io targeted towards developers to protect the parent brand from the impressions created by the experiments. Finally Rally decided to have both brands.

Google Adsenses team validated Partner Problems using Lean Startup Principles. Blair Beverly said that they faced problem with new projects scaling too early and failing as they lacked historical data to go by. He got coworkers at Google ad senses team to use the Lean Startup. They scheduled office time to read the book, being helpful and not pushy. They also gave them a reading guide with questions. In the end they identified three hypotheses; put together templates like the partner problem hypothesis. People felt good about invalidating their own hypotheses as it saved them work that would’ve been wasted.

Vox Media launched Vox.com in 9 weeks using analytics to guide customer validation. Melissa Bell got her co founders and others from Vox media in the same room to get everyone on the same page about her vision. Many editorial staff came from Washington Post whereas Vox was an agile technology company. They used card stacks for flexibility. They had problems with the way editors used card stacks, as it was difficult to navigate-hence they analytics were used to solve the problem. Now Vox.com has 22m users. Delivering content to users where they are-on social channels such as Facebook or YouTube instead of own URL.

You need to eat your own dog food. Use your product to solve your own problems. If not you are at an enormous disadvantage.

Invert the org chart :customers & customer facing team should be on top. They should be heard and not told what to do.

Force yourself to pretend at the earliest possible moment what you want to be- to learn whether its worth being what you want to be. Landing pages, Concierge or Wizard of Oz are ways to pretend.

Don’t speed up for the sake of it. For startups not going fast enough is not the main risk. False summit is the reality. Journey of a startup is slow like that of a mountaineer. A new goal appears once you have reached what seemed like the ultimate goal.

According to Grace Ng success criterion for any experiment is the weakest outcome that will give you enough confidence to move forward.

Testing the riskiest assumption on buy side in a two-sided market place could be deceptive in a sellers’ market. Sellers may not automatically follow even if you find many buyers.

Validated hypothesis doesn’t necessarily lead to a viable business. Grace Ng tested a hypothesis whether birdwatchers will post photos to ask questions. The Hypothesis was valid but the problem turned out to be too small – not a big pain-point.

Don’t validate the solution before validating the problem. As in the case above; the problem was not big enough though the solution was right.

As you go through build-measure-learn cycles for product the same way you should be going thru build-measure-learn cycles for process compliance.

Beware of developers’ tendency to focus on how to do things than on outcomes. Developers tend to ignore security as they are dazzled by technology, so they focus on doing things faster-not safer. Security testing, threat model and risk metrics should be included right from the beginning and not at the end.

Don’t as two questions that kill breakthrough innovation – what is the roi? When do we get it? In order to answer these questions we have to look at existing markets which kills innovation -innovator’s dilemma. We need to build cultures or safety net for innovators.

Conclusion

Most of the takeaways and dos and don’ts are common sense for any practicing entrepreneur. According to Eric Ries The Lean Startup process is more widely practiced than talked about. Most entrepreneurs are agents of long term change. They don’t think The Lean Startup is a big deal. As with most profound thoughts- it seems obvious after its well thought through, well organized and well presented.

New Age Start-ups – Advantage India!

7. Variety and fragmentation? Indians have always been dealing with them.

As discussed in Chris Andersen’s “The long tail” businesses have to deal with millions of players with their unique offerings competing for a part of the pie. Niche offerings and the internet’s ability to address the “audience of one” has resulted in fragmentation of every market. India has always been a fragmented market given more than 26 languages, hundreds of sub cultures with their own religious customs. Hindus believe in worshiping a different deity (God or Goddess) depending on the purpose or day of prayer. Even food habits demonstrate the same variety.

It’s time Indian entrepreneurs realized that they have these special advantages while competing with others in general and the western entrepreneurs in particular.

New Age Start-ups – Advantage India!

6. Indian entrepreneurs are used to a crowded world where one has to compete for everything.

Startups have to compete hard to get a part of the pie. Indian entrepreneurs are used to fierce competition from childhood days. Its not easy getting into a good primary school in India. Parents have to prepare their children for months for the test/interview conducted by recognized primary schools. This is the first taste of competition for the child which continues to develop through high-school , college and finally in the job-market and beyond. Its very difficult for startups to get funded. Recent business plan competition by The Economic Times had sixteen thousand participants out of which only a handful got funded. The economy of shortages has forced Indians to find innovative and efficient ways of using scarce resources. Fierce competition will eliminate those who don’t.

New Age Start-ups – Advantage India!

5. Indian entrepreneurs are more comfortable in communities than corporations

In my earlier blog “Bottom up, employee driven corporate metamorphosis” I have extolled the virtues of communities in this age of crowdsourcing and how they are more valuable than corporations. Corporations are a western phenomenon. Deep hierarchies in the armed forces and Government were introduced by the British in India. Typically an Indian village was ruled by a group of elderly villagers called Panchayat. Spread of the Hindu religion happened in a viral way with many Gurus and saints acting as change agents propagating the message of their religious belief unlike Churches that had a well organized hierarchy managing them. More recently communities have been organized as co-operative societies in India. Amul – a dairy co-operative is competing well with international brands like Nestle and Kraft. Co-operative societies organized by communities dominate banking, sugar, poultry and dairy business in India. Most of the trading businesses are managed by communities that specialize in the specific merchandize being traded. It will be interesting to see how these traders compete with large retailers like Wal-Mart.

Starfish and Spider
The Starfish and the Spider argues that organizations fall into two categories: traditional “spiders,” which have a rigid hierarchy and top-down leadership, and revolutionary “starfish,” which rely on the power of peer relationships.

The Black Swan
For author Nassim Nicholas Taleb, black swans underlie almost everything about our world, from the rise of religions to events in our own personal lives.

The Four Steps To The Epiphany
The book offers the practical and proven four-step Customer Development process for search and offers insight into what makes some startups successful and leaves others selling off their furniture. Rather than blindly execute a plan, The Four Steps helps

The Lean Startup
The movement that is transforming how new products are built and launched

The World is Flat
The World Is Flat 3.0 is an essential update on globalization, its opportunities for individual empowerment, its achievements at lifting millions out of poverty, and its drawbacks–environmental, social, and political, powerfully illuminated by the Pulit

The Year Without Pants
Automattic, Inc., whose 120 employees work from anywhere in the world they wish, barely use email, and launch improvements to their products dozens of times a day. With a fraction of the resources of Google, Amazon, or Facebook, they have a similar impact