Random thoughts of a computer scientist who is working behind the enemy lines; and lately turned into a double agent.

Thursday, February 23, 2012

Crowdsourcing and the end of job interviews

When you discuss crowdsourcing solutions with people that have not heard the concept before, they tend to ask the question: "Why is crowdsourcing so much cheaper than existing solutions that depend on 'classic' outsourcing?"

Interestingly enough, this is not a phenomenon that appears only in crowdsourcing. The Sunday edition of the New York Times has an article titled Why Are Harvard Graduates in the Mailroom?. The article discusses the job searching strategy in some fields (e.g., Hollywood, academic, etc), where talented young applicants are willing to start with jobs that are paying well below what their skills deserve, in exchange for having the ability to make it big later in the future:

[This is] the model lottery industry. For most companies in the business, it doesn’t make economic sense to, as Google does, put promising young applicants through a series of tests and then hire only the small number who pass. Instead, it’s cheaper for talent agencies and studios to hire a lot of young workers and run them through a few years of low-paying drudgery.... This occupational centrifuge allows workers to effectively sort themselves out based on skill and drive. Over time, some will lose their commitment; others will realize that they don’t have the right talent set; others will find that they’re better at something else.

Interestingly enough, this occupational centrifuge is very close to the model of employment in crowdsourcing.

In crowdsourcing, there is very little friction in entering and leaving a job. In fact, this is the key crucial difference with traditional modes of employment: There is no interview and the employment is truly at will. You want to work on a task? Start working. You are bored? Stop working. No friction with and interviewing and hiring process, and no friction if the worker decides to stop working.

As in the case of Hollywood and academia, the evaluation is being done on-the-job. While currently the model is mainly applied to small tasks, there is nothing that fundamentally prevents this model from being applied to any other form of employment. With the Udacity and Coursera model, we start seeing that concept being applied to education. Later on, we may see other jobs adapting this model for their purposes (stock trading, anyone)?

What you observe in such settings is that the distribution of participation and engagement is heavy-tailed, tending to follow a power-law: A few participants will provide a significant amount of input, while there will be a long tail of participants that will come, do a few things (complete HITs on MTurk, write Wikipedia articles, watch lectures and homeworks in Coursera, trade stocks, pick your task...) and then leave.

What does it mean to have a power law distribution of participation in crowdsourced projects?

It means that the long-tail of the occasional participants is just not naturally attracted to the task. The persistent few are the good matches for the task. This is self-selection at its best.

No interview needed, and only the people that are truly interested stick around.

Crowdsourcing is the new interview.

The selection of the best participants happens naturally, without the artificial introduction of a selection process mediated through am interview. The interview is an artificial process. It tries to keep out from the task the participants that are not qualified and tries to identify the ones that are the best. This is an imperfect filter. It has false positives and also false negatives. Many people are hired with great hopes, just to be later proven to be ill-suited for the task (false positives). And many good people do not get the chance to work on a task just because they do not look good on paper (false negatives; I am dying to make a Jeremy Lin joke here...)

Think now of an environment where everyone gets a shot to try working on something they are interested in. No friction of getting hired and getting fired. You have a benefit where the best people work on the tasks that they are best at.[You ask what if there are fewer dream jobs than available labor? What to do when training on the job is not possible (cough, doctors, cough). Let me dream for now, and let's bury under the carpet the millions of details need to be addressed before this mode of operation has a shot in becoming reality.]

To answer the question posed at the beginning of the post, "Why is crowdsourcing so much cheaper than existing solutions that depend on 'classic' outsourcing?" The process of self-selection in matching workers and tasks is the key reason on why crowdsourcing is typically cheaper than the traditional process of assigning directly tasks to people. The easier it is for the crowd to find jobs they like, the more efficient the matching and execution.

When you effectively have the most interested and self-selected people working on a given task, the productivity of a team for the task is much higher than the performance of a team consisting of people that may simply be bored or not very interested in the task. Just consider the productivity of five programmers that are dedicated and enthusiastic about what they are building, compared to a similar team of five programmers that were assigned the task by someone and they have to implement it.

At oDesk, there is a significant effort to improve the matching process of projects and contractors, by showing to contractors the best projects for them and to employers the best contractors for a task. My own dream is to be able to eliminate the friction of interviewing and get the process of finding a job and working to be as seamless as possible.