skill is a marginal distribution of
the posterior joint distribution of a
multivariate variable that consists of
individual skills and individual performances. This posterior joint distribution consists of several factors
that are conveniently represented by
a graphical model, a way to represent
the information about which factors
depend on which variables. The marginal posterior distributions of skills
can be computed using standard
message-passing methods for inference in graphical models, such as the
sum-product algorithm. It is common
to approximate a marginal distribution of a skill variable with a distribution from an assumed family of distributions; for example, assuming the
family of Gaussian distributions, as
done in the TrueSkill rating system.
The approximate Bayesian inference
amounts to approximating marginal
posterior distributions of skills by
distributions from the given family of
distributions, assuming that marginal
prior distributions belong to this family of distributions.

Future Directions

Strategic game models of contests provide plenty of interesting hypotheses
about what strategic user behavior
may arise in different contest situations. Future work must be devoted to
narrowing the gap between theoretical
results and empirical validations. The
availability of online services whose
design is based on contests and the
collected data provides us with an opportunity to test the existing theories
and guide the development of new
contributions to contest theory. Another research direction is to study statistical inference methods for various
contest designs, such as in the recent
study of A/B testing for auctions. 6

While the skill-rating methodshave been studied extensively overmany years, some interesting ques-tions still remain open. Most skill-rat-ing methods represent an individual’sskill by a scalar parameter. In manysituations, however, it is of interestto consider an individual’s skill overmultiple dimensions; for example,an online worker may have differ-ent types of skills such as analyticalproblem solving, strategic businessplanning, and software programmingskills. Another interesting directionis to study statistical inference meth-ods for statistical models of rankingoutcomes that allow for a larger set ofunknown parameters. For example, inan online labor platform, a ranking ofjob applicants would depend not onlyon the idiosyncratic skills of the ap-plicants, but also on the specific jobrequirements, both of which may haveuncertainties. Another direction is todevelop solid theoretical foundationsfor individual skill rating based onobserved team performance outputs.Current statistical inference meth-ods used in practice assume simplemodels of team performance, such asthat a team performance is the sum ofindividual performances, which maynot always be valid in practice.