Leaderboard

Leaderboard formula

Submissions will be evaluated according to classification accuracy in each of the 3 typing conditions. Classification accuracy (ACC1) is the proportion of correctly labeled samples. For each submission, the following formula is used:

For each participant, the highest ACC1 in each condition (both, left, right) is taken over all submissions.

A rank is determined for each condition. Each participant receives a rank for each condition.

The leaderboard is determined by the sum of ranks. Thus, the best score is (1 + 1 + 1) = 3 for having the highest ACC1 in each condition. This would achieve first position in the leaderboard. Ties are broken by the median ACC1