The researchers studied the impact of the annoying ones [ads] on a group of people paid to carry out the e-mail categorization task, versus two other groups doing the same, who were shown non-annoying ads or no ads at all.

People doing the task looked at one e-mail per page, and each page had two banner ads on either side – either two annoying ones, or two that were not annoying – or just white space in the margins.

The researchers experimented with different levels of pay per e-mail the participants reviewed; unsurprisingly, people who were paid more looked at more pages. But in every case, people shown annoying ads looked at fewer pages than people in the same pay scale who were shown non-annoying or no ads.

The math tells a troubling tale: Looking at the behaviours across pay scales, people in this experiment who saw bad ads had to be paid .115 cents more per page to match the level of work done by someone shown good ads, and .135 cents more to match the level of someone shown no ads at all.

That may not sound like a lot (the pay per-page for the task was relatively low because it took so little time to do), but when expanded to the cost per thousand impressions (CPM or cost-per-mille) – the metric by which ads are commonly sold – it is significant. It comes out to $1.15 CPM difference between bad ads and good ads. Considering that the CPMs of many online ads can be very low, depending on a number of factors, the revenue they provide to publishers may be outstripped by users dropping off their sites.

The figure above shows the estimates of a model the researchers fit to the data.

On the x axis we see how much people were paid to classify an email. The three pay conditions were .2, .4 and .6 cents per email.

The y axis shows the number of emails people actually to classified before quitting.

The colored lines show the advertising conditions people were randomly assigned to. The red condition saw annoying ads, the blue condition saw harmless ads, and the green condition saw no ads. The graph shows that the more people are paid, the more emails they classify, but the more annoying the ads, the fewer emails they classify.

A little algebra shows that you need to pay people more than a dollar CPM (cost per thousand impressions) to make up for the dropout caused by the annoying ads.

Interestingly, websites are typically paid less than $1 CPM to run annoying ads.