Manipulating javascript can overcome publishers' software.

Share this story

Advertising pays much of the budget for most online publishers, making the growth of adblockers an existential threat. As such, adblocking has set off a software-based arms race, with publishers finding software solutions that keep ads appearing or entreat people using adblocking software to white-list them. Adblockers readily respond with modified software that targets these specific responses, triggering the publishers to try again.

Some academics have recently stepped into the middle of this arms race, performing an analysis that allows them to identify the specific methods used by publishers to avoid having ads blocked. And the team has gone on to try a couple of different approaches, both of which modify a webpage's contents to keep the anti-adblocking software from having an effect.

Outside of the economics of it all, there's an interesting computer science problem here. The code on the webpage is attempting to identify software present on a user's browser. How do you recognize when that's happening, and how can you possibly intervene?

The adblocking wars

The approach the researchers took involved following code execution as a browser loaded and displayed the page. This was done with a modified version of Google's V8 JavaScript engine, one that allowed them to extract information about the downloaded code that was being processed and executed as the webpage loaded. By doing this with and without an ad blocker installed, they were able to identify differences in the code that was executed when ads were displayed or blocked.

As they note, typical anti-adblocking code might wait for the page to load and then check on the size of an element that's meant to contain an ad. If the ad isn't loaded, this area will never get defined, and its size will end up either being undefined or zero. This allows the code to perform some other action, like putting up an alternative ad or displaying a dialog to ask for the adblocking software to be disabled.

By following code traces, the authors could look for conditional tests—things like "is the size of this element 0?"—followed by execution of different code depending on whether an adblocker is present. By examining the code at that location, they could determine which condition was being tested for.

On its own, this provided an indication of just how prevalent anti-adblocking software is. The authors claim to have found an anti-adblocking response on more than 30 percent of the Alexa Top-10,000 websites, but it's somewhat more complicated than that. In many cases, adblocking software was detected, but there was no visible response; the software simply logged the presence of the adblocker, often through Google analytics.

Setting the software loose on webpages that normally don't show ads indicated it didn't produce any false-positive identifications. And a test of more than 400 sites known to use anti-adblocking software showed that it was more than 85 percent accurate at identifying them.

The false negatives came about for a variety of reasons. One of these is simply that Javascript has a variety of mechanisms by which programmers can test for specific conditions, and the team didn't trigger their analysis on all of them. The second is just random variability; each page was loaded six times, three of them with and without adblocking. Random differences among these, like slower or faster loading of some page components, could obscure the tests for the presence of anti-adblockers. There was at least one approach that the software missed entirely: it loaded a warning message about adblocking, then tried to load an ad on top of it; if the more complex one was blocked, the warning showed.

Intervention

WIth that success in hand, the authors decided to enter the arms race on the side of the adblockers. Since they knew what condition was being tested to determine whether an adblocker was being used, they could intervene in the page's JavaScript in a way that forced it to execute the adblocker-free branch of the code. This is relatively simple to do on the code side by simply rewriting the JavaScript so all the relevant branches do the same thing. Rewriting, however, required the installation of specially modified proxy software on the same computer and redirecting all the browser's requests so they went through this software.

This approach had a success rate of more than 80 percent on the websites it was tested with. And, despite the potentially significant mangling of the underlying code, only one site showed a visual defect.

An alternative approach they tried was somewhat more precise. Since they could identify the condition that was being tested for, they could modify the variables used by the site so that the condition would always evaluate as if an adblocker was not present. This only requires a browser extension. And, in the 15 websites it was tested on, it worked every time.

Motivation?

The authors are very upfront about their motivation for this work: "We want to develop a comprehensive understanding of anti-adblockers, with the ultimate aim of enabling adblockers to be resistant against anti-adblockers." They cite user privacy and security as the reason for choosing a side in the arms race, but it's not clear that their approach makes much sense in this regard. Running everything through a modified proxy or manipulating page-wide variables would seem to create a whole host of privacy and security risks on their own. In addition, it's not clear how blocking the mere logging of the existence of an adblocker, which their software would do, helps anyone.

And they admit that, as soon as publishers are aware of the methods they use to test for anti-adblocking software, workarounds will be possible. This could be as simple as finding a means of searching for an adblocker that won't be picked up by the researchers' approach. Or it could involve intermingling the code for the adblocking test with code that's essential for the page to work. Or it could involve re-using the variable that's manipulated by the researchers' software. Any of these, and presumably other approaches, would all work.

Finally, the researchers seem to be actively avoiding considering the consequences. Part of their introduction states flatly that "Adblocking results in billions of dollars' worth of lost advertising revenue for online publishers." And their own analysis confirms that the majority of the sites running anti-adblocking software are producing news. If they're aware that the success of their goals will involve crippling a lot of news sources, it's not apparent from this paper.

Great, but is anyone researching a real solution? Like dumping JavaScript in ads entirely. Running 3rd party JavaScript you have no control over on your site is ridiculously insecure. It's gotten to the point where users feel they have to run ad-blockers to avoid malicious code.

There is no practical or functional reason to not restrict ads to images and videos, there is really no difference between clicking and ad and doing whatever in the ad and clicking the ad taking you to the advertisers page to do the same thing.

It all comes down to the same issue in the end, the more insecure and annoying these ads are, the more people install blockers and anti-blocker blockers. The industry needs to get a clue and self-regulate, before this isn't an industry anymore.

IMO it's too late for online ads. Advertising companies could change their ads to be what everyone wants. (Static, noiseless, non-page breaking, etc). People will still continue using adblock. Not sure what the solution is, since I don't want to pay a "subscription" fee to view content either...

The biggest issue I have is that my company blocks dubious content, so most ads get blocked by default. Some websites detect this and will prevent me from accessing the content until I disable the blocker, which I'm unable to do.

The solution for me is to disable the site's JavaScript. Most of the site still works, and the ads stay blocked.

If publishers want us not to block ads, they should start with contracts which penalize advertising networks that serve malware. Less intrusive ads would help, too.

Pretty much this. The problem isn't ads, it isn't ad blockers, and it isn't anti-ad blockers. These are symptoms of the problem.

The problem itself is the entire advertising "model" used on the internet puts the casual browser into what is basically a compulsory game of Russian Roulette- made worse by the fact that- at least in Russian Roulette, you have a pretty good idea who pulled the trigger.

Advertisers and others that get revenue from them have no incentive to change it because the entire thing is tilted grossly in their favor.

Sadly, none of this will get solved as long as there are stupid people out there. They are the ones who click on the intrusive and malware-infested ads (if they didn't, the ads wouldn't persist.) They are the ones who also support the spam industry. They are why we can't have nice things.

One minor irony here is that Conde Nast (so Ars, by extension) hands us some of the most obnoxious ads I see in my daily surfing. (That's on my work computer - at home, uMatrix takes care of 99% of ads just by blocking some Javascript.) I don't understand why Prada and the latest perfumes keep popping up on Ars for this 40-something male - at least the super-obnoxious LG ads are a little more on-target.

And all this in formats including:

* A gigantic banner ad that expands from the top to push all of the content down about half the screen height just as you're about to click on something

Finally, the researchers seem to be actively avoiding considering the consequences. Part of their introduction states flatly that "Adblocking results in billions of dollars worth of lost advertising revenue for online publishers." And their own analysis confirms that the majority of the sites running anti-ad blocking software are producing news. If they're aware that the success of their goals will involve crippling a lot of news sources, it's not apparent from this paper.

Well, in all honesty, the third paragraph does mention that this subject is an interesting computer science problem. And since this work is done by academics, one might imagine that the social consequences are secondary to the question being researched.

That said I have to side towards the first poster, Bob.Brown , and admit I use a lot of ad-blocking and scripts that lessen my exposure to malware.

I don't disagree with the article quote of "Adblocking results in billions of dollars worth of lost advertising revenue for online publishers" but I'm curious what the other side of the coin looks like. How many dollars are lost on businesses and individuals hit with variants of the cryptolocker virus? What happens when you add in other types of malware? How much money is lost by businesses due to beefing up infracture to defend against this? How many man hours of productivity are lost and what does that add up to. How can you quantify a dollar amount around grandmothers precious files being crypto-locked forever?

It's a balancing act. The company I work for lost hundreds of thousands of dollars over a number of cryptolocker events. A lot of these came though advertising vectors (there's a whole TL;DR there, so I'll leave it as a statement and not explain here for the sake of brevity).

I run adblockers to be safe. The exception being this site where I sub in order to pay it forward as far as content is concerned. I should whitelist more, but I'm skittish seeing real damage having been done first hand in my line of work.

[Note: As a sub two adblockers report zero ads, flash control reports no flash, yeah. Ghostery reports some trackers, but that's a horse of a different color)

Assuming that ads get me to buy products that I normally wouldn't, running an adblocker saves me hundreds, if not thousands of dollars a year.

Missing in both the original research and the Ars article is the ergonomic affects on end-users. I simple cannot stand seeing a web page with animations. It literally makes me nauseous. Some of us need to use adblockers to be able to use the internet.

I don't mind paying for content. I do mind needing to set up an account and give out/maintain payment information for each site/service individually.

If many, many sites can use the same ad networks, why aren't there also widespread subscription networks? Let me pay one fee per month for access to the sites I want. I'm specifically NOT asking for or advocating cable-TV style bundles; just a central point where I can choose the sites I want to pay for and maintain one payment account.

I'm also not asking for or advocating single-sign-on. A federated system where by Ars could check that my Ars-Account has a current paid status would be ideal.

For sites that I don't care to pay for, feel free to lock me out if I decline ads. That's fair.

. Running everything through a modified proxy, or manipulating page-wide variables would both seem to create a whole host of privacy and security risks on their own.

I would trust a proxy I chose and installed over the unvalidated ads that ad networks serve. Until the ad networks are clear of malware, ad blocking is the only logical solution. I feel a little sorry for the publishers, but they are the ones choosing to use automated ad networks. Malware through ads has hit every big site, including Ars, and they should all be ashamed they are putting profits above the safety of their readers.

Are the advertisers and publishers who want ads served up putting in the same effort to prevent the spread of malware via ads as they are to bypassing ad blockers? Why is it that they seem to ignore the primary reason many of us block ads? Why are they at war with users and not at war with purveyors of malware?

Makes one miss the internet of old, when the worst ads were pop-ups that could easily be blocked and the majority was a bunch of gifs on the upper part and either side of the page.Nowadays, it's all about 5 layers of clickjacking, auto playing videos and "Click here to close" that redirect to another site.

Are the advertisers and publishers who want ads served up putting in the same effort to prevent the spread of malware via ads as they are to bypassing ad blockers? Why is it that they seem to ignore the primary reason many of us block ads? Why are they at war with users and not at war with purveyors of malware?

When has paying attention to the consumer's needs ever done anything meaningful in business?

Though in some seriousness, advertising on the internet has been a wonderful, sparsely-regulated and (most importantly) low-cost advertising medium for so long that none of them wants to give in to the idea that spending more money to make it safe for the end users might be in order.

a) I can be reasonably sure I'm not going to get served malware by malicious adsb) Websites stop making their site unreadable trash with autoplaying video, massive ads, interstitial ads, and modal dialog boxes

Wake me up when both of those things happen. I'll just wait over here.

A few sites asked to turn off ad-blocking on their site. I usually often comply with that request (one click with uBlock Origin)... but often that makes site harder to use, and sometimes even night unusable - then I turn ot on again.

IMO it's too late for online ads. Advertising companies could change their ads to be what everyone wants. (Static, noiseless, non-page breaking, etc). People will still continue using adblock. Not sure what the solution is, since I don't want to pay a "subscription" fee to view content either...

Agreed. Many people use adblockers because the ads make pages and computers run so slow.

There are other offenders like autoplay ads with sound. And pop ups. But the driving forces for me is how long the ads take to load, that they are always loading/never stop loading, and how much my machine slows down.

Unless you’re into affiliate marketing, online ads are worthless. Do enough saps seriously click and buy through those links to make it viable? If so, that’s our fault, not the advertisers. Where’s the FCC now? Oh that’s right, they aren’t worried about consumer protections.

IMO it's too late for online ads. Advertising companies could change their ads to be what everyone wants. (Static, noiseless, non-page breaking, etc). People will still continue using adblock. Not sure what the solution is, since I don't want to pay a "subscription" fee to view content either...

Your "it's too late" assessment is absolutely correct. I remember the peaceful days of static banners and the occasional GIF. But once they let advertising get out of control with Adobe Flush pegging CPU's, malware hijacking systems, popups and popovers, autoplaying videos, and all other forms of ruining the internet experience, the stage was set. Ad blocking has now become default behavior for many many millions of computer users because they value their sanity and safety. If it had never gotten out of control, we wouldn't have needed to fight back in the first place.

I see lots of comments here on malware. That's half the issue for me. The other half is tracking. I realize it's a lost cause but I will not stop fighting against it until they pry my add-ons from my cold, dead hands. You have no business keeping a profile on me and tracking what sites and information I peruse without my permission. Period. End of statement.

The big problem for ad companies is that mainstream consumers are simply getting use to how great the internet is with robust ad blocking. Pretty soon it will feel normal. That is the big danger. Similar to people cutting the cable the problem isn't just the small numbers doing it today it is the stats that show that one someone cuts they never go back which makes growth of cable subscribers nearly impossible over any extended period of time.

The whole race to the bottom in terms of abusive, intrusive ads (and ars is no better than anyone else in this respect) has poisoned the well.

While ads were not intrusive most consumers just accepted them but once they got bad enough for consumers to make the conscious choice to block them that was the cutting the cable moment. Website owners (ars included) have likely lost those consumers forever. Most will never come back under any circumstances.

The ad industry has turned internet advertising into a dangerous, obnoxious, annoying time-wasting mess and increasingly people have reacted with adblockers. It's not rocket science why people use an adblocker. I've had enough of these ads and have no sympathy left to give them.

If many, many sites can use the same ad networks, why aren't there also widespread subscription networks? Let me pay one fee per month for access to the sites I want. I'm specifically NOT asking for or advocating cable-TV style bundles; just a central point where I can choose the sites I want to pay for and maintain one payment account.

In the french web, such a model exists. it's called lapresselibre.fr ("the free press.fr")it works just like that, you pick the news site or subscription based sites you want to support, get a discount the more you pick, and give monthly money to a single point.

you then get access to the subscription only content (or an ad-free experience) on all of the chosen sites.

It was made to take advertising out of the editorial loop for news sites.It does seem to work out, economically.

Oh not just malware, how about the sites/ads that use your energy to mine cryptocurrency in the background? I’d be curious to see a more detailed investigation on the true prevalence of that, particularly after this recent hype and inflation.

The biggest issue I have is that my company blocks dubious content, so most ads get blocked by default. Some websites detect this and will prevent me from accessing the content until I disable the blocker, which I'm unable to do.

The solution for me is to disable the site's JavaScript. Most of the site still works, and the ads stay blocked.

My workplace (NHS/UK public sector) still uses Windows 7 and the latest version of IE available for that platform, but had to install Chrome (why not Firefox? Who knows?) as some crappy training package they paid for wouldn't work in anything that wasn't produced this decade. Anyway, a little while ago I found the IT department had started enforcing uBlock Origin on the Chrome install, something both surprising and welcome as looking up healthcare information is usually a guarantee of being bombarded with a large amount of ads. If even large-ish corporate IT departments are blocking ads to increase network security, the ad companies might as well just give up.

I don't mind paying for content. I do mind needing to set up an account and give out/maintain payment information for each site/service individually.

If many, many sites can use the same ad networks, why aren't there also widespread subscription networks? Let me pay one fee per month for access to the sites I want. I'm specifically NOT asking for or advocating cable-TV style bundles; just a central point where I can choose the sites I want to pay for and maintain one payment account.

I'm also not asking for or advocating single-sign-on. A federated system where by Ars could check that my Ars-Account has a current paid status would be ideal.