Google Web Accelerator significantly reduces the time that it takes broadband users to download and view web pages. The Google Web Accelerator appears as a small speedometer in the browser chrome with a cumulative "Time saved" indicator.

Here's how it works. A user downloads and installs the client and begins browsing the web as she normally would. In the background, the Google Web Accelerator employs a number of techniques to speed up the delivery of content to users.

Looks like some of the Mozilla hires are paying dvidends.

mrMister

10:52 pm on May 4, 2005 (gmt 0)

If so then it could be fairly trivial to block it.

deny from 72.14.192.0/20 added to the rest of banned IP's works well ;)

Does this app pre-fetch every link that it sees?

I can't believe Google would do that?

I can't see ISPs being too happy about this, It's a tremendous waste of bandwidth. With Google's massive market share and user base this thing has a serious chance of slowing down the Internet as a whole.

If so, I think we should all make a stand and block this nuisance. If enough of us do it, then Google will see sense.

madmatt69

11:01 pm on May 4, 2005 (gmt 0)

Agreed.

First auto-link and now this. Soon they'll have a full-scale webmaster revolt on their hands.

Brett_Tabke

11:03 pm on May 4, 2005 (gmt 0)

> proxy

I don't think it can be used as any type of general proxy to do any auto site ripping. The best you could do, would be to drive IE as a spider (which is trivial to do at current and has nothing to do with Google).

So before we go down this road of lynching them, lets be sure what the issues and consequences are for us. I don't think we can see those yet and there is a disturbing trend to disinformation and misinformation in this thread. No one has yet detailed how the "prefetch" works and Google is ambiguous about it on their site.

incrediBILL

11:16 pm on May 4, 2005 (gmt 0)

Food for thought...

If you currently let Google cache your pages, and Google caches your entire site, someone can already rip most of your site direct from Google cache without you even knowing about it.

As far as this new "Web Accelerator" goes, most of my sites and my customers are ecommerce and if cookies don't work there will be a buttload of screaming about empty carts. I'm tortured with using the knowledge of how to block the Web Accelerator or just tell my customers that "It's a GOOGLE issue with the Web Accelerator not handling cookies properly, sorry - nothing I can do, write GOOGLE and see what they say" and stand back and watch the flames.

pontifex

11:17 pm on May 4, 2005 (gmt 0)

brett, very nice find, thanks!

A question: did you analyze the ORDER of which the links of a page are pre-fetched? I guess this would say a lot about how google sees a page and the importance of links... first link fetched = most important link on a page?

just another idea :)

P!

Brett_Tabke

11:19 pm on May 4, 2005 (gmt 0)

incrediBILL, it isn't about the site ripping. We all know there are people ripping regularly through the three engines Caches.

This is about bandwidth usage with no direct reimbursement for that usage.

incrediBILL

11:22 pm on May 4, 2005 (gmt 0)

Not to mention the fact that pre-fetching will just ad more hysteria to the AdSense crowd already posting bi-weekly at a minimum "Why doesn't my AdSense impressions match my web stats?" so add to the list pre-fetched pages cached but never displayed don't display AdSense.

mrMister

11:22 pm on May 4, 2005 (gmt 0)

It annoys me that they've specifically marketed this at users of broadband connections.

I've never come across anyone that's complained about broadband being too slow for web page viewing!

I can't really see any point in speeding up an already adequately fast connection, especially if it puts a drain on my network's bandwidth as I'm doing it.

I can't see admins of corporate networks being too happy about it either. If they have to go and block this at firewall level and then uninstall each copy from the machines of everyone using it.

incrediBILL

11:25 pm on May 4, 2005 (gmt 0)

incrediBILL, it isn't about the site ripping. We all know there are people ripping regularly through the three engines Caches.

I'm aware of the issue - just thought it might bear pointing out as numerous people mentioned this as a new vehicle for ripping pages.

I'm more concerned with what the visitor will see if I block Google's Web Accelerator that it could negatively impact my traffic if it invokes any secondary action on the part of the visitor.

My site is also dynamic so bandwidth is the LEAST of my problems if Google overloads the CPU it will all come to a screeching halt.

[added thought]

Ah crap, if it's not handling cookies right kiss all that affiliate income away....

[edited by: incrediBILL at 11:59 pm (utc) on May 4, 2005]

chrisk999

11:26 pm on May 4, 2005 (gmt 0)

Is this pre-fetching going to auto-click on lots of embedded ppc links in feed-based websites? (i.e. sponsored links hardcoded in on the server side).

I can't imagine how the application could filter out such links, especially if they are masked by a local redirect (e.g. example.com/rd?sponsoredlink1).

Could be expensive for advertisers?

GaryK

11:27 pm on May 4, 2005 (gmt 0)

There's a post at Search Engine Watch which states that, when Google prefetches a page, webmasters will see a Google user agent in their user logs.

But there's no mention of what the user agent is.

Has anyone here figured it out yet?

I don't want to panic, yet. I just want to know more about this thing before making a decision on banning it or not.

philaweb

11:27 pm on May 4, 2005 (gmt 0)

Well, I just installed and uninstalled this new geek toy.

Surfing about 15 bookmarked pages saved me 10 seconds of download time - so what? I have cable (1 meg), I'm paying a monthly fee for being connected 24/7. I have no beef with time consumption.

On the other hand... Some people have cable charged per minute, some people have wireless GSM (mobile) connections. For those people I can see the point of saving on time.

Perhaps this thingy isn't a shortsighted project just for now, but for when we all get Wi-Fi?

mrMister

11:36 pm on May 4, 2005 (gmt 0)

On the other hand... Some people have cable charged per minute, some people have wireless GSM (mobile) connections. For those people I can see the point of saving on time.

People payting by the minute will end up being charged more by the minute if enough users start wasting bandwidth by using prefetching.

The ISPs will see their bandwith requirements rising and therefore the costs will rise and that cost will be passed on to the customers in the form of increased charges.

It doesn't matter what pricing structure your ISP uses. there's no getting away from the fact that wasted bandwidth is wasted money somewhere along the line.

schwartz

11:36 pm on May 4, 2005 (gmt 0)

I use/manage/build many sites in a cms called "webgui". It has a feature which maker says may do well with this issue:

"WebGUI has a built in feature to "prevent proxy caching" in the settings. Basically it adds a random string of crap to the end of the URL to make sure that each page load is unique. It basically circumvents the proxy server for everyone and gives you really ugly URLs to boot. Unfortunately you can never trust what proxy rules any given proxy server might use for caching content, so this is the one consistent way that you can guarantee will work. "

chrisk999

11:41 pm on May 4, 2005 (gmt 0)

Just tested a price-comparison site of mine, which charges advertisers on a cpc basis.

After visiting the page and waving the mouse around a bit, it logged clicks on nearly all advertisers' links. Bad news for advertisers like myself.

The ppc companies that offer server-side ppc integration scripts will have to quickly set the new Google IP to generate $0 clicks before publishers get a one-off windfall.

GaryK

mrMister: It sure seems like Google is going out of its way to obfuscate this issue. According to the article at SEW, Google's Marissa Mayer was quoted as claiming a Google user agent would be used.

Mods: Am I allowed to post the URL to the page I'm talking about on SEW?

karmov

12:02 am on May 5, 2005 (gmt 0)

For me the big issue isn't so much the bandwidth consumption (though that will be more than a little annoying), but rather the fact that as Brett's shown, there doesn't seem to be any identifying fingerprints to this thing. This is a pretty significant issue for log file analysis. I'm not a tracking addict, but I like to have a good idea of what's going on with my traffic. I really hope there's some resolution to this issue. Much testing to do I suppose :)

mrMister

12:03 am on May 5, 2005 (gmt 0)

OK, I've done a limited bit of research on this little Web Corrupter of Google's. Heres my findings...

It doesn't take note of the last modified date of the page. if the page has been modified since the last cache, Google will still show you the cached version.

It doesn't obey robots.txt

It is capable of adding items to shopping baskets without you having to click on anything.

incrediBILL

12:04 am on May 5, 2005 (gmt 0)

Basically it adds a random string of crap to the end of the URL to make sure that each page load is unique.

Not an option if you run AdSense as I had to yank SessionIDs from the URL to get their contextual targetting to work properly.

if the page has been modified since the last cache, Google will still show you the cached version.

Deadly for dynamic sites and ecommerce.

It is capable of adding items to shopping baskets without you having to click on anything.

Fatal for ecommerce sites as that will invoke customer hysteria about site security: "I see someone else's products in my cart!"

The Contractor

12:09 am on May 5, 2005 (gmt 0)

I don't think it can be used as any type of general proxy to do any auto site ripping. The best you could do, would be to drive IE as a spider (which is trivial to do at current and has nothing to do with Google).

Uhm...have you ever looked at the plug-ins available for ripping sites from within firefox? They use your IP and UserAgent (unless you change the user-agent), so yes it would allow "very fast" and easy site-ripping behind a proxy.

I'm not trying to spread misinformation, but I fail to see one good point of this "for the user or site owner" - can someone mention one please?

Lemme see - it's meant for broadband user (which most are already able to download at 1-4Mbs). No gain here.

Doesn't work with cookies - there goes affiliate income for some, login info for others, and broken shopping carts for others (or adding products to a cart from prefetch).

Prefetch - Clicks/registers clicks and downloads a page you may not even vist. Great for small PPC or bandwidth for sites not even being visited.

Sorry, but I have yet to see a benefit mentioned?

[edited by: The_Contractor at 12:10 am (utc) on May 5, 2005]

mrMister

12:10 am on May 5, 2005 (gmt 0)

if the page has been modified since the last cache, Google will still show you the cached version.

Deadly for dynamic sites and ecommerce.

Just what I was thinking, especially in conjunction with my 3rd point.

In theory, it could add an item to your shopping basket, but it will show you the cached version of your on-page shopping basket and you'd be none the wiser unless you double check your basket contents on the checkout page.

philaweb

12:10 am on May 5, 2005 (gmt 0)

Hmm... With this accellerator thingy enabled it seems that no matter how obscure a search query I try on Google, the first three results are pre-fetched for me instantly.

I haven't clicked them, but they are in my accellerator cache, which can be seen when enabling the "Highlight Links to Prefetched Pages" feature.

mrMister

12:14 am on May 5, 2005 (gmt 0)

I'm not trying to spread misinformation, but I fail to see one good point of this "for the user or site owner" - can someone mention one please?

Tch, haven't you been reading...

Benefit for the user: You can really get back at that nasty system administrator at work by telling all your colleagues about the Web Corruptor. he'll soon be doing overtime trying to sort out the bandwidth problems!

Benefit for the site owner: Automatic clicks on your site ads, automatic additions to your customers' shopping baskets!

;-)

incrediBILL

12:19 am on May 5, 2005 (gmt 0)

I've already fired off an email asking them how to officially disable the Web Accelerator from hitting my site without impacting my visitors experience with intermediate failure pages. Got an instant canned reply about them putting their energy into making Google Web Accelerator better so I probably won't get a personal reply.

Considering I don't want it hitting my server or interfering with my visitors why would I care if they are busy making it better? I WANT MY PERSONAL REPLY! :)

Have I mentioned just how smart Google's PhDs are lately?

Just thought it beared mentioning all that money Google spent on hiring only those with the highest education, the supposedly bestest and brightest, and how it's paying off right now making the world a better place.