Forcing Gzip Compression

Tony Gentilcore’s Beyond Gzipping presentation at Velocity 2009 (http://en.oreilly.com/velocity2009/public/schedule/detail/9072) identified that ~15% of Internet users receive uncompressed responses. This is mostly due to proxies or client security software stripping or mangling the Accept-Encoding header from the web browser.

RFC 2616 states:
If no Accept-Encoding field is present in a request, the server MAY
assume that the client will accept any content coding. In this case, if “identity” is one of the available content-codings, then the
server SHOULD use the “identity” content-coding, unless it has
additional information that a different content-coding is meaningful
to the client.

Google web search now tests these browsers’ ability to understand compressed content. If the test is successful, Google serves gzipped content. This has significantly lowered latency for users at the tail end of the latency distribution.

Andy Martone

Google

Andy Martone is a software engineer at Google. He works on the websearch infrastructure team, trying to make Google search even faster.

Andy holds a Master’s degree in Computer Science from George Mason University.

Comments on this page are now closed.

Comments

Andy Martone

07/13/2010 9:41am PDT

@Brian Gibson: I’m not an expert on HTTPS behavior through proxies by any means, though it would be possible for a man-in-the middle HTTPS proxy to modify request content.

The nice thing about this technique is that it should silently fail if the test doesn’t succeed.

Brian Gibson

07/05/2010 4:21am PDT

Hi Andy,
I attended your session about forcing GZIP compression at Velocity conference. I found it really informative and came away thinking this could potentially be a huge issue for my company’s product, where all our customers will be accessing it from educational establishments behind firewalls and proxy servers.

After thinking about if for a while I wanted to ask you – Do you think that websites that use HTTPS exclusively are exempt? The reason I am thinking is that local proxies and firewalls will not see the Accept-Encoding header in HTTPS traffic as it is encrypted and therefore it cannot be stripped?

Great presentation, Andy. Please post the slides when you get a chance.

Ryan Witt

06/23/2010 5:04pm PDT

Thanks so much for doing a follow-up! Hopefully this detailed how-to will spur more implementations.

Steve Souders

05/18/2010 4:11pm PDT

This is an amazing follow-up on Tony’s Beyond Gzipping preso from last year. Who had any idea that 15% of browsers that support gzip ARE NOT GETTING COMPRESSED RESPONSES! Andy’s team took the next step to identify browsers that could accept gzipped responses successfully, and quantified the performance win. It’s big. Come hear the numbers.