Some QUIC Performance Results

Brian Prodoehl and I spent a couple hours doing some performance tests on QUIC with networks of different bandwidths, latencies and reliabilities. We wrote up our results, and posted our test scripts and data in this blog post: http://www.connectify.me/taking-google-quic-for-a-test-drive/ We'd be interested to hear how these numbers line up with what others are seeing or would expect to see.

We're still nailing elements of "correctness," where that means "matching TCP's performance." Bottom line: We hope to soon be past this phase, as we are now starting to add additional optimizations which should help us to shine.

I think there was (for example) a bug where (contrary to TCP) when more than one packet was lost during an epoch (a single round trip time), then the congestion window could be reduced more than once (i.e., multiplied by a reduction factor more than once). I think that might explain one of your results. (We should have upstreamed that change... but probably have not yet landed it back into Chromium where you were testing).

We also had (perhaps still have) a clamp on the "maximum congestion window" (re: CWND), and that may have prevented the bandwidth from rising endlessly in your 0-packet-loss experiment. We put in the clamp as were were working to straighten out our RTO response (which had a bug), but it should be removed shortly (even though it would not have impacted most common user scenarios... but did show in your test).

We appreciate the report.... and hope you'll post again in a few weeks as we hopefully progress.

Great work! Instead of just comparing QUIC with TCP, it would be interesting to compare QUIC with native SCTP. The QUIC team seem to try to promote the advantages of QUIC by talking about the disadvantages of SCTP over UDP--however, they seem to have dismissed native SCTP.

If you're interested in other tools that kind of do a similar thing, you should try out a CrushTunnel. It takes a TCP socket input, splits it across a dynamic number of other sockets (channels) to get to the other side of the tunnel where pieces are put back together in order and continue as a single socket connection to their destination. It overcomes the latency aspect as you still get the full TCP speed you would have gotten, but multiplied by the number of channels. Its great for file transfer and large continuous pushes of data, but not good for conversation activity because it is adding its own form of latency... We have tested it at up to 800Mbit on the WAN where it would normally have only been roughly 10Mbit to 20Mbit because of latency. Email us if you are interested, we currently are beta testers with your switchboard product.

When you mention QUIC, you specified that it is HTTP over QUIC in your report. However in the latest version of the quic_client/quic_server we see that it's using SPDY. ( src/net/tools/quic/quic_client_session.cc, line 45, QuicSpdyClientStream is created by default).

Is it possible that the code might have changed to use SPDY instead of HTTP between now and the time when you ran the experiments?

Sorry for the silence, we got pulled into getting a new release of Switchboard out the door. That's just about done, and so we're back to fiddling around here. Right now we're playing with QUIC and the suggestion of SCTP. Hopefully we'll have more to share next week. I think Brian will be chiming in here soon with some answers about how we actually ran the test.

---Alex

On Friday, November 8, 2013 7:53:59 PM UTC-5, B Galliart wrote:

Great work! Instead of just comparing QUIC with TCP, it would be interesting to compare QUIC with native SCTP. The QUIC team seem to try to promote the advantages of QUIC by talking about the disadvantages of SCTP over UDP--however, they seem to have dismissed native SCTP.

Within Chromium, QUIC shares a lot of code with SPDY. Don't be concerned with the overlap and reuse of some SPDY code to handle multiple streams.

HTTP is a higher layer, and HTTP requests are multiplexed into a SPDY or QUIC connection, with each such request corresponding to a stream.. Although SPDY is potentially forming the basis of HTTP/2, you should think of your "good old HTTP requests" as continuing to exist, and only ask if the request went over a SPDY, vs QUIC, vs TCP (or TLS over TCP) transport.

To see if you have QUIC traffic in Chromium, you can visit about:net-internals and verify how indeed QUIC is being used, and what requests it conveyed.