Efficient utilization of 40/100 Gbps long-distance network by a single TCP stream

Efficient utilization of single or a few TCP streams by user
applications is the key to promote high-speed internet technology to
scientific community and general communities. Our research group has
developed several technologies to fully utilize 10 Gbps long-distance
internet by TCP protocol, achieved more than 90% utilization of 10 Gbps
TPC single stream performance using both artificial software and actual
Web applications.

We started a new project whose goal is to achieve very high efficiency
of TCP single streams on 40G/100G long distance network. We found that
technologies we developed are not sufficient to achieve high-efficiency
on 40/100 Gbps long-distance internet because the amount of on the fly
is much larger than 10 Gbps network. Currently we are working on
following optimization techniques:

Modification of TCP buffer structure to accomodate with huge amount
of on the fly data,

Classification and discrimination of packet losses caused by
congestion and caused by L1 level errors using machine learning,

Estimation of the amount of counter traffic by measuring RTT of very
low bandwidth probe packets.

As for performance evaluation, we performed local experiments using two
servers with Mellanox PCI-express 3.0 NIC and two 40 Gbps switches. We
observed 36.5Gbps (payload) bandwidth of a single TCP stream by iperf.
Unfortunately, we do not have either a 40 Gbps delay emulator or
long-distance 40/100 Gbps internet routes in Japan. This is the reason
we need collaboration to Europe and US network institute to do empirical
study of efficient use of a single TCP stream over 40/100G internet
connection.