Archive

It’s that time again! Having finalized the RDMA over Converged Ethernet (RoCE) test results from Plugfest 29, the IBTA is pleased to announce the availability of the new RoCE Interoperability List. Designed to support data center managers, CIOs and other IT decision makers with their planned RoCE deployments for enterprise and high performance computing, the latest edition features a growing number of cable and equipment vendors and Ethernet test speeds.

In April 2016, Plugfest 29 saw nine member companies submit RoCE capable RNICS, switches, QFSP, QFSP28 and SFP28 cables for interoperability testing. This is an encouraging sign for the RoCE ecosystem as more and more vendors begin to offer solutions that are proven to work seamlessly with each other, regardless of brand. Furthermore, the new list now features 50 and 100 GbE test scenarios, which complements the IBTA’s existing 10, 25 and 40 GbE interoperability testing. This expansion gives RoCE deployers confidence in knowing that as they integrate faster Ethernet speeds in their systems, their applications can still leverage the advantages of tested RDMA technology.

The RoCE Interoperability List is created twice a year following bi-annual IBTA-sponsored Plugfests, which take place at the University of New Hampshire InterOperability Lab (UNH-IOL). The IBTA Integrators’ Program, made up of both the InfiniBand Integrators’ List and the RoCE Interoperability List, is founded on rigorous testing procedures that establish compliance and real-world interoperability.

The InfiniBand Integrators’ List, which features InfiniBand Host Channel Adapters (HCAs), switches, SCSI Remote Protocol (SRP) targets and cables, will be available soon via the IBTA Integrators’ List page. Additionally, mark your calendars for Plugfest 30 – October 17-29, 2016 at UNH-IOL. Registration information and event details will be available on the IBTA Plugfest page in the coming month.

The concept of hyperconverged integrated systems (HCIS) emerged as data centers considered new ways to increase resource utilization by reducing infrastructure inefficiencies and complexities. HCIS is primarily a software-defined platform that integrates compute, storage, networking resources. The HCIS market is expected to grow 79 percent to reach almost $2 billion this year, driving it into mainstream use in the next five years, according to Gartner.

Since this market is growing so rapidly, Gartner released an exciting new report, “Use Networking to Differentiate Your Hyperconverged System.” In the report, Gartner advises HCIS vendors to focus on networking to gain competitive market advantage by integrating use-case-specific guidelines and case studies in go-to-market efforts.

According to the report, more than 10 percent of HCIS deployments will suffer from avoidable network-induced performance problems by 2018, up from less than one percent today. HCIS vendors can help address expected challenges and add value for buyers by considering high performance networking protocols, such as InfiniBand and RDMA over Converged Ethernet (RoCE), during the system design stage.

The growing scale of HCIS clusters creates challenges such as expanding workload coverage and diminishing competitive product differentiation. This will force HCIS vendors to alter their product lines and marketing efforts to help their offerings stand out from the rest. Integrating the right networking capabilities will become even more important as a growing number of providers look to differentiate their products. The Gartner report states that by 2018, 60 percent of providers will start to offer integration of networking services, together with compute and storage services, inside of their HCIS products.

Until recently, HCIS vendors have often treated networking simply as a “dumb” interconnect. However, when clusters grow beyond a handful of nodes and higher workloads are introduced, issues begin to arise. This Gartner report stresses that treating the network as “fat dumb pipes” will make it harder to troubleshoot application performance problems from an end-to-end perspective. The report also determines that optimizing the entire communications stack is key to driving latency down and it names InfiniBand and RoCE as important protocols to implement for input/output (I/O)-intensive workloads.

As competition in the HCIS market continues to grow, vendors must change their perception of networking and begin to focus on how to integrate it in order to keep a competitive edge. To learn more about how HCIS professionals can achieve this market advantage, download the full report from the InfiniBand Reports page.

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally, and is used herein with permission. All rights reserved.

Next week, storage experts will gather at the 2016 Flash Memory Summit (FMS) in Santa Clara, CA, to discuss the current state of flash memory applications and how these technologies are enabling new designs for many products in the consumer and enterprise markets. This year’s program will include three days packed with sessions, tutorials and forums on a variety of flash storage trends, including new architectures, systems and standards.

NVMe technology, and its impact on enterprise flash applications, is among the major topics that will be discussed at the show. The growing industry demand to unlock flash storage’s full potential by leveraging high performance networking has led to the NVMe community to develop a new standard for fabrics. NVMe over Fabrics (NVMe/F) allows flash storage devices to communicate over RDMA fabrics, such as InfiniBand and RDMA over Converged Ethernet (RoCE), and thereby enabling all flash arrays to overcome existing performance bottlenecks.

Attending FMS 2016?

If you’re attending FMS 2016 and are interested in learning more about the importance of RDMA fabrics for NVMe/F solutions, I recommend the following two educational sessions:

Representatives from the IBTA will join a panel to discuss the value of RDMA interconnects for the NVMe/F standard. Attendees can expect to receive an overview of each RDMA fabric and the benefits they bring to specific applications and workloads. Additionally, the session will cover the promise that NVMe/F has for unleashing the potential performance of NVMe drives via mainstream high performance interconnects.

Beer, Pizza and Chat with the Experts
Tuesday, August 9, 2016 (7-8:30 p.m.)

This informal event encourages attendees to “sit and talk shop” with experts about a diverse set of storage and networking topics. As IBTA’s Marketing Work Group Co-Chair, I will be hosting a table focused on RDMA interconnects. I’d love to meet with you to answer questions about InfiniBand and RoCE and discuss the advantages they provide the flash storage industry.

Additionally, there will be various IBTA member companies exhibiting on the show floor, so stop by their booths to learn about the new InfiniBand and RoCE solutions:

·HPE (#600)

·Keysight Technologies (#810)

·Mellanox Technologies (#138)

·Tektronix (#641)

·University of New Hampshire InterOperability Lab (#719)

For more information on the FMS 2016 program and exhibitors, visit the event website.

The InfiniBand Trade Association recently had the opportunity to speak on RDMA technology at the 2015 Storage Developer Conference. For the first time, SDC15 introduced Pre-conference Primer Sessions covering topics such as Persistent Memory, Cloud and Interop and Data Center Infrastructure. Intel’s David Cohen, System Architect and Brian Hausauer, Hardware Architect spoke on behalf of IBTA in a pre-conference session and discussed “Nonvolatile Memory (NVM), four trends in the modern data center and implications for the design of next generation distributed storage systems.”

Below is a high level overview of their presentation:

The modern data center continues to transform as applications and uses change and develop. Most recently, we have seen users abandon traditional storage architectures for the cloud. Cloud storage is founded on data-center-wide connectivity and scale-out storage, which delivers significant increases in capacity and performance, enabling application deployment anytime, anywhere. Additionally, job scheduling and system balance capabilities are boosting overall efficiency and optimizing a variety of essential data center functions.

Trends in the modern data center are appearing as cloud architecture takes hold. First, the performance of network bandwidth and storage media is growing rapidly. Furthermore, operating system vendors (OSV) are optimizing the code path of their network and storage stacks. All of these speed and efficiency gains to network bandwidth and storage are occurring while single processor/core performance remains relatively flat.

Data comes in a variety of flavors, some of which is accessed frequently for application I/O requests and others that are rarely retrieved. To enable higher performance and resource efficiency, cloud storage uses a tiering model to access data based on what is accessed most often. Data that is regularly accessed is stored on expensive, high performance media (solid-state drives). Data that is hardly or never retrieved is relegated to less expensive media with the lowest $/GB (rotational drives). This model follows a Hot, Warm and Cold data pattern and allows you faster access to what you use the most.

The growth of high performance storage media is driving the need for innovation in the network, primarily addressing application latency. This is where Remote Direct Memory Access (RDMA) comes into play. RDMA is an advanced, reliable transport protocol that enhances the efficiency of workload processing. Essentially, it increases data center application performance by offloading the movement of data from the CPU. This lowers overhead and allows the CPU to focus its processing power on running applications, which in turn reduces latency.

Demand for cloud storage is increasing and the need for RDMA and high performance storage networking grows as well. With this in mind, the InfiniBand Trade Association is continuing its work developing the RDMA architecture for InfiniBand and Ethernet (via RDMA over Converged Ethernet or RoCE) topologies.

At IBTA, we are pleased to announce the launch of the RoCE Initiative, a new effort to highlight the many benefits of RDMA over Converged Ethernet (RoCE) and to facilitate the technology’s adoption in the enterprise data centers. With the rise of server virtualization and big data analytics, data center architects are demanding innovative ways to improve overall network performance and to accelerate applications without breaking the bank in the process.

Remote Direct Memory Access (RDMA) is well known in the InfiniBand community as a proven technology that boosts data center efficiency and performance by allowing the transport of data from storage to server with less CPU overhead. RDMA technology achieves faster speeds and lower latency by offloading data movement from the CPU, resulting in more efficient execution of applications and data transfers.

Before RoCE, the advantages of RDMA were only available over InfiniBand fabrics. This left system engineers that leverage Ethernet infrastructure with only the most expensive options for increasing system performance (i.e. adding more servers or buying faster CPUs). Now, data center architects can upgrade their application performance while leveraging existing infrastructure. There is already tremendous ecosystem support for RoCE; it is supported by server and storage OEMs, adapter and switch vendors, and all major operating systems.

On May 4-8, IT professionals and enterprise developers gathered in Chicago for the 2015 Microsoft Ignite conference. Attendees were given a first-hand glimpse at the future of a variety of Microsoft business solutions through a number of sessions, presentations and workshops.

Of particular note were two demonstrations of RDMA over Converged Ethernet (RoCE) technology and the resulting benefits for Windows Server 2016. In both demos, RoCE technology showed significant improvements over Ethernet implementations without RDMA in terms of throughput, latency and processor efficiency.

Below is a summary of each presentation featuring RoCE at Ignite 2015:

During the presentation, the use of RoCE compared to TCP/IP showcased drastically different performance. With RDMA enabled, the SMB3 server was able to achieve about twice the throughput, half the latency and around 33 percent less CPU overhead than that attained by TCP/IP.

The goal of the demo was to highlight the value of running RoCE on a system as it related to performance, latency and processor utilization. The system was able to achieve a combined 680,000 4KB IOPS and 2ms latency when RoCE was disabled. With RoCE enabled, the system increased the 4KB IOPS to about 1.1 million and reduced the latency to 1ms. This translated roughly to a 40 percent increase in performance with RoCE enabled, all while utilizing the same amount of CPU resources.

For additional information, watch a recording of the presentation (demonstration starts at 57:00).

On April 14-16, Ethernet designers and experts from around the globe gathered at the Ethernet Technology Summit 2015 to discuss developments happening within the industry as it pertained to the popular networking standard. IBTA’s Diego Crupnicoff, co-chair of the Technical Working Group, shared his expertise with attendees via a presentation on “Accelerating Data Movement with RDMA over Converged Ethernet (RoCE).” The session focused on the ever-growing complexity, bandwidth requirements and services of data centers and how RoCE can address the challenges that emerge from new enterprise data center initiatives.

Here is a brief synopsis of the points that Diego covered in his well-attended presentation:

People are living in an ever-increasing digital world. In the last decade, there’s been an explosion of connected devices that are running many applications and creating massive amounts of data in the process that must be accessible anytime, anywhere.

Over time, the data center has emerged as the workhorse of the networking industry, with the increased pace of the ‘information generation’ generating many new data center initiatives, such as the cloud, virtualization and hyper-converged infrastructure. Expectations for enhanced accessibility to larger sets of data are straining enterprise data networks, bringing about a variety of new challenges to the industry, including the following needs:

The Transmission Control Protocol (TCP) has had difficulty keeping up with some traffic stemming from newer, more demanding applications. In these cases, packet processing over TCP saturates CPU resources, resulting in networks with low bandwidth, high latency and limited scalability. The industry was in need of a capability that would bypass the CPU altogether to enable faster, more efficient movement of data between servers.

The advent of Remote Direct Memory Access (RDMA) did just that, utilizing hardware offloads to move data faster with less CPU overhead. By offloading the I/O from the CPU, users of RDMA experience lower latency while freeing up the CPU to focus its resources on applications that process data as opposed to moving it.

Recently, RDMA expanded into the enterprise market and is now being widely adopted over Ethernet networks with RDMA over Converged Ethernet or RoCE. The RoCE standard acts as an efficient, lightweight transport that’s layered directly over Ethernet, bypassing the TCP/IP stack. It offers the lowest latency in the Ethernet industry, which enables faster application completion, better server utilization and higher scalability. Given these advantages, RoCE became the most widely deployed Ethernet RDMA standard, resulting in millions of RoCE-capable ports on the market today.

For additional details on the benefits of RDMA for Ethernet networks, including RoCE network considerations and use cases, view the presentation in its entirety here.

The Ethernet Technology Summit 2015 kicks off today at the Santa Clara Convention Center in Santa Clara, CA. The three-day conference offers seminars, forums, panels and exhibits for Ethernet designers of all levels to discuss new developments, share expertise and learn from prominent industry leaders. Sessions run April 14-16 and exhibits are open April 15-16.

Additionally, Diego Crupnicoff, co-chair of the Technical Working Group from member company Mellanox Technologies, will participate in the “Forum 1B: Ethernet in Data Centers (Data/Telco Centers Track)” session on Wednesday, April 15 from 8:30 to 10:50 a.m.

Specifically, Diego will present on “Accelerating Data Movement with RDMA over Converged Ethernet (RoCE)” and how RoCE addresses the challenges that are emerging from new data center initiatives. This is an excellent opportunity to learn about RoCE and other enhancements required to meet the ever-growing needs of an optimized, flexible data center.

In addition there are other presentations, panel discussions, and keynotes given by representatives of IBTA members Broadcom, Cisco, Intel, Mellanox, Microsoft, Molex, and QLogic. For a complete schedule and description of the 2015 Summit sessions, click here.

Big news! The IBTA today announced the updated specification for Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE), RoCEv2. This is a huge development and one that will expand RoCE’s adoption.

RoCEv2 extends the original RoCE specification by enabling routing across Layer 3 networks and as a result provides better isolation and enables hyperscale data center deployments. This addresses the needs of today’s evolving data centers which require more efficient data movement over a variety of network topologies.

With a number of vendors including Dell and Zadara Storage recently adopting RoCE, this new specification comes at the perfect time. Major cloud providers and Web 2.0 companies are also adopting RoCE in order to combat the challenges of running compute intensive applications and processing massive amounts of data in hyperscale networking environments.

Representatives from IBTA member companies including Emulex, IBM, Mellanox, Microsoft and Software Forge, Inc. participated in the development of this new standard, which will help enterprises to more widely adopt RoCE and improve infrastructure efficiency.

The RoCE specification was released back in April, 2010 to address the need for efficient RDMA in end to end Ethernet networks. With the increase in enterprise data traffic and the emergence of hyperscale infrastructure, the need for efficient networking continues to grow. Here are the latest examples of RoCE in the field.

RoCE Accelerates Microsoft Windows Azure

Albert Greenberg, an architect with IBTA member Microsoft, describes in his keynote address to the Open Network Summit this year how they are using RoCE in their storage offering. RoCE over 40GbE enabled line-rate performance with zero CPU usage. At the same time they were able to use less CPUs than would otherwise be needed.

IBTA member Applied Micro is continuing to roll out its X-Gene family of 64-bit processors. These processors support RoCE, giving 10GbE some of the low-latency capabilities that used to only be available to InfiniBand. The decrease in latency for Ethernet means transaction latencies also decrease, better positioning the X-Gene family to make a difference in modern workloads.

RoCE Provides Exceptional Performance for Storage Solutions

Zadara Storage announced a new high performance STaaS solution offer for private clouds that uses an Ethernet transport mechanism for exceptional performance at reduced costs. Developed as a collaboration with IBTA member Mellanox Technologies, this solution boosts application performance using the iSCSI Extensions for RDMA (iSER) over RoCE, delivering substantially improved latency and throughput and therefore delivering cost savings and converged enterprise storage for private clouds of all sizes.

RoCE Helps Data Centers Account for New Technology for the Cloud

With the huge acceleration of data centers to the cloud, low latency is becoming increasingly important to avoid bottlenecks and incorporate new technologies seamlessly. RoCE, says SYS-CON Media’s Barbara Porter, is one way to plan for the barrage of new technology that’s coming our way, and to reduce latency overall in the cloud.