D.1 Shared Memory Communications Overview - IBM z13s Technical Manual

D.1 Shared Memory Communications overview

The volume of data being generated and transmitted by technologies driven by cloud, mobile,
analytics, and social computing applications is growing. This increases the pressure on
business IT organizations to be able to provide fast access to that data across the web,
application, and database tiers that comprise most enterprise workloads. Shared Memory
Communications helps to access data faster and with less latency, and helps reduce CPU
resource consumption over traditional TCP/IP for communications.
D.2 Shared Memory Communication over RDMA
IBM z13s delivers improvements for the RoCE exploitation over the previous zEnterprise
generation (zEC12, zBC12). On z13 and z13s servers, the IBM 10GbE RoCE Express
feature can be shared between up to 31 partitions and the two ports are enabled to be used in
z/OS. IBM z13 and z13s servers improve the usability of the RoCE feature by using existing z
Systems servers and industry standard communications technology along with emerging new
network technology:
RDMA technology provides low latency, high bandwidth, high throughput, and low
processor utilization attachment between hosts.
SMC-R is a protocol that allows existing TCP applications to benefit transparently from
RDMA for transferring data:
– SMC-R uses a 10GbE RoCE Express adapter as the physical transport layer.
– Initial deployment is limited to z/OS to z/OS communications with a goal to expand
exploitation to more operating systems and possibly appliances and accelerators.
Single root I/O virtualization (SR-IOV) technology provides the capability to share the
10GbE RoCE Express adapter between logical partitions (LPARs).
D.2.1 RDMA technology overview
RoCE is part of the InfiniBand Architecture Specification that provides InfiniBand transport
over Ethernet fabrics. It encapsulates InfiniBand transport headers into Ethernet frames by
using an IEEE-assigned Ethertype. One of the key InfiniBand transport mechanisms is
RDMA, which is designed to allow transfer of data to or from memory on a remote system
with low latency, high throughput, and low CPU utilization.
Traditional Ethernet transports, such as TCP/IP, typically use software-based mechanisms for
error detection and recovery, and are based on the underlying Ethernet fabric using a
"best-effort" policy. With the traditional policy, the switches typically discard packets in
congestion and rely on the upper-level transport for packet retransmission.
RoCE, however, uses hardware-based error detection and recovery mechanisms that are
defined by the InfiniBand specification. A RoCE transport performs best when the underlying
Ethernet fabric provides a lossless capability, where packets are not routinely dropped. This
goal can be accomplished by using Ethernet flow control where Global Pause frames are
enabled for both transmission and reception on each of the Ethernet switches in the path
between the 10GbE RoCE Express features. This capability is enabled by default in the
10GbE RoCE Express feature.
RDMA has two key requirements as shown in Figure D-1:
476
IBM z13s Technical Guide