Shared Roce Mode Concepts; D.2.4 Hardware - IBM z13s Technical Manual

Figure D-3 shows the concept of the Shared RoCE Mode.
z/OS CS
implements
PCIe RoCE
Device Driver
(with SR-IOV)
Figure D-3 Shared RoCE mode concepts
The Physical Function (PF) Driver communicates to the Physical Function in the PCIe
adapter. The PF Driver has a relatively limited function:
Manages resource allocation
Perform hardware error handling
Perform code updates
Run diagnostics
The Device-specific z Systems Licensed Internal Code (LIC) connects PF Driver to Support
Elements (SEs) and limited system level firmware required services.

D.2.4 Hardware

The 10 Gigabit Ethernet (10GbE) RoCE Express feature (FC 0411) is an RDMA-capable
network interface card (NIC). The integrated firmware processor (IFP) has two Resource
Groups (RGs) which have firmware for the 10GbE RoCE Express feature. See G.1.3,
"Resource groups" on page 523 in this book for reference.
D.2.5 10GbE RoCE Express feature
The 10GbE RoCE Express feature is designed to help reduce the consumption of CPU
resources for applications that use the TCP/IP stack, such as WebSphere accessing a DB2
database. Using the 10GbE RoCE Express feature also helps to reduce network latency with
memory-to-memory transfers that use SMC-R in z/OS V2.1 or later. It is transparent to
applications and can be used for LPAR-to-LPAR communications on a single z/OS system or
server-to-server communications in a multiple CPC environment.
N number of z/OS instances
z/OS LP
RoCE PCI
Virtual Function
Function ID
DLC
(VF n)
(zPCIe)
10GbE RoCE Express
Ports 1 & 2
DMA (via PCIe bus
I/O) path is direct
(adapter to/from LP
memory)
2 Transparent Firmware SPs
PCIe Firmware
Support Partition
Physical Function
(physical resources)
PF
vHCR
I/O Draw
RoCE PCHID
RNIC
Comm Channel
used for vHCR
command
processing
Appendix D. Shared Memory Communications
CPC
Firmware SPs
provide PCIe RoCE
SR-IOV Physical
Function
SE
HMC
479