IBM z13s Technical Manual page 209

By eliminating FICON Express8 (and thus the need to carry forward the I/O drawers), you
can preserve InfiniBand fanout slots for coupling over InfiniBand fanouts. Replace FICON
Express8 with FICON Express8S or (depending on your SAN fabric infrastructure) with
FICON Express16S.
Evaluate configurations for opportunities to eliminate or consolidate InfiniBand links:
– Eliminate any redundant links. Two physical links between CPCs is the minimum
requirement from a reliability, availability, and serviceability (RAS) perspective.
– Share logical CHPIDs on a physical IFB link connection to reduce the usage of IFB
links in z13s servers (even multiple sysplexes can share a single link).
– Coupling Link Analysis: Capacity Planning tools and services can help.
For z13s to z13s, or z13 links, adopt the new ICA SR coupling link. Use ICA SR channel
as much as possible to replace existing InfiniBand links for z13s to z13, and z13s
connectivity.
Install all the ICA SR links that are required to fulfill future short distance coupling
requirements:
– When upgrading a CPC to a z13s system, configure the z13s servers with all the
ICA SR coupling links that eventually will be needed (that is, avoid loose piece MES
with ICA SR links) in your Parallel Sysplex configuration.
– Consider a plan-ahead configuration. In a multi-CPC configuration, a final approach
can be to establish z13s to z13s, and z13 connectivity by using mainly ICA SR
connectivity. Even if the links are not used right away, especially within the first z13s
migration, most coupling connectivity uses InfiniBand. However, after the second z13s
server is migrated, coupling connectivity can be moved to ICA SR links.
Upgrade CPCs with fewer coupling constraints first:
– Consider upgrading CPCs to have sufficient IFB links in the target z13s configuration
first (for example, multi-CPC drawer CPCs).
– Test the new ICA SR link on the least constrained CPCs (for example, z/OS Host CPCs
that have only the lowest number of links). For the CPCs that are involved in this test
that do not have a CF, this test might require adding a CF LPAR and ICF engine to one
of the CPCs.
– When migrating CPCs with more coupling links to z13s servers, use enough ICA SR
links in place of Coupling over InfiniBand (for example, half ICA SR, half 12x) to
maintain the CPC footprint (that is, avoid extra CPC drawers).
Consider replacing two servers that are close to each other at the same time. Assess the
risk, and, if acceptable, allow immediate replacement of 12x IFB with ICA SR links in z13s
peers, thus freeing InfiniBand fanout slots for 1x InfiniBand links.
Consider (even temporarily) increased CHPID/link consolidation on 12x InfiniBand links.
For less critical sysplexes (that is, test, development, and QA) that use the IFB3 protocol
(see 4.6.4, "HCA3-O (12x IFB) fanout (FC 0171)" on page 150), consider temporary
performance reduction when consolidating more than four CHPIDs per 12x link. After the
peer system has been migrated to z13s servers, the InfiniBand links can be migrated to
ICA SR links.
Coupling link considerations
For more information about changing to InfiniBand or ICA SR coupling links, see
Implementing and Managing InfiniBand Coupling Links on IBM System z, SG24-7539, and
the Coupling Facility Configuration Options white paper, which can be found at the following
website:
http://www.ibm.com/systems/z/advantages/pso/whitepaper.html
Chapter 4. Central processor complex I/O system structure
181