Page 1 Front cover IBM System z9 Enterprise Class Technical Guide Structure and design a total systems approach Processor unit, memory, channel subsystems, and subchannel sets Concurrent upgrade and maintenance options Bill White Franck Injey Greg Chambers Marian Gasparovic Parwez Hamid Brian Hatfield...
Page 3 International Technical Support Organization IBM System z9 Enterprise Class Technical Guide June 2007 SG24-7124-02...
Page 4 Third Edition (June 2007) This edition (SG24-7124-02) applies to the IBM® System z9 Enterprise Class server. © Copyright International Business Machines Corporation 2005, 2006, 2007. All rights reserved. Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule...
Page 6 6.1 Operating systems summary ......... . . 166 IBM System z9 Enterprise Class Technical Guide...
Page 7 6.2 Support by operating system ......... . . 167 6.2.1 z/OS .
Page 8 Fiber cabling services options ..........274 IBM Networking Services fiber cabling services ......274 Summary .
Page 9 IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead.
Page 10 Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. viii IBM System z9 Enterprise Class Technical Guide...
Page 11 Marian Gasparovic is an IT Specialist from IBM Slovakia. He worked as an administrator for z/OS® as an IBM Business Partner for six years. He joined IBM in 2004 and now works as a Field Technical Sales Support for System z in the CEMA region as a member of a new workload team.
Page 12 Parwez Hamid is a Consulting IT Specialist working for IBM Server and Technology Group in the UK. During the past 32 years, he worked in various roles within IBM and with a large number of IBM mainframe customers. He also worked on projects introducing new technology.
Page 13 Use the online Contact us review book form found at: ibm.com/redbooks Send your comments in an email to: redbook@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400 Preface...
Page 14 IBM System z9 Enterprise Class Technical Guide...
Page 15 Overview Chapter 1. For over four decades, the IBM mainframe has been a leader in data and transaction serving. The announcement of the System z9 in July of 2005 provided a strong combination of past mainframe characteristics plus new functions designed around scalability, availability, and security.
Page 16 Figure 1-1 z9 EC balanced system design With the System z9, the Parallel Sysplex® cluster takes the commercial strengths of the z/OS platform to improved levels of system management, competitive price/performance, scalable growth, and continuous availability. IBM System z9 Enterprise Class Technical Guide...
Page 17 The z9 EC provides a significant increase in system scalability and opportunity for server multi-book consolidation by providing a system structure. All books are interconnected with a very high-speed internal communications links through the L2 cache, which allows the system to be operated and controlled by the PR/SM™ facility as a symmetrical, memory-coherent multiprocessor.
Page 18 I/O features or channel types supported on the z9 EC are: – FICON Express (upgrades from z990), FICON Express2, and FICON Express4 – Coupling Links - Peer mode only – OSA-Express (upgrades from z990) and OSA Express2 – ESCON® – Crypto Express2 IBM System z9 Enterprise Class Technical Guide...
Page 19 Model upgrade paths With the exception of the z900 Model 100, any z900, z990, and z9 Business Class model S07 may be upgraded to a z9 EC, as shown in Figure 1-3. z900* * Except z900 Model 100 z990 z9 EC z9 BC model S07 Figure 1-3 Upgrades Concurrent Processor Unit conversions...
Page 20 Double the available memory per model, with up to 128 GB per book. Increased host bus bandwidth between memory and I/O. Up to 16 Self-Timed Interconnects (STIs) per book, with 2.7 GBps each for I/O, a 35 percent increase in STI speed compared to z990. IBM System z9 Enterprise Class Technical Guide...
Page 21 OSA-Express2 OSN (OSA for NCP) is designed to provide Channel Data Link Control (CDLC) protocol support on the z9 EC for the IBM Communication Controller for Linux on System z (CCL), which allows system administrators to configure, manage, and operate their CCL Network Control Programs (NCPs) in the same manner as their ESCON-attached 374x NCPs.
Page 22 The ability to specify a separate logical partition weight for shared zAAPs or zIIPs helps to simplify capacity planning and management of the configured logical partitions and their associated processor resources. IBM System z9 Enterprise Class Technical Guide...
Page 23 Channel Subsystem (CSS) The z9 EC CSS supports multiple Channel Subsystems on a single server. It provides: Four Channel Subsystems, each CSS having from one to 256 channels. Multiple Subchannel Sets. Each CSS can be configured with one to 15 logical partitions. The I/O subsystem continues to be viewed as a single entity, through an Input/Output Configuration Data Set (IOCDS) across the entire system.
Page 24 FCP mode of operation and CHPID type definition. FCP mode is available in long wavelength (LX) and short wavelength (SX) operation. The z9 EC supports FCP channels, switches, and FCP/SCSI devices with full fabric connectivity under Linux on System z. IBM System z9 Enterprise Class Technical Guide...
Page 25 FICON CTC function FICON CTC connectivity increases the bandwidth between z9 EC, z9 BC, z990, and z890. As the FICON CTC function is included as part of the native FICON (FC) mode of operation on System z, a FICON channel used for FICON CTC is not limited to intersystem connectivity;...
Page 26 3366) have the capability to provide channel connectivity from System z operating systems to IBM Communication Controller for Linux on System z (CCL) using the Open Systems Adapter for the Network Control Program (OSA for NCP) supporting the Channel Data Link Control (CDLC) protocol.
Page 27 Improves the ability to manage ATM conversions. Improved key Exchange with Non-CCA Cryptographic systems. IBM Common Cryptographic Architecture (CCA) The features in CCA provide the ability to exchange keys between CCA systems, and systems that do not use Control Vectors, by allowing the CCA system owner to define the permitted types of key import and export while preventing uncontrolled key exchange that can open the system to an increased threat of attack.
Page 28 EC (2094) 701 is 1.35 times the z990 (2084) 301 (LSPR mixed workload). This is a comparison of the z9 EC 54-way and the z990 D32 and is based on the LSPR mixed workload average. IBM System z9 Enterprise Class Technical Guide...
Page 29 The LSPR contains the Internal Throughput Rate Ratios (ITRRs) for the z9 EC and the previous generation processor families based upon measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user may experience will vary depending upon such considerations as the amount of multiprogramming in the user's job stream, the I/O configuration, and the workload processed.
Page 30 STI-3, in the I/O cage. ICB-4 links (2 gigabytes per second) should be used to provide coupling communication between z9 EC, z9 BC, z990, and z890 servers, because they deliver improved performance over ICB-3 links (1 gigabyte per second). IBM System z9 Enterprise Class Technical Guide...
Page 31 Customers interested in deploying System-Managed CF Structure Duplexing should read the technical paper System-Managed CF Structure Duplexing, ZSW01975USEN. See the Parallel Sysplex Web site at: http://www.ibm.com/systems/z/pso/index.html 1.3.6 Server Time Protocol Server Time Protocol is a server-wide facility that is implemented in the Licensed Internal Code (LIC) of z9 EC, z9 BC, z990, z890 servers, and Coupling Facilities.
Page 32 V1.7 (PTFs are required to enable STP support) and above. 1.3.7 Intelligent Resource Director (IRD) Exclusive to the IBM z/Architecture is Intelligent Resource Director (IRD), a function that optimizes processor and channel resource utilization across logical partitions based on workload priorities.
Page 33 Both On/Off CoD and CBU can reside on the server, but only one can be activated at a time. On/Off CoD test On/Off CoD allows for a no-charge test. No IBM charges are assessed for the test, including IBM charges associated with temporary hardware capacity, IBM software, or IBM maintenance.
Page 34 LIC level. Once a sync point has passed, it will be required to wait until the next sync point supporting concurrent activation of a new LIC level. Certain LIC updates will not be supported by this function. IBM System z9 Enterprise Class Technical Guide...
Page 35 Dynamic oscillator switch-over The z9 EC has two oscillator cards, a primary and a backup. In the event of a failure of the primary oscillator card, the backup is designed to detect the failure, switch over, and provide the clock signal to the server transparently. Dynamic oscillator switch-over is exclusive to the z9 EC.
Page 36 The IBM Mainframe has successfully been running the core IT systems of many businesses, from medium to very large size, for more than 40 years. During these years, IBM has consistently invested in the evolution of the mainframe’s unparalleled technology.
Page 37 The generally recognized IBM mainframe qualities fall into a number of categories: Security Manageability Virtualization and Workload Management Reliability Scalability Availability Transaction processing Batch processing The SOA philosophy proclaims advantages through strong corporate IT governance and reuse. The mainframe was made to support these tasks.
Page 38 IBM mainframes provide an advanced combination of reliability, availability, security, scalability, and virtualization. The IBM System z9, delivering excellence in large-scale enterprise computing, is designed and optimized for On Demand Business. The IBM System z9 Enterprise Class has been designed to deliver: Great granularity with subcapacity engines and high scalability with up to 54 engines on a single server.
Page 39 connectivity to I/O resources on other books during the removal in support of this capability. I/O availability features include assists for isolation of ESCON and FICON fiber optic cabling problems and FICON link incident reporting. Chapter 1. Overview...
Page 40 IBM System z9 Enterprise Class Technical Guide...
Page 41 System structure and design Chapter 2. This chapter introduces the IBM System z9 system structure. Significant functions and features are described, along with their characteristics and options. The goal of this chapter is to explain how the z9 EC is structured, what its main components are, and how these components interconnect from a physical and logical point of view.
Page 42 (PUs), memory, and connectors to I/O cages and ICB-4 links. Books are located in the CEC cage in Frame A. A z9 EC server has at least one book, but may have up to four books installed. IBM System z9 Enterprise Class Technical Guide...
Page 43 A book and its components are shown in Figure 2-1. Each book contains: 12 or 16 Processor Units (PUs). The PUs reside on microprocessor chips located on a Multi-Chip Module (MCM). 16 GB to 128 GB physical memory. At least four memory cards are present, each containing 4, 8, or 16 GB.
Page 44 MRU1 to be installed, providing all refrigeration needs for a four-book system. Concurrent repair of an MRU is possible by taking advantage of the hybrid cooling implementation described in the next section. The Motor Scroll Assembly (MSA) IBM System z9 Enterprise Class Technical Guide...
Page 45 The Motor Drive Assembly (MDA) MDAs are found throughout the frames to provide air cooling where required. They are located at the bottom front of each cage, and in between the CEC cage and I/O cage, one in combination with the MSAs. Hybrid cooling system The z9 EC has a hybrid cooling system that is designed to lower power consumption.
Page 46 EC Model S18, from 16 to 256 GB z9 EC Model S28, from 16 to 384 GB z9 EC Model S38, from 16 to 512 GB z9 EC Model S54, from 16 to 512 GB IBM System z9 Enterprise Class Technical Guide...
Page 47 Physically, the memory cards are organized as follows: Within a book, memory is organized in two rows (banks) of four memory cards. Since book memory is organized in up to four Processor Memory Arrays (PMAs) and since one memory card only encompasses half a PMA, eight memory cards are needed for a full book with 128 GB.
Page 48 Remember, the minimum physical memory size in a book is 16 GB. During a model upgrade, the addition of a book is a concurrent operation. The addition of the physical memory that is in the added book is also concurrent. IBM System z9 Enterprise Class Technical Guide...
Page 49 If all or part of the additional memory is enabled for installation use, it becomes available to an active logical partition if this partition has reserved storage defined (see 2.5.1, “Reserved storage” on page 86 for more detailed information). Or, it may be used by an already defined logical partition that is activated after the memory addition.
Page 50 A memory-coherent director optimizes ring traffic and filters out cache traffic by not looking on the ring for cache hits in other books if it is certain that the resources for a given logical partition exist in the same book. IBM System z9 Enterprise Class Technical Guide...
Page 51 The Level 2 (L2) cache is implemented on four cache (SD) chips. Each SD chip holds 10 MB, resulting in a 40 MB L2 cache per book. The L2 cache is shared by all PUs in the book and has a store-in buffer design. The connection to processor memory is done through four high-speed memory buses.
Page 52 2094 Book 1 2.7 GB/sec I/O Cage STI-MP STI-MP Interconnect Bandwidth to I/O card: 2 GB/sec 1 GB/sec 500 MB/sec 333 MB/sec Depends on I/O card type Figure 2-7 Redundant I/O Interconnect concept IBM System z9 Enterprise Class Technical Guide...
Page 53 Normally, book 0 MBA/STI connects to the STI-MP (A) card and services domain 0 I/O connections (slots 01, 03, 06, and 08). In the same fashion, book 1 MBA/STI connects to the STI-MP (B) card and services domain 1 (slots 02, 04, 07, and 09). If book 1 is removed, or the connections from book 1 to the cage are removed, connectivity to domain 1 is maintained by guiding the I/O to domain 1 through the interconnect between STI-MP (A) and STI-MP (B).
Page 54 One or two Modular Refrigeration Units (MRUs) that are air-cooled by their own internal cooling fans. The CEC cage, containing up to four books, each with two insulated refrigeration lines to an MRU. IBM System z9 Enterprise Class Technical Guide...
Page 55 I/O cages can house all supported types of channel cards. An I/O cage accommodates up to 420 ESCON channels or up to 112 FICON Express4 channels in the absence of any other card. Up to three I/O cages are supported. Air-moving devices (AMD) providing N+1 cooling for the MBAs, memory, and DCAs.
Page 56 PU chips. Four single-core and four dual-core chips are used, resulting in 12 PUs per MCM. For the z9 EC Model S54, the Processor Units (PUs) on the MCMs in all books are implemented with eight dual-core PU chips, resulting in 16 PUs per MCM. IBM System z9 Enterprise Class Technical Guide...
Page 57 Eight to ten PUs of the 12 PU version may be characterized for customer use. The two standard SAPs are initially allocated to the dual-core processor chips. Optionally, up to two spare chips may be allocated on an MCM. System-wide, two spare chips are available that may be allocated on any MCM of the system.
Page 58 2.7 GB per 2.7 GB per 2.7 GB per second second second second second Maximum number of I/O cages Number of Support Elements External power 3 phase 3 phase 3 phase 3 phase 3 phase IBM System z9 Enterprise Class Technical Guide...
Page 59 The z9 EC offers very high levels of availability, reliability, resilience, and security, and fits in IBM strategy in which mainframes play a central role in realizing an intelligent integrated infrastructure. The z9 EC is designed using a holistic approach. This means not only the...
Page 60 The interface between the L2 cache and processor memory (L3) is accomplished by four high-speed memory buses and controlled by the memory controllers (MSC). Storage access is interleaved between the storage cards, which tends to equalize storage activity across the IBM System z9 Enterprise Class Technical Guide...
Page 61 cards. Each PMA has two ports that each have a maximum bandwidth of 8 GB per second. Each port contains a control and a data bus, in order to further reduce any contention by separating the address and command from the data bus. The memory cards support store protect key caches to match the key access bandwidth with that of the memory bandwidth.
Page 62 ETR Attachment Facility (EEAF) designed to provide a dual External Time Reference (ETR) attachment facility. Two ETR cards are automatically shipped when Coupling Links are ordered and provide a dual path interface to the IBM Sysplex Timers, which are used for timing synchronization between systems in a Sysplex environment. This allows continued operation even if a single ETR card fails.
Page 63 Superscalar processor A scalar processor is a processor that is based on a single issue architecture, which means that only a single instruction is executed at a time. A superscalar processor allows concurrent execution of instructions by adding additional resources onto the microprocessor to achieve more parallelism by creating multiple pipelines, each working on their own set of instructions.
Page 64 This design simplifies error detection during instruction execution, saving additional circuits and extra logic required to do this checking. The z9 EC servers also contain error-checking circuits for data flow parity checking, address path parity checking, and L1 cache parity checking. IBM System z9 Enterprise Class Technical Guide...
Page 65 Processor Branch History Table (BHT) The Branch History Table implementation on processors has a key performance improvement effect. The BHT was originally introduced on the IBM ES/9000® 9021 in 1990 and has been improved ever since. The z9 EC server BHT offers significant branch performance benefits. The BHT allows each CP to take instruction branches based on a stored BHT, which improves processing times for calculation routines.
Page 66 The key point is that Java and C/C++ applications tend to use IEEE Binary Floating Point operations more frequently than earlier applications. This means that the better the hardware implementation of this set of instructions, the better the performance of e-business applications will be. IBM System z9 Enterprise Class Technical Guide...
Page 67 Hardware Decimal Floating Point Base 10 arithmetic is used for most business and financial computation. Floating point computation used for work typically done in decimal arithmetic has involved frequent necessary data conversions and approximation to represent decimal numbers. This has made floating point arithmetic complex and error prone for programmers using it in applications where the data is typically decimal data.
Page 68 Capacity Upgrade on Demand (CUoD), Customer Initiated Upgrade (CIU), On/Off Capacity on Demand (On/Off CoD), or Capacity BackUp (CBU). More information about capacity upgrades is provided in Chapter 8, “Concurrent upgrades and availability” on page 205. IBM System z9 Enterprise Class Technical Guide...
Page 69 PU sparing In the rare event of a PU failure, the failed PU’s characterization is dynamically and transparently reassigned to a spare PU. More information about PU sparing is provided in “Sparing rules” on page 67. A minimum of one PU per z9 EC server must be ordered as one of the following: A Central Processor (CP) An Integrated Facility for Linux (IFL) An Internal Coupling Facility (ICF)
Page 70 The z9 EC has a PU pool of 10 CPs, three zAAPs, two IFLs, and one ICF. The total pool weight for CPs is 1000, for zAAPs it is 200, for IFLs 400, and for ICFs 100. IBM System z9 Enterprise Class Technical Guide...
Page 71 1000 Total PUs (Physical) > Figure 2-15 z9 EC PU weighting For more information about PU pools and processing weights, refer to IBM System z9 Processor Resource/Systems Manager™ Planning Guide, SB10-7041. Central Processors A Central Processor is a PU that has the z/Architecture instruction sets. It can run z/Architecture based operating systems (z/OS, z/VM, z/TPF, z/VSE, Linux), and the Coupling Facility Control Code (CFCC).
Page 72 IFLs do not change the model capacity identifier of the z9 EC. Software product license charges based on the model capacity identifier are not affected by the addition of IFLs. IBM System z9 Enterprise Class Technical Guide...
Page 73 IFL is present. Internal Coupling Facilities An Internal Coupling Facility (ICF) is a PU used to run the IBM Coupling Facility Control Code (CFCC) for Parallel Sysplex environments. Within the capacity of the sum of all unassigned PUs in up to four books, up to 16 ICFs can be characterized, depending on the z9 EC model.
Page 74 The System z Application Assist Processor (zAAP) is a PU that is used exclusively for running Java application workloads under z/OS. zAAPs only run Java code. The IBM SDK for z/OS Java 2 Technology Edition (the Java Virtual Machine), in cooperation with z/OS and PR/SM, directs JVM™...
Page 75 Important: The zAAP is a specific example of an assist processor that is known generically as an Integrated Facility for Applications (IFA). The generic term IFA appears in panels, messages, and other online information relating to the zAAP. zAAPs and logical partition definitions zAAP processors can be defined as dedicated or shared processors in a logical partition and are always related to CPs of the same partition.
Page 76 Software support zAAPs do not change the model capacity identifier of the z9 EC. IBM software product license charges based on the model capacity identifier are not affected by the addition of zAAPs. z/OS Version 1.6 is the minimum level for supporting zAAPs, together with IBM SDK for z/OS Java 2 Technology Edition V1.4.1.
Page 77 Option 4: No Java crossover (IFACROSSOVER=NO) This option specifies that a standard CP will not look for Java work before entering the wait state. This option assures that all Java work is done on a zAAP as long as one is available.
Page 78 The total number of On/Off CoD Active zIIPs plus zIIPs cannot exceed the number of On/Off active CPs plus the number of CPs plus the number unassigned CPs on a z9 EC server. IBM System z9 Enterprise Class Technical Guide...
Page 79 For more information about CUoD, CIU, or On/Off CoD, see Chapter 8, “Concurrent upgrades and availability” on page 205. zIIPs and logical partition definitions zIIP processors can be defined as dedicated or shared in a logical partition. The number of zIIPs in a logical partition can be larger than the number of CPs in that logical partition.
Page 80 The z9 EC server comes with two spare PUs system wide. Depending on the model, CP, IFL, ICF, zAAP, zIIP, and SAP sparing is completely transparent and requires no operating system or operator intervention. IBM System z9 Enterprise Class Technical Guide...
Page 81 With transparent sparing, the status of the application that was running on the failed processor is preserved and will continue processing on a newly assigned CP, IFL, ICF, zAAP, zIIP, or SAP (allocated to one of the spare PUs) without customer intervention. If no spare PU is available, application preservation is invoked.
Page 82 Licensed Internal Code Configuration Control (LIC-CC) will determine how much memory is used from each card. The sum of the LIC-CC provided memory from each card is the amount available for use in the system. IBM System z9 Enterprise Class Technical Guide...
Page 83 Memory allocation Memory assignment or allocation is done at Power-on Reset (POR) when the system is initialized. Actually, PR/SM is responsible for the memory assignments; it is PR/SM that controls the resource allocation of the server. Table 2-2 on page 33 shows the distribution of physical memory across books when a system is initially installed with the amounts of where memory shown in the first column.
Page 84 HSA Estimator for z9 EC, available on Resource Link, to be able to plan for a sufficient amount of memory. When system is activated, the HSA is always allocated in the physical memory of book 0. However it can be moved to another book later. IBM System z9 Enterprise Class Technical Guide...
Page 85 2.3 Model configurations The z9 EC server model nomenclature is based on the number of PUs available for customer use in each configuration. Five models of the z9 EC server are available: Model S08 Eight PUs are available for characterization as CPs, IFLs, ICFs, up to four zAAPs or zIIPs, or up to five additional SAPs.
Page 86 CPs. Even so the maximum number of zIIPs amounts to 27 and requires an equal number of characterized CPs. The sum of all zAAPs, and zIIPs cannot be larger than two times the number of characterized CPs. IBM System z9 Enterprise Class Technical Guide...
Page 87 Not all PUs on a given model are required to be characterized. Only purchased PUs are identified by a feature code. Concurrent PU conversions Assigned CPs, assigned IFLs, unassigned IFLs, ICFs, zAAPs, and zIIPs may be converted to other assigned or unassigned feature codes. Valid conversions are: From a CP to an IFL, ICF, zAAP, or zIIP.
Page 88 Model capacity Model capacity identifier identifier identifier 1077 1850 1127 1889 1177 1927 1226 1936 1274 1998 1314 2033 1353 2067 1400 2101 1436 2135 1481 2168 1524 2201 1567 2233 1609 2265 IBM System z9 Enterprise Class Technical Guide...
Page 89 Model capacity Model capacity Model capacity identifier identifier identifier 1650 2295 1691 2324 1732 2353 1772 2381 1032 1811 2409 Table 2-14 Model capacity identifier and MSU values for subcapacity models Model capacity Model capacity Model capacity identifier identifier identifier Capacity BackUp (CBU) CBU deliver temporary backup capacity on top of what an installation might have installed in numbers of assigned CPs, IFLs, ICFs, zAAPs, zIIPs, and additional SAPs.
Page 90 CPs, but rather the total number of CPs that will make up the configuration when the CBU is activated. The total number of CPs when the CBU is activated is equal to the number of CPs ordered for CBU. IBM System z9 Enterprise Class Technical Guide...
Page 91 For example, when a z9 EC with model capacity identifier 504 specifies a CBU with six CP6, the CBU server being activated will have model capacity identifier 606. When the CBU configuration is activated, the server capacity will grow from 197 to 339 MSUs (see Example 2-2).
Page 92 Table 8-2 on page 223 shows all possible On/Off CoD CP upgrades for granular capacity models. For more information about temporary capacity increases, see Chapter 8, “Concurrent upgrades and availability” on page 205 IBM System z9 Enterprise Class Technical Guide...
Page 93 2.4 Logical partitioning Logical partitioning is a function implemented by the Processor Resource/Systems Manager (PR/SM), available on all z9 EC servers. The z9 EC only runs in LPAR mode. This means that virtually all system aspects are controlled by PR/SM functions. PR/SM is very much aware of the book structure on the z9 EC.
Page 94 LPAR Dynamic Storage Reconfiguration. Refer to 2.5.3, “LPAR Dynamic Storage Reconfiguration (DSR)” on page 87 for more information. IBM System z9 Enterprise Class Technical Guide...
Page 95 Channels Channels can be shared between logical partitions by including the partition name in the partition list of a Channel Path ID (CHPID). I/O configurations are defined by the I/O Configuration Program (IOCP) or the Hardware Configuration Dialog (HCD) in conjunction with the CHPID Mapping Tool (CMT).
Page 96 Linux-only mode, to run: – A Linux operating system, on either: • Dedicated shared IFLs • Dedicated shared CPs – A z/VM operating system, on either: • Dedicated shared IFLs • Dedicated shared CPs IBM System z9 Enterprise Class Technical Guide...
Page 97 Table 2-18 shows all LPAR modes, required characterized PUs, and operating systems, and which PU characterizations can be configured to a logical partition image. The available combinations of dedicated (DED) and shared (SHR) processors are also shown. For all combinations, a logical partition can also have Reserved Processors defined, allowing nondisruptive logical partition upgrades.
Page 98 Reconfiguration (DSR)” on page 87 for further details. Operating systems running under z/VM can exploit the z/VM capability of implementing virtual real memory to guest virtual machines. The z/VM dedicated storage can be “shared” between guest operating systems. IBM System z9 Enterprise Class Technical Guide...
Page 99 allocation usage Table 2-19 shows the z9 EC storage possibilities, depending on the image mode. Table 2-19 Storage definition and usage possibilities Image mode Architecture mode Maximum Central Expanded Storage (addressability) Storage Architecture z9 EC z9 EC Operating definition definable system usage Only by z/VM ESA/390...
Page 100 16 EB. However, the current z9 EC definition limit for logical partitions is 128 GB of storage. Expanded Storage cannot be defined for a Coupling Facility image. Only IBM Coupling Facility Control Code can run in Coupling Facility mode.
Page 101 A concurrent logical partition storage upgrade uses Dynamic Storage Reconfiguration (DSR) and the operating system must use the Reconfigurable Storage Unit (RSU) definition to be able to add or remove storage units in a nondisruptive way. Currently, only z/OS has this support.
Page 102 IBM System z9 Enterprise Class Technical Guide...
Page 103 The following topics are included: 3.1, “Overview” on page 90 3.2, “I/O cages” on page 91 3.3, “I/O and cryptographic feature cards” on page 97 3.4, “Connectivity” on page 101 © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 104 Up to 48 Inter-System Channel-3 (ISC-3) in peer mode only. Up to two External Time Reference (ETR). Up to eight Crypto Express2 features with two PCI-X cryptographic adapters each. Each adapter can be configured as a cryptographic coprocessor or cryptographic accelerator. IBM System z9 Enterprise Class Technical Guide...
Page 105 Note: The maximum number of Coupling Links combined (IC, ICB-3, ICB-4, and active ISC-3 links) cannot exceed 64 per server. The z9 EC has two frames. The A frame holds the CEC cage on top and one I/O cage on the bottom.
Page 106 The card provides the physical interface between the cage controller and the PSC boxes in the frame. The PSC24V card has two jacks for PSC connection and its installation is disruptive. IBM System z9 Enterprise Class Technical Guide...
Page 107 3.2.1 Self-Timed Interconnect (STI) There are up to eight Memory Bus Adapters (MBAs) fanout cards on each z9 EC book. The MBA fanout cards are numbered D1 to D8 and each have two Self-Timed Interconnect (STI) jacks, resulting in a total of 16 STI connections on each z9 EC book. Each STI has a bandwidth of 2.7 GB per second full-duplex, resulting in a maximum bandwidth of 43.2 GB per second per book.
Page 108 (STI-A4) is used. The STI-A4 mother card hosts two STI-MP cards and only has one connector (serving domain 6) to the planar board of the I/O cage. This way Redundant I/O Interconnect also applies. IBM System z9 Enterprise Class Technical Guide...
Page 109 This card setup provides the capability for Redundant I/O Interconnect, assuring connection to I/O devices even when a book, including its MBA/STI connections, is removed. Connection to I/O resources that would have been disconnected (after a book or MBA fanout removal for upgrade or repair) is maintained by using the STI connection from a different book or MBA fanout.
Page 110 Model S18 to a Model S38, the STI links for ICB-4 and STI-3 extender cards (ICB-3) are spread across all books’ MBAs, including the two newly installed books. The result is a balanced STI system, as though a new build z9 EC Model S38 server was initially configured. IBM System z9 Enterprise Class Technical Guide...
Page 111 Important: If the z9 EC STI Rebalance feature (FC 2400) is selected when the server is upgraded, this results in STI rebalancing for ICBs. The z9 EC STI Rebalance feature will change the Physical Channel ID (PCHID) number of ICB-4 links requiring a corresponding update on the server I/O definition through HCD/HCM.
Page 112 Half high card in top of slot xx 0218 ISC D <10KM 3324 FICON Express4 LX (4 port) 3364 OSA Express GbE LX (2 port) 0863 Crypto Express2 Figure 3-4 PCHID Report example IBM System z9 Enterprise Class Technical Guide...
Page 113 I/O slot 01 has an ISC-3 Daughter (ISC-D) half-high card (FC 0218) in the top, connected to STI 0 (Jack J00) from MBA fanout D8 of book 1. Its two enabled ports have PCHID numbers 100 and 101. I/O slot 02 has an ISC-3 Daughter (ISC-D) half-high card (FC 0218) in the top, connected to STI 0 (Jack J00) from MBA fanout D8 of book 0.
Page 114 HCD/HCM, or IOCP. The CHPID assignment associates the CHPID number to a physical channel port location (PCHID). HiperSockets (IQD) and IC links (ICP) do not have PCHIDs, as they are virtual and not physical links, but they do require CHPID numbers. IBM System z9 Enterprise Class Technical Guide...
Page 115 OSA Express2 cards to support Gigabit Ethernet (GbE), 1000BASE-T Ethernet, 100BASE-T Ethernet, and 10BASE-T Ethernet. For additional, detailed information about z9 EC connectivity, see the IBM Redbooks publication IBM System z Connectivity Handbook, SG24-5444. 3.4.1 I/O and cryptographic features support and configuration rules...
Page 116 – Internal Coupling (IC) links. Each IC link pair requires two CHPID numbers. – HiperSockets, also called Internal Queued Direct I/O (iQDIO). Up to 16 virtual LANs can be defined, each one requiring a CHPID number. IBM System z9 Enterprise Class Technical Guide...
Page 117 ICB (copper) cables are orderable. All other cables have to be sourced separately. IBM Fiber Cabling Services offer a total cable solution service to help with cable ordering needs, and is highly recommended. These services take into consideration the requirements...
Page 118 Category 5 UTP 1. SM is single mode fiber. 2. MM is multimode fiber. 3. OSA-E refers to OSA-Express. 4. Brought forward to z9 EC on an upgrade only. 5. UTP is Unshielded Twisted Pair. IBM System z9 Enterprise Class Technical Guide...
Page 119 EC. Fiber optic cables, cable planning, labeling, and installation are all customer responsibilities for new z9 EC installations and upgrades. IBM Fiber Cabling Services offer a total cable solution service to help with cable ordering needs, and is highly recommended.
Page 120 69 16-port ESCON channel cards to be installed. The z9 EC Model S08 can have up to 960 ESCON ports, on 64 channel cards. This number is limited by the number of available STIs on the S08 model. IBM System z9 Enterprise Class Technical Guide...
Page 121 EC server maintenance package Repair and Verify procedure. If sparing can take place, the IBM SR moves the external fiber optic cable from the failing port to the spare port. When sparing occurs, the PCHID moves to the spare port (PCHID 112 in Figure 3-6). If sparing cannot be performed, the 16-port ESCON card is replaced.
Page 122 Fibre Channel Switches that support 1 Gbps, 2 Gbps, 4 Gbps, and FICON LX Fibre Channels. Control units that support 1 Gbps, 2 Gbps, and 4 Gbps FICON LX Fibre Channels. FICON channels in Fibre Channel Protocol (FCP) mode. IBM System z9 Enterprise Class Technical Guide...
Page 123 LC Duplex connector. Unrepeated distances vary with the use of 50 micron or 62.5 micron fiber optic cable and the data rate. Note: IBM has qualified the 50 micron multimode 2000 MHz-km ISO/IEC OM3, TIA 850 nanometer laser-optimized 50/125 micrometer fiber optic cable for use when attaching System z to servers, switches/directors, disks, tapes, and printers.
Page 124 The port supports connection to a 62.5 micron or 50 micron multimode fiber optic cable terminated with an LC Duplex connector. FICON Express FICON Express features (FC 2319 and FC 2320) are carried forward to the z9 EC when upgrading from a z900 or z990 server. IBM System z9 Enterprise Class Technical Guide...
Page 125 Each port supports attachment to the following: only. FICON LX Bridge one port feature of an IBM 9032 ESCON Director at 1 Gbps Fibre Channel Switches that support 1 Gbps and 2 Gbps FICON LX Fibre Channels. Control units that support 1 Gbps and 2 Gbps FICON LX Fibre Channels.
Page 126 SFP. The data flow on the unaffected channels on the same feature can continue. A problem with one FICON port may not require replacement of a complete feature. IBM System z9 Enterprise Class Technical Guide...
Page 127 In a fiber optic infrastructure with long distances, cabling problems can be challenging. To address this issue, FICON supports Request Node Identification Data (RNID). FICON Link incident reports may be captured and analyzed. FICON purge path support takes care of reporting error-related data on a FICON path.
Page 128 Page-Management Assist assists z/VM with hardware assign, lock, and unlock of page frames without z/VM intervention. The performance assists not only apply to FCP, but also to HiperSockets (CHPID type IQD), and all OSA features with CHPID type OSD specified. IBM System z9 Enterprise Class Technical Guide...
Page 129 3.4.4 OSA-Express2 and OSA-Express adapters What follows is a discussion of the connectivity options by both the OSA-Express2 and OSA-Express environments. The following OSA-Express2 features can be installed on new build z9 EC systems: OSA-Express2 Gigabit Ethernet (GbE) Long Wavelength (LX), feature code 3364 OSA-Express2 Gigabit Ethernet (GbE) Short Wavelength (SX), feature code 3365 OSA-Express2 Gigabit Ethernet 10 GbE LR, feature code 3368 OSA-Express2 1000BASE-T Ethernet, feature code 3366...
Page 130 OSA-Express2 feature port ability to auto-negotiate with its attached Ethernet switch. The OSA-Express2 1000BASE-T Ethernet feature supports CHPID types, OSC, OSD, OSE, and OSN. For OSN mode, see “OSA-Express2 OSN - Open System Adapter for NCP” on page 121. IBM System z9 Enterprise Class Technical Guide...
Page 131 Any one of the following settings for the OSA-Express2 1000BASE-T Ethernet and OSA-Express2 1000BASE-T Ethernet features can be chosen: Auto-negotiate 10 Mbps half-duplex or full-duplex 100 Mbps half-duplex or full-duplex 1000 Mbps / 1 Gbps full-duplex LAN speed and duplexing mode default to auto negotiation. The feature port and the attached switch automatically negotiate these settings.
Page 132 Each port supports connection to either a 100 Mbps or 10 Mbps Ethernet LAN. The LAN must conform either to the IEEE 802.3 (ISO/IEC 8802.3) standard or the Ethernet V2.0 specifications, and the 10BASE-T or 100BASE-TX standard transmission schemes. IBM System z9 Enterprise Class Technical Guide...
Page 133 Each port has an RJ-45 receptacle for cabling to an Ethernet switch that is appropriate for the LAN speed. The RJ-45 receptacle is required to be attached using an EIA/TIA category 5 unshielded twisted pair (UTP) cable with a maximum length of 100 m (328 ft). It is possible to choose any one of the following settings for the OSA-Express FENET feature: Auto negotiate 10 Mbps half-duplex or full-duplex...
Page 134 TCP/IP support in both Linux on System z and z/VM. Adapter interruptions apply to all of the OSA-Express features available on z9 EC, whether offered as a new build or on an upgrade when in QDIO mode (CHPID type OSD). IBM System z9 Enterprise Class Technical Guide...
Page 135 OSA-Express2 GbE and OSA-Express2 1000BASE-T Ethernet features can provide channel connectivity from an operating system in a z9 EC to the IBM Communication Controller for Linux on System z (CCL) with the Open Systems Adapter for NCP, in support of the Channel Data Link Control (CDLC) protocol (see Figure 3-7).
Page 136 Allow WebSphere Application Server content-based routing to work with z/OS in an IPv6 network. Allow z/OS to use a "standard" interface ID for IPv6 addresses. Remove the need for PRIROUTER/SECROUTER function in z/OS. IBM System z9 Enterprise Class Technical Guide...
Page 137 OSA Layer 3 VMAC is exclusive to System z9, and is applicable to the OSA-Express2 and OSA-Express features when configured as CHPID type OSD (QDIO), and is supported by z/OS V1.8. OSA-Express2 QDIO Diagnostic Synchronization QDIO Diagnostic Synchronization is designed to provide system programmers and network administrators the ability to coordinate and simultaneously capture both software and hardware traces.
Page 138 Internal Channel, IC in Peer mode; Licensed Internal Code (LIC-CC) function defined using HCD/IOCP ISC-3 link The ISC-3 feature is made up of the following feature codes: ISC-3 mother card, feature code 0217 ISC-3 daughter card, feature code 0218 ISC-3 Port, feature code 0219 IBM System z9 Enterprise Class Technical Guide...
Page 139 The ISC-3 mother card occupies one slot in the I/O cage. The ISC-3 mother card supports up to two ISC-3 daughter cards. Each ISC-3 daughter card has two independent ports with one PCHID associated with each active port. The ISC-3 ports are activated through Licensed Internal Code (LIC-CC).
Page 140 9037 Sysplex Timer. The 9037 Sysplex Timer provides the synchronization for the Time-of-Day (TOD) clocks of multiple servers, and thereby allows events started by different servers to be properly sequenced in time. When multiple servers IBM System z9 Enterprise Class Technical Guide...
Page 141 EC. Fiber optic cables, cable planning, labeling, and installation are all customer responsibilities for new z9 EC installations and upgrades. IBM Fiber Cabling Services offer a total cable solution service to help with cable ordering needs, and is highly recommended.
Page 142 IBM System z9 Enterprise Class Technical Guide...
Page 143 Control unit overhead for data sets that utilize striping and compression (VSAM, PDSE, HFS, and zFS). The following topics are discussed: 4.1, “Channel Subsystem (CSS)” on page 130 4.1.9, “Configuration management” on page 143 4.2, “The MIDAW facility” on page 144 © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 144 Subchannels from the SCP to the channel subsystem. Channels Channels: The communication path from the channel subsystem to the I/O network and the connected control units / devices. Figure 4-1 Channel Subsystem overview IBM System z9 Enterprise Class Technical Guide...
Page 145 The structure provides up to four Channel Subsystems (see Figure 4-2). Each CSS has from one to 256 CHPIDs, and may be configured with up to 15 logical partitions that relate to that particular Channel Subsystem. CSSs are numbered from 0 to 3, and are sometimes referred to as the CSS Image ID (CSSID 0, 1, 2, and 3).
Page 146 The channel definitions of a CSS are not bound to a single book. A CSS may define resources that are physically connected to any STIs of any book in a multi-book z9 EC. To keep the diagram simple, Model S54 is not represented. IBM System z9 Enterprise Class Technical Guide...
Page 147 Each set provides 64 K addresses. (Subchannel set 0, the first set, still reserves subchannels for IBM use although the number of reserved subchannels is being reduced from 1024 to 256.) Subchannel set 1 provides a full range of 64 K subchannels.
Page 148 The appropriate subchannel set number must be included in IOCP definitions (or in the HCD definitions that produce the IOCDS). The subchannel set number defaults to zero and IOCP changes are needed only when using subchannel set 1. IBM System z9 Enterprise Class Technical Guide...
Page 149 63.75 K subchannels Systems prior to the z9 EC reserved 1024 subchannels out of the potential maximum of 64 K subchannels. The z9 EC has reduced this reserved number to 256 subchannels, thus increasing the number of subchannels available. The reserved subchannels only exist in subchannel set 0;...
Page 150 IOCDS that is used for Power-On Reset. A reserved partition is defined with a partition name placeholder, a MIF ID, a usage type, and a description. The reserved partition can be assigned a logical partition name to be later used in I/O commands of HCD. IBM System z9 Enterprise Class Technical Guide...
Page 151 Important: Some HCD and HCM panels may still refer the user to the definition of a logical partition number . For a z9 EC configuration, this is incorrect, and the user should understand that the panel refers to the definition of a MIF ID. As previously mentioned, on a z9 EC, the logical partition number is assigned by PR/SM during Power-on Reset and cannot be visualized or modified by the user.
Page 152 HCD. In the case of internal channels (for example, IC links and HiperSockets), the same applies, but there is no PCHID association. They are defined with the same CHPID number in multiple CSSs. IBM System z9 Enterprise Class Technical Guide...
Page 153 CHPIDs that span CSSs reduce the total number of channels available on the z9 EC. The total is reduced, since no CSS can have more than 256 CHPIDs. For a z9 EC with two CSSs, a total of 512 CHPIDs are supported. If all CHPIDs are spanned across the two CSSs, then only 256 channels are supported.
Page 154 Figure 4-9 IOCP Example with CSSs, MSS, and PCHIDs MSG1='IODF01', MSG2='MY.IODF01.WORK - 2006-03-23 10:44', SYSTEM=(2094,1), TOK=('SCZP101',008001136A3A2084135941210105143F00000000,* 00000000,'06-03-23','10:44',' RESOURCE PARTITION=((CSS(0),(A01,1),(A02,2)),(CSS(1),(A11,1),(* A12,2)),(CSS(2),(A21,1),(A22,2))), MAXDEV=((CSS(0),65280,0),(CSS(1),65280,0),(CSS(2),65280,* 65535)) CHPID PATH=(CSS(0,1,2),80),SHARED,SWITCH=61,PCHID=140,TYPE=FC CHPID PATH=(CSS(0,1,2),81),SHARED,SWITCH=61,PCHID=1C0,TYPE=FC CHPID PATH=(CSS(0,1,2),82),SHARED,SWITCH=62,PCHID=240,TYPE=FC CHPID PATH=(CSS(0,1,2),83),SHARED,SWITCH=62,PCHID=340,TYPE=FC CNTLUNIT CUNUMBR=8000, PATH=((CSS(0),80,81,82,83),(CSS(1),80,81,82,83),(CSS(2),* 80,81,82,83)),UNITADD=((00,256)), LINK=((CSS(0),08,0C,08,0C),(CSS(1),08,0C,08,0C),(CSS(2),* 08,0C,08,0C)),CUADD=0,UNIT=2105 IODEVICE ADDRESS=(8000,069),CUNUMBR=(8000),STADET=Y,UNIT=3390B IODEVICE ADDRESS=(8045,187),CUNUMBR=(8000),STADET=Y, SCHSET=((CSS(2),1)),UNIT=3390A IBM System z9 Enterprise Class Technical Guide...
Page 155 Key elements in this IOCP include the following: Three Channel Subsystems (CSS0, CSS1, and CSS2) are defined. The CSSs used in the defines IOCDS created from this IOCP are stated in the RESOURCE statement. This CSSs. Two logical partitions are defined in each Channel Subsystem (A01, A02, and so forth). These are also defined in the RESOURCE statement.
Page 156 IODF format will not be saved and the copy on disk will remain in the V4 IODF format. After migration to the Version 5 IODF, only a z/OS V1R7 system can make updates to this IODF version. IBM System z9 Enterprise Class Technical Guide...
Page 157 Tools are provided to help maintain and optimize the I/O configuration of a z9 EC. IBM Configurator for e-business (e-Config) The e-Config tool is available to your IBM representative. It is used to configure new configurations or upgrades of an existing configuration, and maintains installed features of those configurations.
Page 158 ESCON or FICON channel, an OSA port, or an ISC-3 link is shared across logical partitions on IBM System z9 server. When an I/O card is replaced for a repair, it usually has some failed channels and some still functioning channels.
Page 159 MIDAW is for Modified IDAW. An IDAW is an Indirect Address Word that is used to specify data addresses for I/O operations in a virtual environment. The existing IDAW design allows the first IDAW in a list to point to any address within a page. Subsequent IDAWs in the same list must point to the first byte in a page;...
Page 160 MIDAW operations are on a single I/O block, in the manner of command data chaining. Do not confuse this operation with CCW chaining. IBM System z9 Enterprise Class Technical Guide...
Page 161 Extended format data sets z/OS extended format data sets use internal structures (usually not visible to the application program) that require scatter-read (or scatter-write) operation. This means that CCW data chaining is required and this produces less than optimal I/O performance. Since the most significant performance benefit of MIDAWs is achieved with Extended Format (EF) data sets, a brief review of the EF data sets are included here.
Page 162 The performance of a specific workload may vary according to the conditions and hardware configuration of the environment. IBM laboratory tests found that DB2 gains significant performance benefits using the MIDAW facility in the following areas: Table scans Logging Utilities...
Page 163 5.2, “CP Assist for Cryptographic Function (CPACF)” on page 154 5.3, “Crypto Express2” on page 154 5.4, “TKE workstation feature” on page 159 5.5, “Cryptographic functions comparison” on page 160 5.6, “Software support” on page 161 © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 164 5.1 Cryptographic functions The z9 EC includes both standard cryptographic hardware and optional cryptographic features for flexibility and growth capability. IBM has a long history of providing hardware cryptographic solutions, from the development of Data Encryption Standard (DES) in the 1970s to delivering integrated cryptographic hardware in a server to achieve the US Government's highest FIPS 140-2 Level 4 rating for secure cryptographic hardware.
Page 165 DES key generation and distribution PIN generation, verification, and translation functions Pseudo Random Number Generator (PRNG) Public Key Algorithm (PKA) Facility These commands are intended for application programs using public key algorithms, including: – Importing RSA public-private key pairs in clear and encrypted forms –...
Page 166 Key Exchange with Non-CCA Cryptographic Systems Allows for the changing of the operational keys between the remote site and the non-CCA system like the ATM. IBM Common Cryptographic Architecture (CCA) employs Control Vectors to control usage of cryptographic keys. Non-CCA systems use other mechanisms, or may use keys that have no associated control information.
Page 167 The Web site will direct the customer's request to an IBM Global Services location appropriate for the customer's geographic location. A special contract will be negotiated between IBM Global Services and the customer, covering development of the UDX by IBM Global Services per the customer's specifications, as well as an agreed-upon level of the UDX.
Page 168 When the PCI-X adapter is configured as a coprocessor, the adapter also provides equivalent functions (plus some additional) as the PCI CA card on previous systems with the same level of performance. IBM System z9 Enterprise Class Technical Guide...
Page 169 16 logical partitions per adapter). To have access to the adapter requires setup in the image profile for the partition, the adapter must be in the candidate list. For details on setting up the image profile, refer to IBM Systems z9-109 Configuration Setup SG24-7203. 5.3.1 Crypto Express2 coprocessor...
Page 170 A Crypto Express2 coprocessor operates with the Integrated Cryptographic Service Facility (ICSF) and IBM Resource Access Control Facility (RACF®), or equivalent software products, in a z/OS operating environment to provide data privacy, data integrity, cryptographic key installation and generation, electronic cryptographic key distribution, and personal identification number (PIN) processing.
Page 171 Note: Cryptographic coprocessors are not tied to logical partition numbers or MIF IDs. They are set up with PCI-X adapter numbers and domain indices that are defined in the partition image profile. The customer can assign them to the partition and change or clear them when needed.
Page 172 For availability, assignment of multiple PCI-X adapters of the same type (Crypto Express2 accelerator or coprocessor) to one logical partition should be spread across multiple features. IBM System z9 Enterprise Class Technical Guide...
Page 173 The Crypto Express2 feature is supported on the z990 (as a PCI-X cryptographic coprocessor only) and is carried forward when upgraded to a z9 EC. 5.4 TKE workstation feature The TKE workstation is an optional feature that offers key management functions. The TKE 5.0 workstation (with TKE 5.0 or later Licensed Internal Code) is required to support cryptographic key management on the z9 EC.
Page 174 Crypto performance RMF™ monitoring Requires system master keys to be loaded System (master) key storage Retained key storage Tamper-resistant hardware packaging Designed for FIPS 140-2 Level 4 certification Supports SSL functions IBM System z9 Enterprise Class Technical Guide...
Page 175 Functions or attributes CPACF Crypto Express2 Crypto Express2 Coprocessor Accelerator Supports Linux applications doing SSL handshakes RSA functions High performance SHA-1, and SHA-256 Clear key DES/T-DES Advanced Encryption Standard (AES) for 128-bit keys Pseudo Random Number Generation (PRNG) Clear key RSA Double length DUKPT support Europay Mastercard VISA (EMV) support Public Key Decrypt (PKD) support for...
Page 176 See the following sources for Web deliverables. For z/OS and z/OS.e Web deliverables, check the z/OS Web site at: http://www.ibm.com/eserver/zseries/zos/downloads For Linux on System z, support is delivered through IBM and distribution partners. For more information, check the DeveloperWorks Web site at: http://www.ibm.com/developerworks/linux/linux390/ 5.6.4 z/OS ISCF FMIDs...
Page 177 Table 5-4 lists the ICSF FMIDs and Web deliverables for z/OS V1.6 to V1.8. Table 5-4 z/OS ICSF FMIDs z/OS and ICSF Web deliverable name supported function z/OS.e FMID HCR770B z990 and z890 Enhancements to PCI-X Adapter Coprocessor only Cryptographic Support V1.6 HCR7720 ICSF 64-bit Virtual Support for...
Page 178 IBM System z9 Enterprise Class Technical Guide...
Page 179 Preventive Service Planning (PSP) bucket for 2094DEVICE. The System z9 EC functions that are supported will depend on the operating system version and release. © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 180 TPF V4R1 z/TPF V1R1 a. Linux 64-bit distributions are supporting the IBM System z architecture. System z servers can also run code built for the 31-bit mainframe systems. Note: Refer to the z/OS, z/VM, z/VSE, and z/TPF subsets of the 2094DEVICE Preventative Planning (PSP) bucket prior to installing an IBM System z9 EC.
Page 181 6.2 Support by operating system In this section, we discuss support by operating system. 6.2.1 z/OS Table 6-2 summarizes the z9 EC functions’ support requirements for current z/OS releases. Table 6-2 z/OS support summary Function z/OS V1R9 z/OS V1R8 z/OS V1R7 z/OS V1R6 z9 EC Supported...
Page 182 Support on z9 EC requires at a minimum Separate LPAR management of PUs z/VM V5.1. 60 logical partitions z/VM V5.1. Hardware Decimal Floating Point z/VM V5.2 (guest support). CPACF z/VM V5.1. Enhancements to CPACF z/VM V5.1. IBM System z9 Enterprise Class Technical Guide...
Page 183 Feature Support on z9 EC requires at a minimum Crypto Express2, compatibility support z/VM V5.1 (guest support). Crypto Express2, exploitation support when a z/VM V5.1 (guest support). PCI-X adapter is configured as an accelerator or a coprocessor Remote key loading for ATMs, ISO 16609 CBC z/VM 5.1 (guest support).
Page 184 CHPID type CHPID type CHPID type CHPID type OSN OSA-Express2 z/VSE V3.1 z/VSE 3.1 Gigabit Ethernet OSA-Express2 z/VSE V3.1 z/VSE V3.1 z/VSE V3.1 z/VSE 3.1 1000BASE-T Ethernet OSA-Express2 10 z/VSE V3.1 Gigabit Ethernet LR IBM System z9 Enterprise Class Technical Guide...
Page 185 System-initiated CHPID reconfiguration Note a. IBM is working with its Linux distribution partners so that this function will be provided in future Linux on System z distribution releases or service updates. b. IBM is working with its Linux distribution partners on kernel space exploitation.
Page 186 V5 R1 Supported z/VSE V4 R1 Supported z/VSE V3 R1 Supported z/TPF V1 R1 Supported TPF V4 R1 Supported Linux for System z Distributions of SUSE SLES 9 and Red Hat RHEL 4 IBM System z9 Enterprise Class Technical Guide...
Page 187 Single system image A single system image can control several processors (CPs, zIIPs, zAAPs, or IFLs, as appropriate). Table 6-12 shows the maximum number of processors supported for each operating system image. Table 6-12 Single system image software support Operating system Maximum number of CPs+zIIPs+zAAPs or IFLs per system image z/OS V1R6 and above...
Page 188 V1R7 and above Included. Linux on System z IBM is working with its Linux distribution partners so that this function will be provided in future Linux on System z distribution releases or service updates. Exploitation of Multiple Subchannel Sets is supported in z/OS V1R6.
Page 189 Supported. Linux on System z SUSE SLES 9 SP3 a. IBM is working with its distribution partners to provide this function in future distribution re- leases or service updates. FICON link incident reporting FICON link incident reporting is designed to allow an operating system image (without operator intervention) to register for link incident reports.
Page 190 Channel Data Link Control (CDLC), when used with the Communication Controller for Linux, emulates selected functions of IBM 3745/NCP operations. The port used with the OSN support appears as an ESCON channel to the operating system. This support may be used with any OSA-Express2 feature, except for the 10 GbE LR feature.
Page 191 OSA-Express2 10 Gigabit Ethernet LR Table 6-24 lists the minimum support requirements for OSA-Express2 10 Gigabit (CHPID type OSD). Table 6-24 Minimum support requirements for OSA-Express2 10 Gigabit (CHPID type OSD) Operating system Support requirements z/OS V1R6 and above Supported z/VSE V3.1 and above TPF 4.1 At PUT 13 with PTFs...
Page 192 6.4 Software licensing considerations The IBM System z9 mainframe software portfolio includes operating system software (that is, z/OS, z/TPF, z/VM, z/VSE, and VSE/ESA) and middleware that runs on these operating systems.
Page 193 SALC. Alternatively, you can still choose to license WebSphere MQ under WLC. A reporting function, which IBM provides in the operating system IBM Software Usage Report Program, is used to calculate the daily MSU number. The rules to determine the billable...
Page 194 If the set CPUID command has been issued, bits 0–7 are set to ‘FF’ by z/VM and bits 8–31 are set to the value entered in the set CPUID command. Bits 32–63 are the same as they would have been without running as a z/VM guest. IBM System z9 Enterprise Class Technical Guide...
Page 195 Table 6-28 shows the possible output returned to the issuing program for an operating system running as a guest under z/VM. Table 6-28 STIDP output for z9 EC, VM guest Version CPU identification number Machine logical code type number partition 2-digit indicator Bit position...
Page 196 Figure 6-1 Node descriptor information 6.6 References For the most current planning information, check the Support Web page for each operating system: z/OS http://www.ibm.com/systems/support/z/zos/ z/VM http://www.ibm.com/systems/support/z/zvm/ z/TPF http://www.ibm.com/software/htp/tpf/pages/maint.htm z/VSE http://www.ibm.com/servers/eserver/zseries/zvse/support/preventive.html Linux on System z http://www.ibm.com/systems/z/os/linux/ IBM System z9 Enterprise Class Technical Guide...
Page 197 The following topics are included: 7.1, “Parallel Sysplex” on page 184 7.2, “Coupling Facility considerations” on page 185 7.3, “System-managed CF structure duplexing” on page 195 7.4, “Intelligent Resource Director” on page 197 © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 198 (see Parallel Sysplex Application Considerations, SG24-6523). High capacity: The Parallel Sysplex environment can scale, in a nearly linear fashion, from two to 32 images. The aggregated capacity of this configuration meets every processing requirement known today. IBM System z9 Enterprise Class Technical Guide...
Page 199 Dynamic workload balancing: The entire Parallel Sysplex cluster can be viewed as a single logical resource to users and business applications. Work can be directed to any like operating system image in a Parallel Sysplex cluster having available capacity. Workload management permits you to run diverse applications across a Parallel Sysplex cluster while maintaining the response levels critical to your business.
Page 200 Even though multiple servers can connect to only one Sysplex Timer unit, the typical configuration is usually connected to two different Sysplex Timer units in an Expanded Availability configuration. Refer to IBM System z Connectivity Handbook , SG24-5444, for IBM 9037 Sysplex Timer connectivity information.
Page 201 state. In this state, the port is disabled from stepping, but still receives configuration data from the attached Sysplex Timer. Figure 7-3 HMC workplace - System (Sysplex) Time, ETR configuration After IPL, the configuration can be identified to any attached z/OS image by issuing the Display ETR command.
Page 202 A zSeries server will provide the intermediate CFs that can act as a bridge to 9672 G5/G6 and z9 EC logical partitions in the same Sysplex, until all the z/OS images can be migrated off the 9672 servers. IBM System z9 Enterprise Class Technical Guide...
Page 203 The level of Coupling Facility Control Code (CFCC) must also be considered. See Table 7-1 for Coupling Facility Control Code requirements when the Coupling Facility resides on a non-z9 EC and is connected to a z/OS image on a z9 EC, or when CF duplexing is used and one Coupling Facility resides on the z9 EC.
Page 204 Table 7-3 shows the z9 EC coupling link maximums. Table 7-3 z9 EC coupling link maximums Link type STP supported z9 EC max ISC-3 ICB-3 ICB-4 Maximum number of external and internal coupling links combined per z9 EC IBM System z9 Enterprise Class Technical Guide...
Page 205 Table 7-4 lists the coupling link connectivity options for the various servers. Table 7-4 z9 coupling link connectivity Connectivity options z9 ISC-3 z9 ICB-3 z9 ICB-4 z9, z890, and z990 ISC-3 2 Gbps Peer Mode z900 and z800 ICB-3 1 GBps, Peer mode z9, z990, and z890 ICB-3 1 GBps, Peer mode, recommendation use ICB-4...
Page 206 ICFs. It can also use dedicated or shared CPs. With Dynamic ICF expansion, a Coupling Facility image can also use dedicated ICFs and shared CPs. The z9 EC can have ICF processors defined to CF images. IBM System z9 Enterprise Class Technical Guide...
Page 207 A Coupling Facility image can have one of the following combinations defined in the image profile: Dedicated ICFs Shared ICFs Dedicated shared ICFs Dedicated CPs Shared CPs Dedicated ICFs shared CPs Shared ICFs add flexibility. However, running with shared Coupling Facility Processor Units (ICFs or CPs) only is not a recommended production configuration.
Page 208 Dynamic CF Dispatch function must be activated. Dynamic ICF expansion is available on models that have at least one ICF. Dynamic ICF expansion requires that Dynamic CF Dispatching be activated (DYNDISP ON). IBM System z9 Enterprise Class Technical Guide...
Page 209 Test Sysplex Partition z/OS z/OS z/OS CF CF z/OS Image Test Test Prod Prod Profile Shared Pool of ICF PUs Dedicated ICFs used for most of the CF requests Shared ICFs used when requests rates to CF become excessive Setup Shared ICFs used for take over or spikes (such as alternate CF removed for maintenance or outage occurs) DYNDISP ON (CFCC cmd)
Page 210 EC, z9 BC, z990, z890, z900, z800 z9 EC, z9 BC, z990, z890, z900, z800 Figure 7-9 System-managed CF structure duplexing A technical paper on system-managed CF structure duplexing is available at: http://www-1.ibm.com/servers/eserver/zseries/library/techpapers/gm130103.html IBM System z9 Enterprise Class Technical Guide...
Page 211 7.4 Intelligent Resource Director Intelligent Resource Director (IRD) is only available on System z running z/OS. IRD is a function that optimizes processor CPU and channel resource utilization across logical partitions within a single System z server. IRD overview The Intelligent Resource Director (IRD) is a z/OS feature, extending the concept of goal-oriented resource management by allowing you to group system images that are resident on the same System z server running in LPAR mode, and in the same Parallel LPAR cluster...
Page 212 CPs in relation to the number of equivalent physical CPs, the higher the PR/SM overhead. This is because PR/SM has to do more processing to manage the number of logical CPs that exceeds the number of equivalent physical CPs. IBM System z9 Enterprise Class Technical Guide...
Page 213 z/OS gets more control over how CP resources are distributed. Using CPU management, z/OS is able to manage CP resources in relation to WLM goals for work. This was not possible in the past when a logical partition had CP resources assigned and used these as best it could in one logical partition.
Page 214 DCM is sensitive to bottlenecks, such as: – ESCON channel cards – Processor Self-Timed Interconnect – MBA fanout cards – Books – Director port cards – Control Unit I/O bay – Control Unit Interface card – ESCON Director IBM System z9 Enterprise Class Technical Guide...
Page 215 7.4.3 Channel Subsystem Priority Queuing Channel Subsystem (CSS) Priority Queuing is a function available on System z processors in LPAR mode. It allows the z/OS operating system to specify a priority value when starting an I/O request. When there is contention causing queuing in the Channel Subsystem, the request is prioritized by this value.
Page 216 A range of eight values is recommended for CSS I/O priority-capable logical partitions. If the logical partition is run in compatibility mode or with I/O priority management disabled, the I/O priority is set to the middle of the specified range. IBM System z9 Enterprise Class Technical Guide...
Page 217 Table 7-6 WLM CSS priority range mapping with specified range less than eight WLM CSS priorities Calculated Specified (range width) range (8) range (7) System work. Importance of one and two missing goals. Importance of three and four missing goals. Meeting goals.
Page 218 VARY SWITCH commands to re-enable the prohibited subsystem ports to Dynamic Channel Path Management. When ports are unprohibited or unblocked, these operations need to be followed, as necessary, by VARY SWITCH commands to bring ports ONLINE to Dynamic Channel Path Management. IBM System z9 Enterprise Class Technical Guide...
Page 219 8.1, “Availability enhancements” on page 206 8.2, “Concurrent upgrades” on page 207 8.3, “Enhanced Book Availability (EBA)” on page 232 8.4, “Enhanced Driver Maintenance (EDM)” on page 242 8.5, “Nondisruptive upgrades” on page 244 © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 220 Now let us look in more detail at the key On Demand capabilities for permanent, temporary, emergency, and disaster recovery increase in capacity of the z9 EC. IBM System z9 Enterprise Class Technical Guide...
Page 221 8.2 Concurrent upgrades The z9 EC has the capability of concurrent upgrades, providing additional capacity with no server outage. In most cases, with prior planning and operating system support, a concurrent upgrade can also be nondisruptive to the operating system. Given today's business environment, the benefits of the concurrent capacity growth capabilities provided by the z9 EC are plentiful, and include: Enabling exploitation of new business opportunities...
Page 222 Customer Initiated Upgrade (CIU), or the On/Off Capacity on Demand (On/Off CoD) functions. concurrent and permanent CUoD and CIU are functions available to enable capacity growth. concurrent and temporary On/Off CoD function enables capacity growth. IBM System z9 Enterprise Class Technical Guide...
Page 223 A CBU contract must be in place before the special code that enables this capability can be loaded on the server. The standard CBU contract provides for five 10 days tests and one 90 day disaster activation over a five year period. Contact your IBM Representative for details.
Page 224 CUoD upgrades for I/O are done by either: – LIC-CC activating additional ports on already installed ESCON and ISC-3 cards – Installing additional I/O cards and supporting infrastructure if required on already installed I/O cages IBM System z9 Enterprise Class Technical Guide...
Page 225 CUoD is ordered as a “normal” upgrade, also known as Miscellaneous Equipment Specification (MES). CUoD requires IBM service personnel for the upgrade. In most cases, a very short period of time is required for the IBM personnel to install the LIC-CC and complete the upgrade.
Page 226 CP resources to be available to the operating system. While these actions do not require a Power On Reset (POR), they are disruptive to the logical partition(s) requiring the change. IBM System z9 Enterprise Class Technical Guide...
Page 227 Attention: Up to 54 logical processors, including reserved processors, can be defined to a logical partition. You should not define more processors to a logical partition than the target operating system supports. V1R6 supports up to 32 processors, as a combination of CPs, zAAPs, and zIIPs.
Page 228 If reserved storage has not been defined to the logical partition, the logical partition will have to be deactivated, the image profile changed, and the logical partition IBM System z9 Enterprise Class Technical Guide...
Page 229 reactivated to allow the additional storage resources to be available to the operating system image. Concurrent memory upgrades also require that the memory must not be running in degraded mode. CUoD for I/O concurrently CUoD for I/O can add, , more I/O ports to a z9 EC by either: Enabling additional ports on the already installed I/O cards through LIC-CC LIC-CC-only upgrades can be done for ESCON channels and ISC-3 links, activating ports on the existing 16-port ESCON or ISC-3 daughter (ISC-D) cards.
Page 230 Customer Initiated Upgrade (CIU) is the capability for the z9 EC to initiate a upgrade for CPs, ICFs, IFLs, zAAPs, zIIPs, or memory through the Web, using IBM Resource Link. CIU is similar to CUoD, but the additional resources are added by the customer. The customer also has the ability to unassign previously purchased CPs and IFLs processors through CIU.
Page 231 Ordering and activation of the upgrade is accomplished by the customer logging on to the IBM Resource Link Web site and invoking the CIU application to upgrade a server for CPs, ICFs, IFLs, zAAPs, zIIPs, or memory. It is possible to request a customer order approval to conform to customer operation policies.
Page 232 Figure 8-4 illustrates the CIU ordering process on the IBM Resource Link Web site. ibm.com/servers/resourcelink Customer Internet CIU or On/Off CoD order Optional customer secondary order approval Remote Support Facility Figure 8-4 CIU ordering example The following is a sample list of the windows a customer must follow on Resource Link to initiate an order: 1.
Page 233 Figure 8-5 shows the CIU activation process. IBM Resource Link communicates with the Remote Support Facility to stage the CIU order and prepare it for download. The customer is automatically notified when the order is ready for download. ibm.com/servers/resourcelink Customer...
Page 234 Activation Once an order is placed, Resource Link dynamically enables the appropriate LIC-CC records and make them available to download. IBM System z9 Enterprise Class Technical Guide...
Page 235 When the order is available for download, the customer is sent a note containing an activation number for the order. The order can then be downloaded. To download the order, use any of the Hardware Management Consoles (HMC) attached to the system to be upgraded and perform a Single Object Operation into the Support Element (SE).
Page 236 CI for CPs to help meet customers’ peak workload requirements. On/Off CoD uses the Customer Initiated Upgrade (CIU) process to request the upgrade through the Web, using IBM Resource Link. On/Off CoD requires the CIU Enablement feature (FC 9898) and the On/Off CoD Enablement feature (FC 9896) installed.
Page 237 It is recommended that you use the Large System Performance Reference information to evaluate the capacity requirements according to your workload type. LSPR data for current IBM processors is available at this URL: http://www.ibm.com/servers/eserver/zseries/lspr/ Table 8-2 Valid On/Off CoD upgrades for granular capacity models...
Page 238 Subsequently, the customer may concurrently install temporary capacity up to the limits in On/Off CoD and use it for an indeterminate time. Monitoring will occur through the server call home facility and an invoice IBM System z9 Enterprise Class Technical Guide...
Page 239 will be generated if the capacity has been enabled during the calendar month. The customer will continue to be billed for use of temporary capacity until the server is returned to the original configuration. If the On/Off CoD support is no longer needed, the enablement code will need to be removed.
Page 240 Depending on how the additional capacity was added to the logical partitions, customers may be required to perform tasks at the logical partition level in order to remove the temporary capacity; for example, configure offline CPs that had been added to the partition. IBM System z9 Enterprise Class Technical Guide...
Page 241 On/Off CoD orders can be staged in Resource link so that multiple orders are available. A order can only be downloaded and activated one time. If a different On/Off CoD order is required, it can be downloaded and activated without having to restore the system to its original purchased capacity.
Page 242 CPs must be the same. The number of CBU CPs must be equal to, or greater, than the number of CPs in the base configuration, and all the CPs in the CBU configuration must have the same capacity setting. IBM System z9 Enterprise Class Technical Guide...
Page 243 For example, if the base configuration is a 2-core CP4, providing a CBU configuration of a four way of the same capacity setting requires two CBU feature codes. If the required CBU capacity changes the capacity setting of the CPs, and you go from model capacity identifier 402 to a CBU configuration of a 4-core 504, would require four CBU feature codes with a capacity setting of 5xx.
Page 244 CBU resources. In situations where the RSF cannot be used, CBU can be activated through a password window. In this case, a request by telephone to the IBM Support Center will retrieve the password. The CBU activation cannot be done when an On/Off CoD upgrade is already activated.
Page 245 Capacity BackUp example Figure 8-10 shows an example of a z9 EC Model S18 capacity identifier 704 to a z9 EC Model S18 capacity identifier 712 Capacity BackUp operation. 2094-S18 2094-S18 Target Model: Model 712 Model 712 2094-S18 2094-S18 Model 704 Model 712 Eight CBUs + 8 CPs...
Page 246 Unused PUs or memory not LIC-CC enabled Inactive resources that are LIC-CC enabled, that is, memory that is not being used by any activate logical partitions Memory purchased with the flexible memory option Additional books as discussed previously IBM System z9 Enterprise Class Technical Guide...
Page 247 PUs (CPs, IFL, ICF, zAAP, and zIIP). • Amount of memory, both physically installed and LIC-CC enabled. • Work with your IBM service Personnel to determine the memory card size in each book. The memory card sizes and the number of cards installed for each installed book can be viewed from the SE under the CPC Configuration task list, using the View Hardware Configuration Option.
Page 248 For maximum availability, you may consider purchasing a z9 EC using the guidance in 8.3.1, “Planning consideration” on page 232. For additional information about Reconfigurable Storage Unit, see System z9 Processor Resource/Systems Manager (PR/SM) Planning Guide , SB10-7041. IBM System z9 Enterprise Class Technical Guide...
Page 249 Memory availability Memory resource availability for reallocation/deactivation depends on: Physically installed storage Image profile storage allocations Amount of LIC-CCed memory Flexible memory option There are 23 maximum availability memory features available for models S18, S28, S38, and S54, delivering 32 GB to 384 GB in 16 GB increments (FC 2802 to 2824). MBA to STI-MP connectivity requirements The optimum situation is to maintain maximum I/O connectivity during book removal.
Page 250 If not enough resources are available, the conflicts will be identified so that you can take action to free up resources. I/O connections associated with the removal of the targeted book will be analyzed for any single path I/O connectivity. IBM System z9 Enterprise Class Technical Guide...
Page 251 There are three states that can result from the prepare option: The system is ready to perform the Enhanced Book Availability for the targeted book with the original configuration. The system is not ready to perform the Enhanced Book Availability due to conditions noted from the preparation step.
Page 252 6144 MB. Below that is the breakdown of the in-use memory by partition name. Once the required amount of memory has been made available, rerun the preparation to verify. Figure 8-14 Prepare for EBA - memory conditions IBM System z9 Enterprise Class Technical Guide...
Page 253 The other tab is for Single I/O (see Figure 8-15). The preparation identified one single I/O path associated with the removal of Book 0. The path would need to be placed offline to perform the book removal. Once the condition has been addressed, rerun the prepare to ensure all the required conditions have been met.
Page 254 4. When ready for the book removal, free up the resources indicated in the prepare step. 5. Re-run the step in Figure 8-11 on page 236 to ensure the required conditions are all satisfied. IBM System z9 Enterprise Class Technical Guide...
Page 255 6. When the completed successfully message is received (see Figure 8-18), the system is now ready for the removal of the book. The preparation step can be run multiple times to ensure the conditions have been met. It does not reallocate any resources; all it does is produce a results report. The resources will not be reallocated until the perform part of the book removal.
Page 256 A limited number of specific crossover bundles will be defined by IBM for a driver. Disruptive driver upgrades are permitted at any time. Concurrent crossover from driver level N to driver level N+1, to driver level N+2 must be done serially;...
Page 257 The current plan is to try to have sync points generated at regular intervals, as shown in Figure 8-19. Sync Point Sync Point Driver N Driver N Driver N Driver N+1 Driver N+1 Driver N+1 Driver N+2 Driver N+2 Driver N+2 Figure 8-19 Driver levels and relationship to sync points The EDM function does not completely eliminate the need for planned outages for driver level upgrades.
Page 258 The Coupling Facility Control Code (CFCC) can also configure more processors online to Coupling Facility logical partitions using the CFCC image operations window. IBM System z9 Enterprise Class Technical Guide...
Page 259 Memory Memory can be concurrently added to a z9 EC up to the physical installed memory limit. Additional books can also be installed concurrently, allowing further memory upgrades by LIC-CC enabling memory capacity on the new books. Using the previously defined reserved memory, z/OS operating system images can dynamically configure more memory online, allowing nondisruptive memory upgrades.
Page 260 ‘*’ for future renaming, as described in the previous item, can help to define additional CSSs for future use. Spanned channels can help spread logical partitions across CSSs while maintaining physical channels sharing for some channel types. IBM System z9 Enterprise Class Technical Guide...
Page 261 For detailed planning and configuration information about Crypto Express2 features, see the Redbooks publication IBM System z9 109 Configuration Setup, SG24-7203. Considerations when installing additional books During a z9 EC upgrade, additional books can be installed concurrently. Depending on the number of additional books in the upgrade and the customer’s I/O configuration, a STI...
Page 262 IBM System z9 Enterprise Class Technical Guide...
Page 263 EC server. For more comprehensive physical planning information, refer to IBM System z9 Installation Manual for Physical Planning , GC28-6844.
Page 264 I/O cages, and quantity of each type of I/O feature card. The tool will output an estimate of the power requirements needed for this system: http://www.ibm.com/servers/resourcelink It is designed to help in power and cooling planning for new or currently installed IBM System z9 servers. IBM System z9 Enterprise Class Technical Guide...
Page 265 9.1.2 Internal Battery Feature The optional Internal Battery Feature (IBF) provides sustained system operations for a relatively short period of time, allowing for orderly shutdown. In addition, an external UPS system can be connected to the System z9, allowing for longer periods of sustained operation.
Page 266 Frame Z with covers 767 (30.2) 1476 (58.1) 1941 (76.4) Note: The total machine room area required is 2.49 square meters (26.78 square feet). With service clearance, 5.45 square meters (58.69 square feet) are needed. IBM System z9 Enterprise Class Technical Guide...
Page 267 2006. However, the RoHS Directive contains several exemptions that will allow continued use of these materials in some applications after the phaseout date. IBM System z9 Enterprise Class will fully comply with the EU RoHS requirements by July 1, 2006.
Page 268 IBM System z9 Enterprise Class Technical Guide...
Page 269 HMC, you will need to purchase a new console to control these devices or keep the prior generation HMC for the single task of managing these devices. © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 270 A recurring scheduled operation that occurs at a specified frequency Setting up a schedule to dial out to the time service automatically is done using the Scheduled Operations option at the Support Element using Single Object Operations. IBM System z9 Enterprise Class Technical Guide...
Page 271 At the scheduled time, the SE requests the HMC to dial out to the time service; the HMC sends the information obtained from the time service to the SE, which in turn sends the update to STP. STP makes gradual adjustments by steering CST to the time obtained from the external time source.
Page 272 Therefore, any existing customer-installed firewall between the remote HMC and its managed objects must permit communications between the HMC and SE. The remote HMC also requires connectivity to IBM or another HMC with connectivity to IBM for service and support.
Page 273 The automated interfaces are used by various automation products, including Tivoli® System Automation for z/OS - Processor Operations. If your automation product is using the IBM-supplied APIs to communicate to the HMC, updates for these are found on the IBM resource link Web site: http://www.ibm.com/servers/resourcelink...
Page 274 If you require having uniform HMC information across your HMCs for data, such as data mirroring passwords, password rules, and user settings, IBM has implemented a function for the HMC setup. A user can customize an HMC, and then associate other HMCs in the configuration to the same customized data (see Figure A-2).
Page 275 accessing the HMC. Only the local user interface is allowed to make use of non-encrypted access, since it is inherently a secure environment (closed platform). Since SSL encryption is required for all remote access to the HMC, a certificate is required to provide the keys used for this encryption.
Page 276 IBM System z9 Enterprise Class Technical Guide...
Page 277 The CHPID Mapping Tool is intended for customer use. Since most of the functions and input require a high degree of knowledge regarding the environment, it is extremely difficult for an IBM representative to have the customer’s detailed level of knowledge for the entire configuration.
Page 278 Users should ask their IBM representatives to verify that they have the latest CCN number associated with the machine order. 4. Java Runtime Environment (JRE™) The stand-alone CMT requires a minimum runtime environment of Java 1.3.0.
Page 279 z9 EC CHPID mapping The z9 EC does not have default CHPIDs assigned to ports as part of the initial configuration process. It is the customer’s responsibility to perform these assignments by using HCD/IOCP definitions and, optionally, the CHPID Mapping Tool (CMT). One of the results of using CMT is an IOCP source that will map the defined CHPIDs to the corresponding PCHIDs of the server.
Page 280 Both manual mapping and availability mapping require the following input file to the CMT: Hardware Configuration report. This can be either: – Supplied by your IBM Representative (The output from eConfig and has the extension .CFR.) – Downloaded from Resource Link (This also has the extension .CFR.)
Page 281 The following section explains how the system preserves the config files on an upgrade from a z990 to a z9 EC. However, it is ultimately your responsibility to have a record or backup of the customization data stored in config files. You should always make a backup record: For OSA-Express2 channels, record all user-assigned MAC addresses using the Display or alter MAC address function in Card specific advanced facilities or user-specified OSA Address Tables (OAT).
Page 282 ‘TO PCHID’ column during of the Migrate Channel procedure. Figure B-2 CMT CHPID report Windows® are available in the CMT tool that show which CHPIDs contain configuration files (see Figure B-3). Figure B-3 CMT Configuration file screen IBM System z9 Enterprise Class Technical Guide...
Page 283 Information about CHPIDs that may contain configuration files is contained in the source column field in the CMT tool (Figure B-4). Source Column indicating CHIPDS that may contain Config Files Figure B-4 CMT Source column information Manual mapping The manual method can be used to define the relationships between each logical CHPID and physical ports on the server.
Page 284 S = Greater than half the assigned channels use the same STI. M = All the assigned channels are supported by the same MBA. B = Greater than half the assigned channels are connected to the same book. IBM System z9 Enterprise Class Technical Guide...
Page 285 The tailored reports. All reports should be saved for reference. The Port Report sorted by location should be supplied to your IBM Service Representative for reference. An IOCP with PCHIDs mapped to CHPIDs by Channel Subsystem. This IOCP source can then be migrated back into HCD and a production IODF can be built.
Page 286 IBM System z9 Enterprise Class Technical Guide...
Page 287 Fiber cabling services Appendix C. This appendix describes the IBM Networking Services fiber cabling services options offered by IBM Global Services to customers. The following topics are covered: Fiber cabling services options – Option 1: Fiber-optic jumper cabling package – Option 2: Fiber-optic jumper cable migration and reuse –...
Page 288 IBM Networking Services fiber cabling services IBM Global Services offers customers the option of engaging IBM to help plan and implement their enterprise connectivity with a suite of services offerings. The IBM Networking Services fiber cabling services, being offered by IBM Global Services, provides five options under System z Fiber Cabling Services (options 1 to 3) and Enterprise Fiber Cabling Services (options 4 and 5).
Page 289 The customer provides a bill of materials (BOM) to IBM of the required fiber-optic jumper cables, conversion kits, and MCP cables. The customer provides IBM with the server plugging information, based on this analysis.
Page 290 IBM does the fiber-optic trunking commodities ordering, labeling, and installation based on the customer’s channel configurations. c. IBM documents the new fiber-optic trunking and server channel connections across the Enterprise. This option also supports the Fiber Quick Connect (FQC) feature, which provides factory installed fiber-optic harnesses for ESCON channels in System z servers.
Page 291 Fiber Quick Connect is an option in the eConfig configuration tool when ordering a new build or upgrade of a System z server The FQC features are for factory installation of IBM Fiber Transport System (FTS) fiber-optic harnesses for connection to all ESCON channels in I/O cages of the System z servers. FTS fiber-optic harnesses enable connection to FTS direct-attach fiber-optic trunk cables.
Page 292 FQC, when coupled with the Fiber Transport System (FTS) products from IBM Global Services, delivers a solution to reduce the amount of time required for on-site installation and setup of cabling, to minimize disruptions, and to isolate the activity from the active system as much as possible.
Page 293 These benefits and potential cost savings need to be assessed for each data center and its particular environment. References For further information about the IBM Networking Services fiber cabling services offered by IBM Global Services, and related topics, see: The IBM Resource Link Web site: http://www.ibm.com/servers/resourcelink...
Page 294 IBM System z9 Enterprise Class Technical Guide...
Page 295 (I/O channels) to a computer. (2) Pertaining to devices attached to a CBA. Concurrent Book Add. controlling unit by cables rather than by telecommunication lines. CBP. Integrated cluster bus Coupling Facility peer channel. © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 296 CNC. Mnemonic for an ESCON channel used to communicate to an ESCON-capable device. Coupling Facility. A special logical partition that provides high-speed caching, list processing, and locking functions in a sysplex. IBM System z9 Enterprise Class Technical Guide...
Page 297 (2) The device number that you associate with a subchannel that uniquely identifies an I/O device. CVC. Mnemonic for an ESCON channel attached to an IBM 9034 convertor. The 9034 converts from ESCON DH. Diffie Hellman. CVC signals to parallel channel interface (OEMI) communication operating in block multiplex mode (Bus direct access storage device (DASD).
Page 298 (2) A set of IBM products and distribution panel. (1) In an ESCON or FICON services that provide a dynamically connected environment, a panel that provides a central location for environment within an enterprise.
Page 299 FC-SB-2/3), (2) FCV mode (FICON conversion mode to a these features or facilities are standard with no feature IBM 9032-5), or (3) FCP mode (FICON channel operating code and sometimes they must be specifically ordered, in “open mode”, which is FC-FCP).
Page 300 (IOCDS). The data local area network LAN. See set in the System z processor (in the support element) that contains an I/O configuration definition built by the input/output configuration program (IOCP). IBM System z9 Enterprise Class Technical Guide...
Page 301 Licensed Internal Code (LIC). Software provided for use operating system to create the virtual machine on specific IBM machines and licensed to customers environment. under the terms of the IBM Customer Agreement.
Page 302 Multiple Image Facility (MIF). In the z/Architecture, a fiber function that allows logical partitions to share a channel path by providing each logical partition with its own set of subchannels for accessing a common device. IBM System z9 Enterprise Class Technical Guide...
Page 303 (1) An access point for data entry or exit. (2) A (OEMI). A reference to an IBM guideline for a computer receptacle on a device to which a cable for another device duplex receptacle peripheral interface. More specifically, refers to IBM is attached.
Page 304 (PTF). A temporary solution or SC Connector. An optical fibre cable duplex connector bypass of a problem diagnosed by IBM in a current that terminates both jumper cable fibres into one unaltered release of a program. housing and provides physical keying for attachment to an LC duplex receptacle.
Page 305 CST. subchannel. (1) A logical function of a Channel Sysplex Timer. An IBM table-top unit that synchronizes Subsystem associated with the management of a single the time-of-day (TOD) clocks in as many as 16 processors device.
Page 306 (IML) process Transaction Processing Facility. Transaction Processing Facility is a specialized high availability operating system designed to provide quick response times to very high volumes of messages from large networks of terminals and workstations. IBM System z9 Enterprise Class Technical Guide...
Page 307 VLAN. Virtual Local Area Network. VPD. Vital Product Data. WLM. Workload Manager. z/Architecture. An IBM architecture for mainframe computers and peripherals. Processors that follow this architecture include the System z9 and zSeries servers. zAAP. System z9 and zSeries Application Assist Application Assist Processor (AAP) Processor.
Page 308 IBM System z9 Enterprise Class Technical Guide...
Page 309 Redbooks book. IBM Redbooks For information about ordering these publications, see “How to get IBM Redbooks” on page 296. Note that some of the documents referenced here may be available in softcopy only.
Page 310 You can search for, view, or download Redbooks, Redpapers, Hints and Tips, draft publications and Additional materials, as well as order hardcopy Redbooks or CD-ROMs, at this Web site: ibm.com/redbooks Help from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services IBM System z9 Enterprise Class Technical Guide...
Page 311 211 assigned 73 processor identification 180 CP pool 55–57 CCN Number 264 CP4 feature 58 Central Processor (CP) 3, 46, 51, 54, 57, 130, 149–150, CP5 feature 58 © Copyright IBM Corp. 2005, 2006, 2007. All rights reserved.
Page 312 DCA 29, 92 DCA-CC 92 DCM 200 FCP 10, 113 Decimal Floating Point 53 concurrent patch 114 Dense Wave Division Multiplexing (DWDM) 190 feature code 97, 206, 216 DES 150 CBU 228 DFSMS striping 147 IBM System z9 Enterprise Class Technical Guide...
Page 313 LX 103–104, 108, 110–111 I/O operation 65, 130, 144–145, 202 SX 104, 109–111 I/O request 201–202 FICON Express2 4, 7, 10–11, 41, 47, 95, 97, 101, 167, IBM Fiber Transport System 277 169–171 IC 190 LX 108, 110 IC3 17 FICON Express4 47, 95 ICB-3 link 16, 41, 94–96, 126, 190...
Page 314 LIC-based upgrades 207 fan out card 8, 21, 30, 37–38, 41, 93, 99 LIC-CC 8, 207 fan out plugging sequence 93 memory 207 MCM 2, 4, 7, 29, 42–43 processors 207 chip layout 43 IBM System z9 Enterprise Class Technical Guide...
Page 315 MCP 116–118 active CP 222 Media Manager 148 active ICF 222 memory 7 active IFL 222 allocation 69 active zAAP 222 card 8, 29, 32–34, 46, 207, 210, 213–214 contractual terms 224 size 28, 32, 44, 68 enablement feature 222 upgrade 32 granular capacity 78 Memory Bus Adapter (MBA) 8, 37, 92, 94, 206...
Page 316 SCSI disk 169, 175 feature code 4 SD chip 37, 43, 46 Maximum number 4 Secure Hash Algorithm (SHA-1) 12 pool 55, 173 Secure Sockets Layer (SSL) 13, 46, 51, 150–151, 154, separate management 8 IBM System z9 Enterprise Class Technical Guide...
Page 317 157, 160, 260 Sysplex Timer 126, 184–187, 255 Select Application License Charges 179 System Assist Processor (SAP) 3–4, 31, 65 Self Timed Interconnect (STI) 8, 37, 41, 46, 48, 93–94, system image 69, 79, 84, 103, 126, 173, 175, 184–185, 96, 100, 200, 211, 244 194, 197, 244 granularity 8...
Page 318 BC 48, 159, 185 z990 4, 48, 56, 72, 98, 250 zAAP 19, 31–32, 44, 54–56, 60, 71, 222, 228–229, 244 zAAP pool 55, 60 zIIP 19, 31–32, 44, 54–56, 63, 71 zIIP pool 55–56, 64 IBM System z9 Enterprise Class Technical Guide...
Page 322 IBM System z9 Enterprise Class Technical Guide Structure and design This IBM Redbooks publication discusses the IBM System z9 INTERNATIONAL Enterprise Class (z9 EC), which offers a continuation of the IBM a total systems TECHNICAL scalable mainframe servers. Based on z/Architecture, the System approach...

This manual is also suitable for:

Z9 ec s08 Z9 ec s18 Z9 ec s28 Z9 ec s38 Z9 ec s54

IBM z9 Enterprise Series Technical Manual

Quick Links

Need help?

Questions and answers

Related Manuals for IBM z9 Enterprise Series

Summary of Contents for IBM z9 Enterprise Series

This manual is also suitable for: