1. Manuals
  2. Brands
  3. IBM Manuals
  4. Server
  5. S822LC
  6. Technical overview and introduction

IBM Power Systems S822LC Technical Overview And Introduction

Hide thumbs Also See for Power Systems S822LC:
Table of Contents

Advertisement

Quick Links

Download this manual See also: Manual
Front cover
IBM Power Systems S822LC
for High Performance Computing
Technical Overview and Introduction
Alexandre Caldeira
Volker Haug
Scott Vetter
Redpaper

Advertisement

Table of Contents
loading

Related Manuals for IBM Power Systems S822LC

Summary of Contents for IBM Power Systems S822LC

  • Page 1 Front cover IBM Power Systems S822LC for High Performance Computing Technical Overview and Introduction Alexandre Caldeira Volker Haug Scott Vetter Redpaper...
  • Page 3 International Technical Support Organization IBM Power Systems S822LC for High Performance Computing September 2016 REDP-5405-00...
  • Page 4 Note: Before using this information and the product it supports, read the information in “Notices” on page v. First Edition (September 2016) This edition applies to the IBM Power Systems S822LC model 8335-GTB. © Copyright International Business Machines Corporation 2016. All rights reserved.

  • Page 5: Table Of Contents

    Stay connected to IBM Redbooks ........
  • Page 6 2.3.1 Introduction ............33 2.3.2 IBM terminology versus x86 terminology ....... 35 2.3.3 Error handling .

  • Page 7: Notices

    IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead.

  • Page 8: Trademarks

    IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml...

  • Page 9: Ibm Redbooks Promotions

    Redbooks publication, featuring your business or solution with a link to your web site. Qualified IBM Business Partners may place a full page promotion in the most popular Redbooks publications. Imagine the power of being seen by users who download ibm.com/Redbooks...
  • Page 10 THIS PAGE INTENTIONALLY LEFT BLANK...

  • Page 11: Preface

    NVIDIA NVLink technology for exceptional processor to accelerator intercommunication Four SXM2 form factor connectors for the NVIDIA Tesla P100 GPU This publication is for professionals who want to acquire a better understanding of IBM Power Systems products. The intended audience includes the following roles:...

  • Page 12: Authors

    Estadual Paulista (UNESP) and an MBA in Marketing. His major areas of focus are competition, sales, marketing and technical sales support. Alexandre has more than 16 years of experience working on IBM Systems Solutions and has worked also as an IBM Business Partner on Power Systems hardware, AIX, and IBM PowerVM® virtualization products.

  • Page 13: Stay Connected To Ibm Redbooks

    Find us on Facebook: http://www.facebook.com/IBMRedbooks Follow us on Twitter: http://twitter.com/ibmredbooks Look for us on LinkedIn: http://www.linkedin.com/groups?home=&gid=2130806 Explore new Redbooks publications, residencies, and workshops with the IBM Redbooks weekly newsletter: https://www.redbooks.ibm.com/Redbooks.nsf/subscribe?OpenForm Stay current on recent Redbooks publications with RSS Feeds: http://www.redbooks.ibm.com/rss.html Preface...
  • Page 14 IBM Power Systems S822LC for High Performance Computing...

  • Page 15: Chapter 1. Architecture And Technical Description

    Architecture and technical Chapter 1. description The IBM Power System S822LC for High Performance Computing (8335-GTB) server, the first Power System offering with NVIDIA NVLink Technology, removes GPU computing bottlenecks by employing the high-bandwith and low-latency NVLink interface from CPU to GPU and GPU to GPU.

  • Page 16: Server Features

    The servers support four SXM2 form factor connectors for NVIDIA Tesla P100 GPU (#EC4C, #EC4D, or #EC4F) only. Optional water cooling is available. This summary describes the standard features of the Power S822LC model 8355-GTA server: 19” rack-mount (2U) chassis IBM Power Systems S822LC for High Performance Computing...

  • Page 17: Minimum Features

    Two POWER8 processor modules: – 8-core 3.3259 GHz processor module – 10-core 2.860 GHz processor module Up to 1024 GB of 1333 MHz DDR4 ECC memory Two SFF bays for two HDDs or two SSDs that supports: – Two 1 TB 7200 RPM NL SATA disk drives (#ELD0) –...

  • Page 18: The Nvidia Tesla P100

    Unified Memory, Compute Preemption, and New AI Algorithms Significantly improved programming model and advanced AI software optimized for the Pascal architecture; 16nm FinFET Enables more features, higher performance, and improved power efficiency IBM Power Systems S822LC for High Performance Computing...

  • Page 19: Operating System Support

    Power Systems website: http://www.ibm.com/systems/power/software/linux/index.html The Linux operating system is an open source, cross-platform OS. It is supported on every Power Systems server IBM sells. Linux on Power Systems is the only Linux infrastructure that offers both scale-out and scale-up choices. 1.3.1 Ubuntu Ubuntu Server 16.04, at the time of writing, is the supported OS for the S822LC.

  • Page 20: Operating Environment

    Power consumption and heat load vary greatly by server configuration and usage. Use the IBM Systems Energy Estimator to obtain a heat output estimate that is based on a specific configuration, which is available at the following website: http://www-912.ibm.com/see/EnergyEstimator...

  • Page 21: System Architecture

    Table 1-2 Physical dimensions for the Power S822LC servers Dimension Power S822LC (8335-GTB) server Width 441.5 mm (17.4 in.) Depth 822 mm (32.4 in.) Height 86 mm (3.4 in.) Weight (maximum configuration) 30 kg (65 lbs.) 1.6 System architecture This section describes the overall system architecture for the Power S822LC computing servers.
  • Page 22 28.8 GBps POWER8 POWER8 SCM1 SCM0 12.8 GBps SFF-4 SFF-4 HDD/ HDD/ SATA Controller 1 Gbps Ethernet USB 3.0 Front USB 3.0 Management Rear Figure 1-4 Power S822LC server logical system diagram IBM Power Systems S822LC for High Performance Computing...

  • Page 23: The Power8 Processor

    NVLink interface. The resulting chip grows slightly from 649 mm to 659 mm 1.7.1 POWER8 processor overview The POWER8 processor is manufactured by using the IBM 22 nm Silicon-On-Insulator (SOI) technology. Each chip is 649 mm and contains 4.2 billion transistors. As shown in Figure 1-5,...
  • Page 24 512 KB/6 MB Maximum On-chip L3 cache core/chip 8 MB/96 MB Maximum L4 cache per chip 128 MB Maximum memory controllers SMP design-point 16 sockets with POWER8 processors Compatibility Specific to the 8335-GTB IBM Power Systems S822LC for High Performance Computing...

  • Page 25: Power8 Processor Core

    Figure 1-6 Areas modified on the POWER8 processor core 1.7.2 POWER8 processor core The POWER8 processor core is a 64-bit implementation of the IBM Power Instruction Set Architecture (ISA) Version 2.07 and has the following features: Multi-threaded design, which is capable of up to eight-way simultaneous multithreading...

  • Page 26: Simultaneous Multithreading

    Table 1-4 SMT levels that are supported by a Power S822LC server Cores per system SMT mode Hardware threads per system Single Thread (ST) SMT2 SMT4 SMT8 Single Thread (ST) SMT2 SMT4 IBM Power Systems S822LC for High Performance Computing...

  • Page 27: Memory Access

    SMT8 The architecture of the POWER8 processor, with its larger caches, larger cache bandwidth, and faster memory, allows threads to have faster access to memory resources, which translates into a more efficient usage of threads. Therefore, POWER8 allows more threads per core to run concurrently, increasing the total throughput of the processor and of the system.

  • Page 28: L4 Cache And Memory Buffer

    The performance of eDRAM when implemented on-chip is similar to conventional SRAM but requires far less physical space. IBM on-chip eDRAM uses only a third of the components that conventional SRAM uses, which has a minimum of six transistors to implement a 1-bit memory cell.

  • Page 29: Hardware Transactional Memory

    Figure 1-9 shows a picture of the memory buffer, where you can see the 16 MB L4 cache and processor links and memory interfaces. Figure 1-9 Memory buffer chip 1.7.7 Hardware transactional memory Transactional memory is an alternative to lock-based synchronization. It attempts to simplify parallel programming by grouping read and write operations and running them as a single operation.
  • Page 30 POWER8 processor for general applications and the custom acceleration of specific functions by using a hardware accelerator, with a simplified programming model and efficient communication with the processor and memory resources. IBM Power Systems S822LC for High Performance Computing...

  • Page 31: Nvlink

    For a list of supported CAPI adapters, see 1.13.5, “CAPI enabled Infiniband adapters” on page 27. 1.7.9 NVLink NVLink is NVIDIA’s new high-speed interconnect technology for GPU-accelerated computing. Supported on SXM2 based Tesla P100 accelerator boards, NVLink significantly increases performance for both GPU-to-GPU communications, and for GPU access to system memory. Today, multiple GPUs are common in workstations as well as the nodes of HPC computing clusters and deep learning training systems.

  • Page 32: Memory Subsystem

    While NVLink primarily focuses on connecting multiple NVIDIA Tesla P100s together it can also connect Tesla P100 GPUs with IBM Power CPUs with NVLink support. Figure 1-11 on page 18 highlights an example of a four-GPU system with dual NVLink-capable CPUs connected with NVLink.
  • Page 33 Buffer Cache By adopting this architecture, several decisions and processes regarding memory optimizations are run outside the processor, saving bandwidth and allowing for faster processor to memory communications. It also allows for more robust reliability, availability, and serviceability (RAS). For more information about RAS, see 2.3, “Reliability, availability, and serviceability”...

  • Page 34: Memory Placement Rules

    Table 1-6 Power S822LC single-core bandwidth estimates Single core 8335-GTB 2.860 GHz 3.259 GHz L1 (data) cache 137.28 GBps 156.43 GBps L2 cache 137.28 GBps 156.43 GBps L3 cache 183.04 GBps 208.57 GBps IBM Power Systems S822LC for High Performance Computing...
  • Page 35 The bandwidth figures for the caches are calculated as follows: L1 cache: In one clock cycle, two 16-byte load operations and one 16-byte store operation can be accomplished. The value varies depending on the clock of the core, and the formulas are as follows: –...

  • Page 36: System Bus

    For more information about which slots are connected directly to the processor and which ones are attached to PCIe Gen3 switches (referred as PLX), see 1.7, “The POWER8 processor” on page 9. IBM Power Systems S822LC for High Performance Computing...
  • Page 37 A diagram showing the Power S822LC server buses and logical architecture is shown in Figure 1-13. POWER8 POWER8 SCM1 SCM0 12.8 GBps SFF-4 SFF-4 HDD/ HDD/ SATA Controller 1 Gbps Ethernet USB 3.0 Front USB 3.0 Management Rear Figure 1-13 Power S822LC server buses and logical architecture Each processor has 32 PCIs lanes split into three channels: two channels are PCIe Gen3 x8 and one channel is PCIe Gen 3 x16.

  • Page 38: Internal I/O Subsystem

    Slot 1 PCIe Gen3 x16 Half height, 75 W half length Slot 2 PCIe Gen3 x8 Half height, 50 W half length Slot 3 PCIe Gen3 x16 Full height, 75 W half length IBM Power Systems S822LC for High Performance Computing...

  • Page 39: System Ports

    Linux. Any other application that uses serial ports requires a serial port adapter to be installed in a PCI slot. The integrated system ports do not support IBM PowerHA® configurations. The VGA port does not support cable lengths that exceed 3 meters.

  • Page 40: Lan Adapters

    If you are installing a new feature, ensure that you have the software that is required to support the new feature and determine whether there are any existing update prerequisites to install. To obtain this information, use the IBM prerequisite website: https://www-912.ibm.com/e_dir/eServerPreReq.nsf The following sections describe the supported adapters and provide tables of orderable feature code numbers.

  • Page 41: Fibre Channel Adapters

    Table 1-14 Summary of features for the integrated SATA disk controller Option Integrated SATA disk controller Supported RAID types JBOD Disk bays Two SFF Gen4 (HDDs/SDDs) SATA controllers Single IBM Easy Tier® capable controllers External SAS ports Chapter 1. Architecture and technical description...

  • Page 42: Disk And Media Features

    The servers support the attachment of up to two SATA storage devices that are listed on Table 1-15. Table 1-15 Supported storage devices Feature Description code support ELD0 1 TB 7.2k RPM SATA SFF-4 disk drive Linux IBM Power Systems S822LC for High Performance Computing...

  • Page 43: External I/O Subsystems

    IBM Network Attached Storage (NAS) products provide a wide-range of network attachment capabilities to a broad range of host and client systems, such as IBM Scale Out Network Attached Storage and the IBM System Storage N series. For more information about the hardware and software, see the following website: http://www.ibm.com/systems/storage/network...

  • Page 44: Java

    The storage system can manage a broad scope of storage workloads that exist in today’s complex data center, doing it effectively and efficiently. Additionally, the IBM System Storage DS8000® storage system includes a range of features that automate performance optimization and application quality of service, and also provide the highest levels of reliability and system uptime.

  • Page 45: Chapter 2. Management, Reliability, Availability, And Serviceability

    Chapter 2. Availability, and Serviceability The IBM Power Systems S822LC (8335-GTB) server uses the Open Power Abstraction Layer (OPAL) baremetal firmware for a non-virtualized configuration. This chapter attempts to identify and clarify the tools that are available for managing the S822LC server.

  • Page 46: Main Management Components Overview

    IBM POWER8 processor and memory buffer modules. OPAL is the system firmware in the stack of POWER8 processor-based Linux on Power Systems servers.

  • Page 47: Intelligent Platform Management Interface

    2.3.1 Introduction The IBM Power Systems S822LC servers are bringing POWER8 processor and memory RAS functions into a highly competitive cloud data center with open source Linux technology as an operating system and virtualization.
  • Page 48 Over-Current conditions. They report to a “power good” tree that is monitored by the service processor. I/O subsystem The PCIe controllers are integrated into the POWER8 processor. All the PCIe slots are directly driven by the PCIe controllers. IBM Power Systems S822LC for High Performance Computing...

  • Page 49: Ibm Terminology Versus X86 Terminology

    2.3.2 IBM terminology versus x86 terminology The different components and descriptions in the boot process have similar functions, but have different terms for POWER8 processor-based and x86-based scale-out servers.

  • Page 50: Serviceability

    Customer Feature Install (CFI) Customer Repairable Units (CRU) Warranty service upgrades are offered for an onsite repair (OSR) by an IBM System Services Representative (SSR), or an authorized warranty service provider. The system is designed with a 5 year MTBF. If something needs to be serviced or relocated, Table 2-2 on page 37 lists whether an item is able to be concurrently repaired, and if it requires an IBM SSR to repair.
  • Page 51 Although not all errors are a guaranteed threat to system availability, those errors that go undetected can cause problems because the system has no opportunity to evaluate and act if necessary. POWER processor-based systems employ IBM z™ Systems server-inspired error detection mechanisms, extending from processor cores and memory to power supplies and hard disk drives (HDDs).
  • Page 52 Code to isolate and recover from problems. Potential problems are reported to an operating system device driver, which logs the error. General problem determination Accessing the Advanced System Management GUI interface provides a general overview of sensor information and possible errors. IBM Power Systems S822LC for High Performance Computing...
  • Page 53 Using an event sensor display as a primary interface for problem determination This function has the following aspects: Covers 90% of typical failures Does not handle transient failure scenarios Using SEL logs or operating system syslog records for remainder This function has the following aspects: Sensors can be enabled/disabled by a client.

  • Page 54: Locating And Servicing

    You can find them at the following website: http://www.ibm.com/support/customercare/sas/f/lopdiags/home.html The tools are automatically loaded if IBM manufacturing installs the Linux image or IBM Installation Toolkit. PowerPack is the preferred way to install required service packages from the website. The Linux call home feature is also supported in a stand-alone system configuration to report serviceable events.
  • Page 55 Packaging for service The following service enhancements are included in the physical packaging of the systems to facilitate service: Color coding (touch points) Terracotta-colored touch points indicate that a component (FRU or CRU) can be concurrently maintained. Blue-colored touch points delineate components that may not be concurrently maintained (they might require that the system is turned off for removal or repair).

  • Page 56: Manageability

    IBM Knowledge Center IBM Knowledge Center provides you with a single place where you can access product documentation for IBM systems hardware, operating systems, and server software. The purpose of IBM Knowledge Center, in addition to providing client-related product information, is to provide softcopy information to diagnose and fix any problems that might occur with the system.
  • Page 57 A is an update to an existing release level. IBM is increasing its clients’ opportunity to stay on a given release level for longer periods. Clients that want maximum stability can defer until there is a compelling reason to upgrade,...
  • Page 58 When updating firmware, a service professional can find a window that shows which components will be overwritten or preserved, as shown in Figure 2-3. For this example, the network settings will be preserved. Figure 2-3 Firmware Update window IBM Power Systems S822LC for High Performance Computing...
  • Page 59 4. The next window prompts whether you want to continue to the update mode, as shown in Figure 2-4. Until the firmware update is completed, no other activities can be performed in the Advanced System Management Interface. If you want to proceed, click OK. Figure 2-4 Confirm firmware update mode 5.
  • Page 60 Figure 2-6 Start the firmware upgrade 7. After the firmware update is complete, the system restarts. After the restart, you can verify that the systems firmware was updated by opening the Advanced System Management Dashboard window. IBM Power Systems S822LC for High Performance Computing...

  • Page 61: Appendix A. Server Racks And Energy Management

    Server racks and energy Appendix A. management This appendix provides information about the racking options and energy management-related concepts that are available for the IBM Power Systems 822LC server. © Copyright IBM Corp. 2016. All rights reserved.

  • Page 62: Ibm Server Racks

    If a system is installed in a rack or cabinet that is not an IBM rack, ensure that the rack meets the requirements that are described in “OEM racks” on page 54.

  • Page 63: Ibm 7014 Model T42 Rack

    In the zone 4 earthquake environment, the rack must be configured starting with the heavier drawers at the bottom of the rack. IBM 7014 Model T42 rack The 2.0-meter (79.3-in.) Model T42 addresses the client requirement for a tall enclosure to house the maximum amount of equipment in the smallest possible floor space.
  • Page 64 With industry-standard quick couplings, the water lines in the door attach to the client-supplied secondary water loop. IBM Power Systems S822LC for High Performance Computing...

  • Page 65: Ibm 42U Slimrack 7965-94Y

    24-inch floor tiles. The IBM 42U Slim Rack has a lockable perforated front steel door that provides ventilation, physical security, and visibility of indicator lights in the installed equipment within. In the rear, either a lockable perforated rear steel door (#EC02) or a lockable Rear Door Heat Exchanger (RDHX)(1164-95X) is used.
  • Page 66 To match new power requirements and save previous investments, these power cords can be requested with an initial order of the rack or with a later upgrade of the rack features. IBM Power Systems S822LC for High Performance Computing...
  • Page 67 PDUs in the rack, and to connect each PDU to independent power sources. For detailed power requirements and power cord details about the 7014 racks, see the “Planning for power” section in the IBM Power Systems Hardware IBM Knowledge Center website: http://www.ibm.com/support/knowledgecenter/api/redirect/powersys/v3r1m5/topic/p7ha d/p7hadrpower.htm...

  • Page 68: Rack-Mounting Rules

    The system can be installed in a suitable OEM rack if that the rack conforms to the EIA-310-D standard for 19-inch racks. This standard is published by the Electrical Industries Alliance. For more information, see the IBM Power Systems Hardware IBM Knowledge Center at the following website: http://www.ibm.com/support/knowledgecenter/api/redirect/systems/scope/hw/index.jsp...
  • Page 69 Drawer Rail 494mm (19.45 in.) Mounting 719mm (28.31 in.) Flanges 51mm (2.01 in.) Front, No Door 451mm (17.76 in.) 494mm (19.45 in.) Figure A-3 Top view of rack specification dimensions (not specific to IBM) Appendix A. Server racks and energy management...

  • Page 70: Energy Management

    Intelligent Energy optimization capabilities enable the POWER8 processor to operate at a higher frequency for increased performance and performance per watt, or to reduce dramatically the frequency to save energy. IBM Power Systems S822LC for High Performance Computing...

  • Page 71: Ibm Energyscale Technology

    IBM EnergyScale technology IBM EnergyScale technology provides functions to help the user understand and dynamically optimize processor performance versus processor energy consumption, and system workload, to control Power Systems power and cooling usage. EnergyScale uses power and thermal information that is collected from the system to implement policies that can lead to better performance or better energy usage.
  • Page 72 Therefore, the IBM POWER Hypervisor™ can use nap mode as a general-purpose idle state. When the operating system detects that a processor thread is idle, it yields control of a hardware thread to the POWER Hypervisor.

  • Page 73: On Chip Controller

    POWER8 invested in power management innovations. A new OCC that uses an embedded IBM PowerPC® core with 512 KB of SRAM runs real-time control firmware to respond to workload variations by adjusting the per-core frequency and voltage based on activity, thermal, voltage, and current sensors.
  • Page 74 IBM Power Systems S822LC for High Performance Computing...

  • Page 75: Related Publications

    These publications are also relevant as further information sources: Active Memory Expansion: Overview and Usage Guide http://public.dhe.ibm.com/common/ssi/ecm/en/pow03037usen/POW03037USEN.PDF IBM EnergyScale for POWER8 Processor-Based Systems http://public.dhe.ibm.com/common/ssi/ecm/en/pow03039usen/POW03039USEN.PDF IBM Power Facts and Features - IBM Power Systems, IBM PureFlex System, and Power Blades http://www.ibm.com/systems/power/hardware/reports/factsfeatures.html IBM Power Systems S812L server specifications http://www.ibm.com/systems/power/hardware/s812l-s822l/specs.html IBM Power Systems S814 server specifications http://www.ibm.com/systems/power/hardware/s814/specs.html...

  • Page 76: Online Resources

    IBM Systems Energy Estimator http://www-912.ibm.com/see/EnergyEstimator/ Current information about IBM Java and tested Linux distributions are available here: https://www.ibm.com/developerworks/java/jdk/linux/tested.html Additional information about the OpenJDK port for Linux on PPC64 LE, as well as some pre-generated builds can be found here: http://cr.openjdk.java.net/~simonis/ppc-aix-port/...

  • Page 77: Help From Ibm

    Help from IBM IBM Support and downloads ibm.com/support IBM Global Services ibm.com/services Related publications...
  • Page 78 IBM Power Systems S822LC for High Performance Computing...
  • Page 80 Back cover REDP-5405-00 ISBN Printed in U.S.A ibm.com...

Table of Contents