HP Integrity MC990 X User Manual
  1. Manuals
  2. Brands
  3. HP Manuals
  4. Server
  5. Integrity MC990 X
  6. User manual
HP Integrity MC990 X User Manual

HP Integrity MC990 X User Manual

Hide thumbs Also See for Integrity MC990 X:
Table of Contents

Advertisement

Quick Links

See also: Supplemental Service Manual
HPE Integrity MC990 X Server User
Guide
Abstract
This guide provides an overview of the architecture, general operation and descriptions of the
major components that comprise the HPE Integrity MC990 X Server system. It also provides
the standard procedures for powering on and powering off the system, basic troubleshooting
and maintenance information, Foundation Software usage information, and important safety
and regulatory specifications.
Part Number: 855704-003a
Published: May 2017
Edition: 4

Advertisement

Table of Contents
loading

Related Manuals for HP Integrity MC990 X

Summary of Contents for HP Integrity MC990 X

  • Page 1 This guide provides an overview of the architecture, general operation and descriptions of the major components that comprise the HPE Integrity MC990 X Server system. It also provides the standard procedures for powering on and powering off the system, basic troubleshooting and maintenance information, Foundation Software usage information, and important safety and regulatory specifications.

  • Page 2: Table Of Contents

    Contents HPE Integrity MC990 X Server overview..........6 MC990 X server chassis....................... 6 System components........................6 Unit numbering.........................10 Rack numbering....................... 10 HPE Integrity MC990 X system features..................10 Distributed shared memory (DSM)...................10 Distributed shared I/O.......................11 Rack management controller....................11 Reliability, availability, and serviceability................11 Operational procedures................13...
  • Page 3 ...................... 28 topology command ...................... 29 Enabling CPU frequency scaling....................33 About CPU frequency scaling..................33 CPU frequency scaling for Integrity MC990 X systems........... 34 Additional Foundation Software utilities..................38 Component replacement procedures..........40 Maintenance precautions and procedures..................40 Preparing the system for maintenance or upgrade............
  • Page 4 System technical information.............. 65 System architecture........................65 ccNUMA architecture........................66 Cache coherency......................66 Non-uniform memory access (NUMA)................67 Safety and regulatory information............68 Safety information........................68 Regulatory information........................69 Regulatory Model Numbers..................... 69 Glossary....................70 Contents...

  • Page 5: Revision History

    © 2016, 2017 Hewlett Packard Enterprise Development LP Notices The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

  • Page 6: Hpe Integrity Mc990 X Server Overview

    (RMC) unit, power distribution units (PDU), and optional mass storage units. MC990 X server chassis The basic enclosure within the Integrity MC990 X system is the MC990 X server chassis. The MC990 X server chassis contains one four-socket motherboard connected to support up to 28 NUMAlink ports, each with a maximum bidirectional bandwidth communication rate of up to 7.47 GB/sec.
  • Page 7 • 42U rack—These racks are used for the MC990 X enclosures, RMC, and optional external storage in the Integrity MC990 X system. Up to eight MC990 X enclosures can be installed in each 42U rack. • Server chassis—The Integrity MC990 X system includes one base server chassis and can include one or more expansion server chassis.
  • Page 8 BaseIO board—Optional I/O riser board (connected directly to the motherboard) that supports base system I/O functions including one Gbit Ethernet connector (top), four USB ports, and bays for two optional external 1.8-inch solid state drives (SSDs). Internally, the BaseIO board supports: HPE Integrity MC990 X Server overview...
  • Page 9 RMC---This 1U-high rack management controller provides external LAN and USB connections into the Integrity MC990 X system and acts as a top layer of system control. An internal 24-port Ethernet switch allows system control for up to multiple MC990 X enclosures in an expanded Integrity MC990 X system.

  • Page 10: Unit Numbering

    Distributed shared memory (DSM) In the Integrity MC990 X system, memory is physically distributed both within and among the MC990 X server chassis (compute/memory/I/O); however, it is accessible to and shared by all NUMAlinked devices within the SSI. This means all NUMAlinked components sharing a single Linux operating system operate and share the memory fabric of the system.

  • Page 11: Distributed Shared I/O

    Integrity MC990 X system units within the SSI. In addition, the RMC provides the top layer of system control for Integrity MC990 X system. Through the use of an internal 24-port Ethernet switch, a single RMC can provide system control for multiple MC990 X server chassis in an expanded Integrity MC990 X system.
  • Page 12 Systems can report status inventory information, provide hardware logs of out-of-range conditions, or perform recovery procedures through remote commands. ◦ All system faults are logged in files. ◦ Memory can be scrubbed using error checking code (ECC) when a single-bit error occurs. HPE Integrity MC990 X Server overview...

  • Page 13: Operational Procedures

    SSI. These components in concert are generically known as the system control network. The Integrity MC990 X system control network provides control and monitoring functionality for each motherboard, power supply, and fan assembly in each MC990 X server chassis in the system.

  • Page 14: Accessing The System Control Network

    Supports entry of controller commands to monitor or change particular system functions within a particular MC990 X server chassis. See the HPE Integrity MC990 X Server RMC Software User Guide for a complete list of command line interface (CLI) commands.
  • Page 15 Log in to the RMC as root with the default password “root”, as in this example: asylum$ ssh root@mc990x-rmc root@mc990x-rmc's password: root MC990 X RMC, Rev. 1.1.xx [Bootloader 1.1.x] RMC:r001i01c> help NOTE: HPE recommends changing all default logins and passwords. Once a connection to the RMC is established, system control commands can be entered.

  • Page 16: Powering The System Up And Down

    Type <cmd> --help for help on individual commands. Powering the system up and down This section explains how to power up and power down individual units, or your entire Integrity MC990 X system, as follows: • Preparing to power up on page 16 •...

  • Page 17: Powering Up And Down From The Command Line Interface

    3. If you plan to power up an Integrity MC990 X system that includes optional mass storage enclosures, make sure that the power switch on the rear of each PSU/cooling module (one or two per enclosure) is in the (On) position.

  • Page 18: Remote Lan Connection To The Rmc

    Remote LAN connection to the RMC If you have an Integrity MC990 X system and wish to use a remote or local system to administer the system via LAN, you can connect via Ethernet cable to the RMC node WAN port identified in Figure 9: RMC Ethernet LAN (WAN port) location on page 17.

  • Page 19: Power Up The System Using The Rmc Network Connection

    Power up the system using the RMC network connection About this task You can use a network connection to power on your Integrity MC990 X system as described in the following steps: Procedure 1. You can use the IP address of the RMC to perform an SSH login, as follows: ssh root@<IP-ADDRESS>...

  • Page 20: Monitoring Power Up

    Use CTRL+]+q to exit the console when needed. Depending on the size of your system, it can take 5 to 10 minutes for the Integrity MC990 X system to boot to the EFI shell. When the shell> prompt appears, enter fs0: as in the following example: shell>...

  • Page 21: Power Down The System

    Power down the system To power down the MC990 X system, use the power off command, as follows: RMC:> power off ==== r001i01c (PRI) ==== You can also use the power status command, to check the power status of your system: RMC:>...

  • Page 22: System Control

    The RMC is a separate stand-alone controller installed in the Integrity MC990 X system rack. The RMC acts as a gateway and buffer between the Integrity MC990 X system control network and any other public or private local area networks or systems used to communicate with the Integrity MC990 X system SSI.

  • Page 23: Rmc Overview

    The RMC provides the top layer of system control for the Integrity MC990 X system. This controller is a stand-alone 1U high rack mount chassis.

  • Page 24: Bmc Overview

    Each MC990 X server chassis has a dedicated RMC interface connector on the front of the system chassis. System controllers Each Integrity MC990 X system must have at least one RMC installed. RMC functions on page 25 describe the basic features and functions of the BMC controllers. NOTE: For additional information on controller commands, see the HPE Integrity MC990 X Server RMC Software User Guide.

  • Page 25: Rmc Functions

    RMC functions The following list summarizes the control and monitoring functions that the RMC performs: • Supports a Gigabit Ethernet interface • Supports a limited (IPMI 2.x) interface for power control • Controls powering up/down of the MC990 X server chassis in the system •...

  • Page 26: Using The Foundation Software

    Using the Foundation Software Foundation Software (FS) includes automatic boot-time optimization utilities, reliability features, and technical support tools. Designed for high-performance computing, these tools help maximize system performance and availability. While many FS utilities and tools work in the background to optimize program performance, other tools require configuration information from the system administrator.

  • Page 27: Monitoring System Performance

    U9 DQ4 Temp = 22C r1i0n0:Dec 9 07:30:25 r1i0n0 MEMLOG[4595]: P1-DIMM3A has a failed DRAM and must be replaced soon. Exposure to Uncorrected Error is high r1i0n0:Dec 9 07:30:25 r1i0n0 MEMLOG[4595]: Read ECC P1-DIMM3A Rank 0 Bank 0 Row 0x0 Col 0x8 Temp = 22C •...

  • Page 28: Hubstats Command

    The Linux utilities include w(1), ps(1), top(1), vmstat(8), iostat(1), and sar(1). You can use Linux system monitoring utilities on all MC990 X platforms, including Integrity MC990 X systems. The FS utilities include hubstats, linkstat, nodeinfo, and topology. HPE supports these utilities on all MC990 X platforms.

  • Page 29: Topology Command

    2877 1094 TOT 134148848 131320512 2828336 492 67320 144436 42802 0 35129 7673 Press "h" for help From an interactive nodeinfo session, enter h for a help statement. For example: Display memory statistics by node. quit Increase starting node number. Used only if more nodes than will fit in the current window.
  • Page 30 2 USB Controllers 1 VGA GPU topology command requests system summary and shows node and CPU information mc990x-sys:~ # topology --summary --nodes --cpus System type: MC990 X System name: harp34-sys Serial number: MC-00000034 Partition number: 0 2 Compute chassis 8 CPUs 2 Nodes 30.00 GB Memory Total 15.00 GB Max Memory on any Node...
  • Page 31 4 r001i06s01 0001:01:00.0 56,1511-1526 LSI SAS2308 Fusion-MPT SAS-2 4 r001i06s02 0001:02:00.0 64,1480-1510 Intel P3700 Non-Volatile Memory Controller 4 r001i06s03 0001:03:00.0 66,1527,1533-1562 Intel P3700 Non- Volatile Memory Controller 5 r001i06s05 0002:02:00.0 88,1563-1593 Intel P3700 Non-Volatile Memory Controller 5 r001i06s06 0002:03:00.0 90,1594-1624 Intel P3700 Non-Volatile Memory Controller 6 r001i06s07 0003:01:00.0 104,1625-1655 Intel P3700 Non-Volatile...
  • Page 32 Memory Controller 23 r001i28s12 000c:03:00.0 2559 NVIDIA GK110BGL [Tesla K40m] 28 r001i38s01 000d:01:00.0 344,2509-2524 LSI SAS2308 Fusion-MPT SAS-2 28 r001i38s02 000d:02:00.0 352,2245-2275 Intel P3700 Non-Volatile Memory Controller 28 r001i38s03 000d:03:00.0 354,2276-2306 Intel P3700 Non-Volatile Memory Controller 29 r001i38s05 000e:02:00.0 376,2307-2337 Intel P3700 Non-Volatile Memory Controller 29 r001i38s06 000e:03:00.0 378,2338-2368 Intel P3700 Non-Volatile...

  • Page 33: Enabling Cpu Frequency Scaling

    . 0000:02:00.0 0-14,480-494 Intel I210 Gigabit Network Connection . 0000:06:00.0 0-14,480-494 Matrox G200eR2 4 r001i06s01 8 0001:01:00.0 60-74,540-554 LSI SAS2308 Fusion- MPT SAS-2 4 r001i06s02 8 0001:02:00.0 60-74,540-554 Intel P3700 Non- Volatile Memory Controller 4 r001i06s03 8 0001:03:00.0 60-74,540-554 Intel P3700 Non- Volatile Memory Controller 5 r001i06s05 10 0002:02:00.0...

  • Page 34: Cpu Frequency Scaling For Integrity Mc990 X Systems

    For example, theoretically, you can cut power consumption if you clock the processors from 2 GHz down to 1 GHz. CPU frequency scaling for Integrity MC990 X systems The procedures that explain how to configure CPU frequency scaling on MC990 X systems depend on whether your system includes the intel_pstate directory.
  • Page 35 Enabling CPU frequency scaling systems without the intel_pstate directory About this task The procedure in this topic explains how to enable or disable CPU frequency scaling on MC990 X systems that do not include the intel_pstate directory. To enable CPU frequency scaling: Procedure 1.
  • Page 36 For information, see Enabling CPU frequency scaling systems without the intel_pstate directory on page 35. 2. Decide which governor setting is suitable for your site. ondemand is the default setting. HPE recommends that you change this to a site-specific setting and that you configure the governor to performance.
  • Page 37 cpupower freqency-set -g governor • On SLES 12 platforms, complete the following steps: a. Enter the following command: # cpupower frequency-set -g governor For governor, specify the setting you chose in the previous step. b. Enter the following command and verify that the governor setting you specified appears in the cpupower command output in the current policy field: # cpupower frequency-info NOTE:...

  • Page 38: Additional Foundation Software Utilities

    Proceed to Configuring turbo mode on systems without the intel_pstate directory on page 38 Configuring turbo mode on systems without the intel_pstate directory About this task The following procedure explains how to configure turbo mode. Procedure 1. Make sure that you configured a governor setting. For information about how to configure a governor setting, see Changing the governor setting on systems without the intel_pstate directory on page 35.
  • Page 39 By default, this utility starts every two minutes. To change that interval, set IRQBALANCE_SLEEPTIME in /etc/sysconfig/irqbalance to the desired number of seconds and restart irqbalance or reboot your system. The base-configuration package automatically configures the irqbalance utility. Using the Foundation Software...

  • Page 40: Component Replacement Procedures

    Component replacement procedures This chapter provides information about installing and removing PCIe cards, chassis fans and system disk drives from your Integrity MC990 X system, as follows: • Maintenance precautions and procedures on page 40 • Removing the chassis top cover on page 41 •...

  • Page 41: Removing The Chassis Top Cover

    About this task IMPORTANT: The MC990 X server chassis cover should not be removed while the Integrity MC990 X system is powered up. As shown in Figure 12: MC990 X server chassis cover removal on page 42, the MC990 X server chassis top is a two-piece bifurcated front and rear chassis cover.

  • Page 42: Adding Or Replacing Pcie Or Gpu Cards

    Figure 12: MC990 X server chassis cover removal Adding or replacing PCIe or GPU cards WARNING: Before installing, operating, or servicing any part of this product, read the Safety precautions on page 13. This section provides instructions for adding or replacing a PCIe or GPU-based PCIe card in your Integrity MC990 X system.
  • Page 43 NOTE: The x16 PCIe slots support cards with a maximum power consumption of 300 watts. The procedure requires a T-10 Torx driver and Phillips-head screwdriver to complete. Before installing the PCIe expansion cards, be sure to remove each respective slot cover and use its screw to secure your expansion card in place.

  • Page 44: Installing Or Replacing A Drive

    Installing or replacing a drive The MC990 X server chassis holds up to four hard disk drives in a tray assembly at the front of the chassis. The disk riser assembly occupies the riser card 1 position on the MC990 X server chassis motherboard.

  • Page 45: Remove Or Replace A 2.5-Inch Hard Drive

    Procedure 1. Shut down the Integrity MC990 X system operating system. 2. To remove power from the chassis, unplug all the MC990 X server chassis power supply cords. 3. Grasp the perforated front chassis cover at either end and pull it off the front of the unit.

  • Page 46: Remove Or Replace A 1.8-Inch Ssd Option Drive

    Remove or replace a 1.8-inch SSD option drive About this task The BaseIO card supports two optional “mini” 1.8-inch SATA solid-state-drives (SSDs) which are accessible from the front of the chassis. The two SSD slots are located directly below the BaseIO board’s USB ports.

  • Page 47: Replacing An Mc990 X Server Chassis Power Supply

    1. Ethernet port 2. USB ports (4) 3. SSD drives (2) Replacing an MC990 X server chassis power supply About this task To remove and replace power supplies in an MC990 X server chassis, you do not need any tools. Under most circumstances, a single power supply can be replaced without shutting down the enclosure or the complete system.

  • Page 48: Replacing An Mc990 X Server Chassis Fan Assembly

    Figure 17: MC990 X server chassis power supply and fan locations 1. Enclosure fan assembly (4) 2. HARP fan assembly 3. Enclosure power supplies (4) Replacing an MC990 X server chassis fan assembly About this task Chassis cooling for each MC990 X server chassis is provided by four rear-mounted fan assemblies (as seen in Figure 17: MC990 X server chassis power supply and fan locations on page 48).
  • Page 49 Figure 18: MC990 X server chassis rear fan assembly Component replacement procedures...

  • Page 50: Integrity Mc990 X System Troubleshooting

    Integrity MC990 X system Troubleshooting This chapter provides the following sections to help you troubleshoot your system: • Troubleshooting chart on page 50 • LED status indicators on page 50 Troubleshooting chart Cause Table 1: Problems and recommended actions Problem Description Recommended Action The system will not power on.

  • Page 51: Power Supply Leds

    Power supply LEDs Each power supply installed in an MC990 X server chassis has a bicolor status LED. The LED will either light green or amber (yellow), or flash green or yellow to indicate the status of the individual supply. Table 2: MC990 X server chassis power supply LED states Power supply status Green LED...
  • Page 52 7. Top to bottom: Power Good CPU 0, Power Good CPU 1, Power Good CPU 2, Power Good CPU 3 8. Top to bottom: Power Good P12V A, Power Good P12V B, Power Good P12V C, Power Good P12V D Figure 19: System motherboard status LED locations and functions Integrity MC990 X system Troubleshooting...

  • Page 53: Websites

    Websites General websites Hewlett Packard Enterprise Information Library www.hpe.com/info/EIL Single Point of Connectivity Knowledge (SPOCK) Storage compatibility matrix www.hpe.com/storage/spock Storage white papers and analyst reports www.hpe.com/storage/whitepapers For additional websites, see Support and other resources. Websites...

  • Page 54: Support And Other Resources

    Support and other resources Accessing Hewlett Packard Enterprise Support • For live assistance, go to the Contact Hewlett Packard Enterprise Worldwide website: http://www.hpe.com/assistance • To access documentation and support services, go to the Hewlett Packard Enterprise Support Center website: http://www.hpe.com/support/hpesc Information to collect •...

  • Page 55: Customer Self Repair

    Customer self repair Hewlett Packard Enterprise customer self repair (CSR) programs allow you to repair your product. If a CSR part needs to be replaced, it will be shipped directly to you so that you can install it at your convenience.

  • Page 56: Regulatory Information

    HPE Networking Products www.hpe.com/support/Networking-Warranties Regulatory information To view the regulatory information for your product, view the Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products, available at the Hewlett Packard Enterprise Support Center: www.hpe.com/support/Safety-Compliance-EnterpriseProducts Additional regulatory information Hewlett Packard Enterprise is committed to providing our customers with information about the chemical substances in our products as needed to comply with legal requirements such as REACH (Regulation EC No 1907/2006 of the European Parliament and the Council).

  • Page 57: Technical Specifications And Pinouts

    I/O port specifications on page 60 Integrity MC990 X system specifications Integrity MC990 X system configuration ranges summarizes the Integrity MC990 X system configuration ranges. Note that while each MC990 X server chassis motherboard houses four processor sockets; each socket holds multiple processor cores. The number of cores supported will change based on customer selection and processor availability.

  • Page 58: Integrity Mc990 X System Environmental Specifications

    Front: 48 in. (121.9 cm) Rear: 48 in. (121.9 cm) Top: 18 in. (45.7 cm) Integrity MC990 X system environmental specifications Environmental specifications lists the environmental specifications of the Integrity MC990 X system. Table 6: Environmental specifications Feature Specification Temperature tolerance (operating) +5 °C (41 °F) to +35 °C (95 °F) (up to 1500 m / 5000 ft.)

  • Page 59: Integrity Mc990 X System Electrical Specifications

    40,000 ft. (12,195 m) non-operating Acoustic sound pressure Approximately 77 dBA (at rear of rack) Integrity MC990 X system electrical specifications MC990 X system electrical specifications shows the power specifications for the MC990 X system. Table 7: MC990 X system electrical specifications...

  • Page 60: I/O Port Specifications

    180-264 VAC (North America) 312-457 VAC (International) Frequency 50-60 Hz Total harmonic distortion Less than 10% at full load The Integrity MC990 X system RMC power and technical specifications are provided in RMC specifications. Table 8: RMC specifications Feature Specification Height 1.72 inches (44 mm)

  • Page 61: Motherboard Vga Port Information

    The motherboard VGA interface (see VGA port pin functions) can be used for all basic interaction with your Integrity MC990 X system. Note that it does not provide a direct interconnect to the system RMC. Figure 20: VGA port connector...

  • Page 62: Ethernet Port

    Function VSYNC DDCCLK Ethernet port The system auto-selects the Ethernet port speed and type (duplex vs. half-duplex) when the server is booted, based on what it is connected to. Figure 21: Ethernet port connector on page 62 shows the Ethernet port. Figure 21: Ethernet port connector Ethernet port pinouts shows the cable pinout assignments for the Ethernet port operating in 10/100- Base-T mode and also operating in 1000Base-T mode.

  • Page 63: Serial Port

    Serial port The system motherboards have 9-pin serial interface connectors. These ports provide serial access to the individual chassis and are capable of transferring data at rates as high as 230 kbps. Other features of the ports include the following: •...
  • Page 64 Figure 23: USB type A port connector Table 12: Pin assignments for USB type A connector Signal Wire color -Data White +Data Green Ground Black Technical specifications and pinouts...

  • Page 65: System Technical Information

    The Integrity MC990 X system is based on a cache-coherent non-uniform memory access architecture. Because it is modular, the Integrity MC990 X system architecture combines the advantages of lower entry cost with the ability to scale processors, memory, and I/O independently.

  • Page 66: Ccnuma Architecture

    When a processor modifies a block of data, the processors that have the same block of data in their caches must be notified of the modification. The Integrity MC990 X system uses an invalidation method to ccNUMA architecture...

  • Page 67: Non-Uniform Memory Access (Numa)

    maintain cache coherence. The invalidation method purges all unmodified copies of the block of data, and the processor that wants to modify the block receives exclusive ownership of the block. Non-uniform memory access (NUMA) In DSM systems, memory is physically located at various distances from the processors. As a result, memory access times (latencies) are different or non-uniform.

  • Page 68: Safety And Regulatory Information

    Safety and regulatory information This appendix provides safety information and regulatory specifications for your system. For additional important safety, environmental, and regulatory information, see Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products, available at http:// www.hpe.com/support/Safety-Compliance-EnterpriseProducts. Safety information Read and follow these instructions carefully: Follow all warnings and instructions marked on the product and noted in the documentation included...

  • Page 69: Regulatory Information

    Regulatory information For important safety, environmental, and regulatory information, see Safety and Compliance Information for Server, Storage, Power, Networking, and Rack Products, available at http://www.hp.com/support/ Safety-Compliance-EnterpriseProducts. This product contains a laser that is classified as a Class 1 laser device.

  • Page 70: Glossary

    Glossary ACPI Advanced Configuration and Power Interface APPWT Average Peak Performance in Weighted Teraflops Baseboard Management Controller Command Line Interface CNSL Console abbreviation Cyclic Redundancy Check Distributed Shared Memory Error Checking Code Extensible Firmware Interface FQDN Fully Qualified Domain Name Graphic Processing Unit IPMI Intelligent Platform Management Interface...
  • Page 71 NUMA Non-Uniform Memory Access Power Distribution Unit Quick Path Interconnect Reliability, Availability, Serviceability Rack Management Console Remote Solutions Serial Attached SCSI System Error Log SELinux Security Enhanced Linux SAP HANA System Identifier Scalable Memory Interconnect System Management Node Symmetric Multiprocessing Solid State Drive Single System Image Unit IDentifier...

Table of Contents