1. Manuals
  2. Brands
  3. Cray Manuals
  4. Server
  5. CX1000
  6. Installation manual

Cray CX1000 Installation Manual

Blade system
Hide thumbs Also See for CX1000:
Table of Contents

Advertisement

Quick Links

Download this manual
Blade System
Installation Guide

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the CX1000 and is the answer not in the manual?

Questions and answers

Subscribe to Our Youtube Channel

Related Manuals for Cray CX1000

Summary of Contents for Cray CX1000

  • Page 1 Blade System Installation Guide...
  • Page 2 Quoting of brand and product names is for information purposes only and does not represent trademark misuse. The information in this document is subject to change without notice. Cray will not be liable for errors contained herein, or for incidental or consequential damages in connection with the use of...

  • Page 3: Table Of Contents

    Front view ..........................16 2.5.2 Rear view ..........................22 Chapter 3. Installing the Cray CX1000 blade system in the cabinet ....33 Cray CX1000 blade system cabinet requirements ..............34 Preparing the Cray CX1000 blade chassis rack-mount kit ............35 Preface...
  • Page 4 Connecting the Cray CX1000 blade system to the site LAN ..........44 Testing LAN connections ....................... 46 Setting up the Cray CX1000 blade system for remote access via the CHC ....... 47 Setting up the Cray CX1000 compute blades for remote access via the BHC ....48 Appendix A.
  • Page 5 Installing rack-mounted rail in cabinet ................36 Figure 3-3. Securing the Cray CX1000 blade chassis in the cabinet ..........37 Figure 4-1. Connecting the Cray CX1000 blade system to the site LAN ........... 44 Figure 4-2. Chassis Hardware Console page ..................47 Figure 4-3.

  • Page 6: List Of Tables

    List of tables Table 2-1. Record information .........................8 Table 3-1. Rack-mount kit items ......................35...

  • Page 7: Preface

    Related publications Cray CX1000 blade system Maintenance and Troubleshooting Guide explains how to maintain, service, and upgrade the Cray CX1000 blade system. This guide is intended for use by qualified support personnel Cray CX1000 blade chassis User's Guide explains how to use the Cray CX1000 blade chassis.

  • Page 8: Legal Information

    Legal information Regulatory declarations and disclaimers Declaration of the manufacturer or importer We hereby certify that this product is in compliance with: European Union EMC Directive 2004/108/EC, using standards EN55022 (Class A) and EN55024 and Low Voltage Directive 2006/95/EC, using standard EN60950 International Directive IEC 60297 and US ANSI Directive EIA-310-E Safety compliance statement UL 60950-1 USA...

  • Page 9: Fcc Declaration Of Conformity

    Pursuant to Part 15.21 of the FCC Rules, any changes or modifications to this equipment not expressly approved by Cray, Inc may cause harmful interference and void the FCC authorization to operate this equipment. An FCC regulatory label is affixed to the equipment.

  • Page 10: Laser Compliance Notice

    Laser compliance notice This product that uses laser technology complies with Class 1 laser requirements. A CLASS 1 LASER PRODUCT label is affixed to the laser device. Class 1 Laser Product Luokan 1 Laserlaite Klasse 1 Laser Apparat Laser Klasse 1 Safety information For Your Safety, this manual contains important information, required to operate the server safely.
  • Page 11 Do not push any objects into the openings of your system components. Doing so can cause fire or electric shock by shorting out interior components Do not use top blade handle to lift the Cray CX1000 blade chassis. These are used only to install and remove the blade only...

  • Page 12: Electrical Safety

    Unplug the power cable before removing the power supply If the equipment has multiple sources of power, disconnect power from the system by unplugging all power cables from the power supplies to reduce shock hazard Never open the power supply module for any reason The power supplies in your system may produce high voltages and energy hazards.

  • Page 13: Data Integrity And Verification

    Data integrity and verification CAUTION Cray products are designed to reduce the risk of undetected data corruption or loss. However, if unplanned outages or system failures occur, users are strongly advised to check the accuracy of the operations performed and the data saved or transmitted by the system at the time of outage or failure.

  • Page 14: Installation Flowchart

    Installation flowchart Step Installation process Remarks Pointers in guide Unpacking the Inspect the Cray CX1000 blade Chapter 1 Cray CX1000 blade system packing, unpack and system unload, and store packing items. Installing the Cray Fix the rack mount brackets and...

  • Page 15: Chapter 1. Delivery

    Chapter 1. Delivery This chapter explains delivery, unpacking, and inspection for the Cray CX1000 blade system. Chapter 1. Delivery...

  • Page 16: Introduction

    The Cray CX1000 blade system is delivered in a box that contains all the data and items required for installation. Figure 1-1. Cray CX1000 blade system CAUTION It is mandatory for the Cray CX1000 blade system to be transported vertically.

  • Page 17: Inspecting The Cray Cx1000 Blade System Packing

    Before unpacking, inspect the Cray CX1000 blade system packing to check its condition: Check for any damages or any visible defects on the packaged Cray CX1000 blade system Shockwatch label should be in white color, which indicates that the package...

  • Page 18: Unpacking The Cray Cx1000 Blade System

    1.3.1 Removing the Cray CX1000 blade system packing CAUTION To avoid personnel injury, before lifting the Cray CX1000 blade system, remove all the Cray CX1000 compute blades to reduce the weight. The following steps help you to remove the packing: Cut the straps covering the package.

  • Page 19: Checking The Cray Cx1000 Blade System

    Perform a preliminary visual inspection after unpacking the Cray CX1000 blade system. The following list helps in your inspection: Check that the Cray CX1000 blade system delivered is compliant with the Purchase Order Check for internal damage, alterations and obvious safety hazards, such as...

  • Page 21: Chapter 2. Getting To Know The Cray Cx1000 Blade System

    Chapter 2. Getting to know the Cray CX1000 blade system This chapter gives an overview at the Cray CX1000 blade system-level architecture and a high-level description of each of these modules. Chapter 2. Getting to know the Cray CX1000 blade system...

  • Page 22: Overview

    Overview Your Cray CX1000 blade system is a high-density server system providing cluster architecture. It has 18 Cray CX1000 compute blade bays, making it ideally suited for High Performance Computing (HPC) cluster environment that requires a large number of high-performance servers in a small space. The Cray CX1000 blade system provides common resources that are shared by the Cray CX1000 compute blades, such as power, cooling, system management, network connections, and I/O switch.

  • Page 23: Figure 2-1. User Label

    The serial number and part number are on a label on the top of the Cray CX1000 blade chassis, as shown in the following illustration. Figure 2-1. User label Six Cray CX1000 blade systems can be simultaneously housed in a 42U cabinet. The...

  • Page 24: What Your Cray Cx1000 Blade System Offers

    18 blades. Hot-swap/Hot-plug capabilities The front bays on the Cray CX1000 blade system are hot-plug blade bays. The rear switch and management bays on the Cray CX1000 blade system are hot-plug module bays. You can add, remove or replace Cray CX1000 compute blades, CMM...
  • Page 25 System management capabilities Your Cray CX1000 blade system comes with a service processor in the CMM. This service processor, in conjunction with the system-management firmware that is provided with the service processor in each Cray CX1000 compute blade, enables you to remotely manage the Cray CX1000 blade system components, and the Cray CX1000 compute blades.

  • Page 26: Reliability, Availability, And Serviceability (Ras)

    The following is a list of RAS features that your Cray CX1000 blade system supports: Shared key components, such as power, cooling, and I/O...

  • Page 27: Features And Specifications

    Features and specifications Following is a summary of the features and specifications for your Cray CX1000 blade system: The Cray CX1000 blade system, 18 server, 7U Chassis, rack-able within a standard 19 cabinet includes front panel, power supplies and Midplane Dual CPU socket Cray CX1000 compute blades using the EP family of Xeon processors from Intel®...

  • Page 28: External Connections/Interfaces

    Three1Gb Ethernet ports -RJ45 connectors and a serial COM port for maintenance purpose are present on the CMM There is a control panel on the front of the Cray CX1000 blade chassis Cray CX1000 blade chassis power On/Off switch Cray CX1000 blade chassis power indicator LED Cray CX1000 blade chassis indicator blue LEDs –...

  • Page 29: Components, Controls, And Leds

    E. Chassis Management Module F. Power Supply Unit module G. Quad Switch Module H. Ultra Capacitor Module Ethernet Switch Module Note The illustrations in this document might differ slightly from your hardware. Chapter 2. Getting to know the Cray CX1000 blade system...

  • Page 30: Front View

    2.5.1.1 Cray CX1000 blade chassis In the front side, the Cray CX1000 blade chassis has a LCP, 18 bays to host Cray CX1000 compute blades and two bays to host fan blades. The Cray CX1000 blade chassis includes two handles that help in moving the Cray CX1000 blade chassis. The...

  • Page 31: Figure 2-4. Cray Cx1000 Compute Blade (18 In Total)

    Intel‟s Quick Path Interconnect (QPI) feature. Each Cray CX1000 compute blade interconnects with other Cray CX1000 compute blades through an embedded ConnectX QDR component and the Quad Switch Module. Figure 2-4. Cray CX1000 compute blade (18 in total) Figure 2-5.
  • Page 32 G. Identification LED 2.5.1.3 Cray CX1000 compute blade specific features The other Cray CX1000 compute blade features are: Powered down Cray CX1000 compute blades can be inserted and removed without disturbing operations Two processor sockets Up to two Westmere-EP processor...
  • Page 33 This indicator flashes On and Off to indicate traffic over the SATA link. This indicator is managed by the hardware. DANGER Hazardous energy is present when the Cray CX1000 compute blade is connected to the power source. Always replace the blade cover before inserting the blades into the Cray CX1000 blade chassis.

  • Page 34: Figure 2-6. Fan Blades

    Replace a failed Fan blade as soon as possible to restore cooling redundancy. See the Cray CX1000 blade system Maintenance and Troubleshooting Guide on the Cray CX1000 blade system Resource and Documentation CD for instructions. Figure 2-6. Fan blades A.

  • Page 35: Figure 2-7. Local Control Panel

    2.5.1.5 Local Control Panel The Local Control Panel (LCP) is the panel related to the Cray CX1000 blade chassis. This panel is placed in the center of the Cray CX1000 blade chassis. Figure 2-7. Local Control Panel A. Power On/Off button B.

  • Page 36: Rear View

    D. Quad Switch Module E. UCM F. PSU2 G. ESM H. PSU1 2.5.2.1 Cray CX1000 blade chassis The rear side of the Cray CX1000 blade chassis provides bays for Cray CX1000 blade system PSUs, CMM, Quad Switch Module, UCM, and the ESM modules.

  • Page 37: Figure 2-9. Power Supply Unit

    2.5.2.2 Power Supply Unit module There are four PSU modules in the Cray CX1000 blade system, each of 2900W rating. These take in the AC input of 180VAC to 264VAC and provide 12V main and 3.3V standby outputs. The four PSU modules provide N+1 redundancy. The typical system power dissipation is around 7900W depending on the configuration and the customer‟s applications.
  • Page 38 LED indicators Fault LED The Fault LED indicator is amber. Slow blinking or Solid On state indicates that the PSU module has failed or reached a critical state warranting its replacement. Power LED The Power LED indicator is green. Slow blinking green indicates that AC is being applied to the PSU module and also that the 3.3V stand by voltage is available.

  • Page 39: Figure 2-10. Chassis Management Module

    2.5.2.3 Chassis Management Module The CMM‟s primary function is to provide the Cray CX1000 blade chassis-level management functionality for the Cray CX1000 blade system. This includes detection, power On/Off, management status monitoring for the Cray CX1000 compute blades and other hardware modules within the Cray CX1000 blade chassis, PSU management status monitoring, and system fan controls.
  • Page 40 The identification LED indicator is blue. This indicator is a unit identifier dedicated to the maintenance operations to localize physically Cray CX1000 blade chassis from the rear side. This LED is switched On simultaneously with the LCP identification LED indicator (for the front side).

  • Page 41: Figure 2-11. Quad Switch Module

    2.5.2.4 Quad Switch Module The Cray CX1000 blade system supports a Quad Switch Module, based on a 36 port QDR switch Board (QSB) to provide the quad switching function between the 18 Cray CX1000 compute blades and the external QSFP ports. The Quad Switch Module plugs into the Midplane from the rear side of the Cray CX1000 blade chassis.

  • Page 42: Figure 2-12. Ultra Capacitor Module

    LED indicators Identification LED In the CHC web interface, select Maintenance tab and click Identification LED. This gives you an option to switch On/Off the identification LED of QSM. Power LED The power LED indicator is bicolor: Amber/green indicating the power state: Amber: 3.3V stand-by power presence Green: 12V main power presence IB link activity LED...
  • Page 43 This indicator is a unit identifier dedicated to the maintenance operations to localize physically the module. The switching On/Off of this indicator is driven by the maintenance operator (external SNMP remote command to the Cray CX1000 blade chassis). Charge LED The charge LED indicators are tricolor: green, amber, and red.

  • Page 44: Figure 2-13. Ethernet Switch Module

    2.5.2.6 Ethernet Switch Module The ESM, an optional module, is present only to provide the Ethernet switching function through the three external Ethernet ports. Figure 2-13. Ethernet Switch Module A. Dynamically configurable stacking or Ethernet ports B. Handle C. Thumb screw D.
  • Page 45 Note The LAN activity for the buried links of the embedded Ethernet switch (between the Ethernet switch and each Cray CX1000 compute blade) are displayed on a LED in the front of each Cray CX1000 compute blade. Chapter 2. Getting to know the Cray CX1000 blade system...

  • Page 47: Chapter 3. Installing The Cray Cx1000 Blade System In The Cabinet

    Chapter 3. Installing the Cray CX1000 blade system in the cabinet This chapter explains how to install the Cray CX1000 blade system in a Cray Cabinet. It includes the following topics: Preparing the Cray CX1000 blade chassis rack-mount kit Installing rack-mount rails in the cabinet...

  • Page 48: Cray Cx1000 Blade System Cabinet Requirements

    Cray CX1000 blade system cabinet requirements The following condition is required: Cray CX1000 blade system has to be installed in a 42U Cray rack Figure 3-1. Recommended position of Cray CX1000 blade system...

  • Page 49: Preparing The Cray Cx1000 Blade Chassis Rack-Mount Kit

    Preparing the Cray CX1000 blade chassis rack-mount kit 3.2.1 Checking the rack-mount kit The Cray CX1000 blade system is delivered with a mounting kit for installation in a Cray rack. The rack-mount kit contains the following items: Item P/N/MPN Rack-mount kit-left...

  • Page 50: Installing Rack-Mount Rails In The Cabinet

    Installing rack-mount rails in the cabinet Note The rack rails each comprise a fixed part and a mobile part. The following steps help you to install rack-mount rails in the cabinet: Fully extend the rack rail. Prepare two brackets. Assemble the two brackets on the fixed part (vertical rails) of the rack rail, taking care to place the right-angle bracket bends toward the front and rear ends of the fixed part: Front and Rear ends: Insert the two M5X12 screws with M5 washers through...

  • Page 51: Securing The Cray Cx1000 Blade Chassis In Position

    Cray CX1000 blade chassis. From the front of the cabinet, insert Cray CX1000 blade system into the rack and ensure that it is sitting on the Teflon tape properly, which is stuck on the Cray CX1000 blade chassis.
  • Page 52 B. Rack-mounted rails C. Cray CX1000 blade chassis...

  • Page 53: Chapter 4. Connecting And Testing The Cray Cx1000 Blade System

    Chapter 4. Connecting and testing the Cray CX1000 blade system This chapter provides instructions for connecting and configuring the Cray CX1000 blade system. Chapter 4. Connecting and testing the Cray CX1000 blade system...

  • Page 54: Connecting The Cray Cx1000 Blade System To The Site Power Supply

    Distribution Unit (PDU) that is plugged into an appropriate electrical outlet. Note The drawer powering depends on the setting Start Mode. With the setting Deep Standby the Cray CX1000 compute blades are standby off, Light Standby standby on, and Unlocked powered off.

  • Page 55: Powering On The Cray Cx1000 Blade System For The First Time

    Powering on the Cray CX1000 blade system for the first time After you connect the Cray CX1000 blade system to the power supply, the CMM comes up first. However, the Cray CX1000 compute blades in the Cray CX1000 blade system are not turned on. You can power on the blades with either of the two...

  • Page 56: Checking Correct Operation

    Power LED on the blade should be Amber (the blade is in stand by mode) If Start Power Mode is set to Full Power On, then the Cray CX1000 blade chassis fans should start running and the Power LED on the blade should be Amber (the blade is in stand by mode).

  • Page 57: Powering Off The Cray Cx1000 Blade System

    Powering off the Cray CX1000 blade system There are two types of powering off the Cray CX1000 compute blades in the Cray CX1000 blade system: Power Off, also called as soft power off, initiates a graceful shutdown of the Operating System (OS). The user intervention may be required based on the power policy set in the OS.

  • Page 58: Connecting The Cray Cx1000 Blade System To The Site Lan

    Connecting the Cray CX1000 blade system to the site To manage the Cray CX1000 blade system and Cray CX1000 compute blades, you must first set up the remote connection through the Ethernet port on the CMM. Complete the following steps to connect the Ethernet cable to the CMM.
  • Page 59 (BHC IP, Eth0 IP, and Eth1 IP) from the DHCP server. If you are configuring the CHC‟s IP address for the first time, you need to give specific IP address to CHC based on: MAC address Cray Chassis_CMC Chapter 4. Connecting and testing the Cray CX1000 blade system...

  • Page 60: Testing Lan Connections

    Cray CX1000 blade system or LAN outlet ends. Ping to the IP address of the Ethernet switch (IP address of Ethernet switch can be found from the DHCP server log based on MAC address or Cray Blade_NCB_BMC. If you cannot ping, then check the LAN connections or test the Ethernet...

  • Page 61: Setting Up The Cray Cx1000 Blade System For Remote Access Via The Chc

    SNMP V3 IPMI V2.0 WEB GUI SMASH/CLP Figure 4-2. Chassis Hardware Console page Note For more information on configuring the Cray CX1000 blade system, see Cray CX1000 blade chassis User’s Guide. Chapter 4. Connecting and testing the Cray CX1000 blade system...

  • Page 62: Setting Up The Cray Cx1000 Compute Blades For Remote Access Via The Bhc

    Setting up the Cray CX1000 compute blades for remote access via the BHC Log on to Ethernet Switch page to check if the link is up and running for the corresponding BHC, if not then you should log into CHC web interface and from CHC configure BHC network link.

  • Page 63: Figure 4-3. Blade Hardware Console Page

    Click Remote Control to see the Console Redirection window option. Click the Console Redirection to see Launch Console option. Click Launch Console to see a pop up of the KVM console window. This connects the console to the blade. Chapter 4. Connecting and testing the Cray CX1000 blade system...
  • Page 64 For more information on configuring the Cray CX1000 compute blades, see Cray CX1000 compute blade User’s Guide.

  • Page 65: Appendix A. Cray Cx1000 Blade System Specifications

    Appendix A. Cray CX1000 blade system specifications The Cray CX1000 blade system is delivered rack-mounted in 42U cabinet. Cabinet Dimensions /Weight Unpacked Packed Height: 30.7 cm (12 in) Height: 62 cm (24.4 in) Width: 44.6 cm (17.55 in) Width: 60 cm (23.62 in) Depth: 74.7 cm (29.4 in)
  • Page 66 United States of America Nominal voltage 230 VAC (Phase / Neutral) Voltage range 210 to 240V AC Frequency 60 Hz 0.3% Breaker Protection (Mains Power) 16A per PSU Maximum inrush current 210A / per quarter period Table A.1. Cray CX1000 blade system specifications...

  • Page 67: Glossary

    Glossary BHC: Blade Hardware Console Web based GUI, used to control and monitor an individual Cray CX1000 compute blade in a Cray CX1000 blade chassis. ABR: Automatic BIOS Recovery BIOS: Basic Input Output System This invention describes a recovery method...
  • Page 68 COM1 or COM2: Webs based GUI, used to control and monitor The name assigned to a serial port to set or the whole of the CRAY CX1000 BLADE SYSTEM. change its address. See Serial Port. Chipset CPU: Central Processing Unit...
  • Page 69 MiniPCI card, Communications Daughter Card (CDC), or integrated on the system DIMM socket board. The socket used in Cray CX1000 blade system is for 240-pin DDR-3 memory modules. Ethernet interconnect It is used to aggregate traffic between clients DN: Distinguished Name and „server farms,‟...
  • Page 70 FCC: Federal Communications Commission High Performance Computing cluster A computer cluster is a group of linked Firmware computers, working together closely so that in An ordered set of instructions and data stored many respects they form a single computer. to be functionally independent of main storage.
  • Page 71 ILB: I/O Legacy Board Video, LAN, USB, etc... Cray CX1000 compute blades JTAG: Joint Test Action Group Refers to the Cray CX1000 compute blade It is the common name used for the IEEE Cray CX1000 blade system 1149.1 standard entitled Standard Test Access...
  • Page 72 LED: Light Emitting Diode A small electronic device that glows when current flows through it. Nehalem CPU Board This refers the early version of the Cray CX1000 compute blade or NCB. MAC: Mandatory Access Control Network interface A type of access control by which the...
  • Page 73 NVRAM: Non-Volatile Random Access PCI - Express slots Memory Slots for add in PCI-E cards. Cray CX1000 A type of RAM that retains its contents even blade system does not have any PCI-E slots. when the computer is powered off.
  • Page 74 These factors help to ensure the integrity of the fan on and off. By altering the relative on the data stored on your Cray CX1000 to off times the average voltage "seen" by the compute blade; that your Cray CX1000 fan is also altered.
  • Page 75 Cray CX1000 compute blade SSL: Secure Socket Layer The blades that provide dual QDR InfiniBand They are cryptographic protocols that (IB) channel to the InfiniBand Switch. provide security and data integrity for communications over networks such as the SIMD: Single Instruction, Multiple Data...
  • Page 76 VLAN: Virtual Local Area Network XML: eXtended MarkUp Language A local area network with a definition that General-purpose specification for creating maps workstations on some other basis than custom markup language geographic location (for example, by department, type of user, or primary application). VT-d: Virtualization for Directed I/O No entries.

  • Page 77: Index

    Power On, 41 Fan blade, 20 Power Supply Unit, 23 firmware interfaces, 47 Powering off, 43 Forced power Off, 43 Powering on the Cray CX1000 blade system, 41 Front view, 16 Preparing Functionality, 10 rack mount kit, 33 hot swappable, 23...
  • Page 78 via BHC, 47 SNMP command, 26 via the CHC, 47 Removing shipping bracket, 4 Turned On, 21 RJ45, 25 Ultra Capacitor Module, 28 Charge LED, 29 Server blade, 16, 18 Fault LED, 29 Fault LED, 18 Identification LED, 29 HDD activity LED, 19 IB activity LED, 19 LAN 0 activity LED, 19 LAN1 activity LED, 19...

Table of Contents