Table of Contents About This Manual......................9 Audience..............................9 Organization............................9 HP Cluster Platform Documentation.....................9 Cross-Platform Documents......................11 Platform-Specific Documents......................11 Related Documentation........................11 Cluster Component Documentation....................11 Web Resources..........................11 HP Encourages Your Comments......................12 Text Annotations...........................12 Conventions............................12 Safety Considerations.....................15 Information on Ergonomics........................16 Recycling...............................17 1 Rack Installation Overview..................19 2 Platform Core Components..................21 2.1 Cluster Components........................21 2.2 Cluster Models..........................24...
Page 4
2.6.4.2 Rear View of the ProCurve 2848 Switch.................42 2.6.5 ProCurve 2848 LEDs and Buttons...................43 2.6.5.1 Reset Button........................44 2.6.5.2 Clear Button........................44 2.6.6 ProCurve 2824 Switch Overview....................45 2.6.6.1 ProCurve 2824 Network Ports..................45 2.6.6.2 ProCurve 2824 LEDs and Buttons...................45 2.7 KVM and Console Switches......................45 2.7.1 HP TFT7600..........................46 2.7.1.1 TFT7600 Rear Ports......................46 2.7.1.2 Brace Rail and Cable Management Arm.................47...
Page 5
A.6 Monitored Power Distribution Units for SVA Specifications............79 A.7 TFT5600 Rack-Mount Keyboard and Monitor Specifications............79 A.8 TFT7600 Rack-Mount Keyboard and Monitor Specifications............80 A.9 Eight-Port Console Switch Specifications..................80 B Preventing Electrostatic Damage................81 B.1 Electrostatic Precautions.........................81 B.2 Grounding Methods........................81 Index..........................83 Table of Contents...
Page 6
List of Figures CP6000 32-Node Dense Cluster, Component Rack Locations............23 Example of a 32-Node, Two-Rack CP6000 Cluster...............24 Example Cluster of 243 Application Nodes and 13 Utility Nodes..........27 42U HP 10000 Series Rack......................28 HP Rack 10000 G2 Series 42U Cabinet..................29 HP Modular Cooling System Front View..................31 HP Modular Cooling System Rear View..................32 HP ProCurve 2610-48 Switch......................34 HP ProCurve 2610-48 and 2610-24 Switch Rear View..............37...
Page 7
List of Tables Dense Clusters: Ratio of Nodes to Racks and Interconnects............25 Components used in a Cluster of 243 Application Nodes and 13 Utility Nodes......26 HP Rack 10000 Series Rack Callouts and Dimensions..............28 Ethernet Switch Models — Cluster Roles..................33 ProCurve 2848 LED Identification....................43 HP ProCurve Switch Diagnostics....................75...
About This Manual This manual provides an overview of the core infrastructure components for all HP Cluster Platform models, and associated platforms. Core infrastructure components, such as the rack, power distribution units, and network switches are common to all HPC Cluster Platform models. The HP Cluster Platform CPxxxx model consists of the following three groups of components: •...
Page 10
The typical HP Cluster Platform documentation set contains both cross-platform and platform-specific documents, as well as supplementary documents available at the time of release. For the latest information on Cluster Platform and supported software, go to the Unified Cluster Portfolio website: http://h20311.www2.hp.com/HPC/cache/275420-0-0-225-121.html...
Cross-Platform Documents Cross-platform documents are as follows: • Cluster Platform Customer Letter • Cluster Platform Overview • Cluster Platform Site Preparation Guide • Core Components • Server and Workstation Overview Platform-Specific Documents Platform-specific documents are as follows: • One or more system interconnect guides •...
• ProCurve (for cluster networking components, including Gigabit Ethernet interconnect). • Integrity (for cluster nodes). • ”Rack 10000” (the HP 10000 Series Rack used in clusters). • techservers or HPC (The High-Performance Computing home page). http://www.hp.com/products1/servers/integrity/index.html – The US home page for Integrity servers.
Page 13
In command and interactive examples, this type denotes literal items Boldface type entered by the user (typed user input). Boldface type in text indicates a word that is defined in the glossary. Italic (slanted) type indicates variable values, placeholders, and function Italic type argument names.
Safety Considerations To avoid bodily harm and damage to electronic components, read the following safety and comfort guidelines before unpacking and configuring the cluster components. Heed the following additional warnings and see the Site Preparation Guide to obtain specific information on safety issues.
Appendix Afor information on where to find information about the weight of a component. Removing and Replacing For your safety, never remove the cover from a cluster Component Covers component without first disconnecting the power cord from the power outlet and removing any connection to a telecommunications network.
Recycling Shipping an integrated cluster generates far less packaging than the individual components that it contains. However, large clusters use a substantial amount of packaging material that is not reusable. The bulk of the packaging material is recyclable, and is labeled as such. You should plan on providing a number of dumpsters into which this packaging can be sorted and recycled.
1 Rack Installation Overview Your HP Cluster Platform configuration is fully integrated and tested at the factory. The cluster racks are then disconnected from each other, packed, and mounted on shipping pallets for delivery to your site. The Site Preparation Guide describes the assembly status of the cluster when it is delivered to an installation site.
2 Platform Core Components A cluster is a set of independent computers combined into a unified system through system software and networking technologies. An overview of the generic HP Cluster Platform architecture is provided in the Cluster Platform Overview. The HP Cluster Platform consists of a combination of three component groups: the servers (nodes), the interconnect, and the core components that are common to all models of HP Cluster Platform.
Page 22
• Depending on the operating environment, the cluster configuration might be restricted to specific components, such as gigabit Ethernet switches rather than 10/100 Base-T switches. • The operating environment might also require specific connections between components, such as a connection between the control node and the interconnect, which is an optional configuration.
Figure 2-2 Example of a 32-Node, Two-Rack CP6000 Cluster 32-Node Dense Cluster 18-node CBB 2.2 Cluster Models As described in the Cluster Platform Overview, there are several models of HP Cluster Platform, each of which is available as prepackaged dense clusters, or as modular clusters. The clusters are assembled from compute building blocks (CBBs), interconnect building blocks (IBBs), and utility building blocks (UBBs).
are built with either a full-bandwidth 64-port interconnect or a reduced bandwidth 128-port interconnect as follows: — Reduced Bandwidth — Reduced bandwidth clusters are available in configurations of 129 to 256 nodes or 257 to 512 nodes. A reduced bandwidth cluster uses a switch configuration that has fewer switch cards and interconnect network routes, trading off hardware cost against system performance.
is delivered to the site, a floor plan and cable routing plan must be completed. For more information, see the Site Preparation Guide. Table 2-2 provides an example of the components used in a cluster of 243 application nodes and 13 utility nodes.
2.3 HP 10000 Series Racks The HP 10000 Series racks (see Figure 2-4) used in HP Cluster Platform are either 42U (78.838 inches) or 22U (43.838 inches) high, depending on the configuration requirements. The HP 10000 Series racks house the components on industry-standard 19-inch wide rails. The HP 10000 Series racks have interchangeable perforated front and rear doors, a set of side panels for each row of bayed racks, and can have up to four PDUs.
approximately eight inches of room outside of the internal mounting rails for routing the 288-port InfiniBand interconnect cables. The 800 mm-wide cabinet mounts flush with the standard 42U racks, so there is no need to offset this particular model. 2.4 HP Rack 10000 G2 Series The HP Rack 10000 G2 Series are designed for greater ease-of-use maintenance.
Page 30
environmental monitor controls the air temperature to the servers and adjusts automatically to meet the pre-established air temperature goal. The environmental monitor interacts with HP SIM (Systems Insight Manager) via SNMP traps. The Water Hookup kit is available for both versions as a separate kit to prepare the water hookup ahead of the arrival of the MCS.
Figure 2-7 HP Modular Cooling System Rear View The following items correspond to the callouts shown in Figure 2-7. Rear view of the HP 10000 Series 42U Rack Rear view of the HP Modular Cooling System (bolted to an HP 10000 Series 42U Rack) 2.5.1 Cabling For HP Cluster Platform solutions, all cables for rack-mount components in the HP 10000 Series 42U rack will be routed down the opposite side from where the HP Modular Cooling System is...
• An administrative subnetwork handles management and installation tasks, and isolates the administration traffic from application traffic in an HP Cluster Platform. • A console network provides node operations such as booting and shutdown. This network is connected to the server's dedicated management processor (MP) network port. •...
2.6.1 HP ProCurve 2610-48 and 2610-24 Switch Overview The HP ProCurve Switch 2610 Series includes the 24-port HP ProCurve Switch 2610-24 with 24 10/100Base-TX ports, and the 48-port HP ProCurve Switch 2610-48 Switch with 48 10/100Base-TX ports. There are three other models in the ProCurve Switch 2610 Series, the 2610-PWR models, which are not discussed in this document.
Clear button Link and Mode LEDs on 10/100Base-TX RJ-45 ports 10/100Base-TX RJ-45 ports (24 or 48, depending on which model is in the configuration) Link and Mode port status indicators on 1000Base-T ports 10. 1000Base-T ports (two) 11. Mini-GBIC ports (two) The following sections describe the ports, buttons, and indicator LEDs for HP ProCurve 2610-24 and 2610-48 switches.
but it may be powering another switch or the external power supply is not functioning correctly. • The green/orange Fan status LED is on when the fan is operating normally. When this LED is blinking orange, the cooling fan has failed and the switch Fault LED will also be blinking simultaneously.
• Deleting Passwords — When pressed by itself for at least 1 second, the button deletes any switch console access passwords that you may have configured. Use this feature if you have misplaced the password and need console access. • Restoring Factory Default Configuration —...
The difference between the two HP ProCurve Series 2600 models is in the number of ports, as shown in Figure 2-10. The following sections describe the HP ProCurve 2650 and 2626 switches. Figure 2-10 HP ProCurve Switch 2650 and 2626 The following items correspond to the callouts in Figure 2-10.
2.6.2.3 Self Test and Fan Status LEDs The HP ProCurve Series 2600 switches have a green Self Test LED. When the Self Test LED is off the switch is in normal state and not undergoing a self-test. When the Self Test LED is green, the switch self-test and initialization are in progress after the switch receives power or is being reset.
No voltage range settings are required. The console port on the back of the ProCurve Series 2600 switches (see Figure 2-11) connect the switch to a console via a serial cable. The console can be a PC or a workstation running a VT-100 terminal emulator. Figure 2-1 1 HP ProCurve Series 2600 Switch, Rear View Console Console port connector...
• VLAN support and tagging: support complete 802.1Q (4,096 VLAN IDs) and 256 VLANs simultaneously • Traffic prioritization (802.1p): allows real-time traffic classification into 8 priority levels mapped to 4 queues • Per-port broadcast throttling: selectively configure broadcast control on heavy traffic port uplinks •...
Figure 2-13 HP ProCurve Switch 2848 Front View hp P roCurve Switch 2848 J4904A T M 48 Status Power Mode Test Fault Dual Personality Ports 10/100/1000-t(T) or Mini-GBIC(M) Reset Clear Spd Mode off = 10Mbps flash = 100Mbps on = 1000Mbps Power and Fault LEDs Reset and Clear buttons Test and Fan status LEDs...
Note: The switch is installed in the rack with the rear panel facing the front of the rack. 2.6.5 ProCurve 2848 LEDs and Buttons The front panel of the HP ProCurve 2848 switch has eight types of LEDs, as described in Table 2-5.
Table 2-5 ProCurve 2848 LED Identification (continued) Item Description State Meaning LED Mode 4 green Link = Indicates that the port LEDs are displaying link information. If the view LEDs port LED is on, the port is enabled and receiving a link indication from the connected device.
misplaced the password and need console access. This button is provided for your convenience, but its presence means that if you are concerned with the security of the switch configuration and operation, make sure the switch is installed in a secure location, such as a locked wiring closet.
• TFT7600 rack-mount keyboard and monitor (see Section 2.7.1) • TFT5600 rack-mount keyboard and monitor (see Section 2.7.2) • Four-port console switch for Cluster Platform. (see Section 2.7.3) • Console switch for Cluster Platform and Scalable Visualization Array (SVA). (See Section 2.7.4).
Figure 2-18 TFT7600 Rear Ports The following list corresponds to the callouts in Figure 2-18: USB keyboard/mouse USB pass-through (connects to a KVM switch virtual media port) PS2 keyboard PS2 mouse VGA input Power connection Cable tray Back of TFT7600 Serial firmware port —...
Figure 2-19 Brace Rail and Cable Management Arm The following list corresponds to the callouts in Figure 2-19: Brace rail (which mounts to both the left and right adjustable rails) Power supply Cable management arm Power supply cable for the TFT7600 2.7.1.3 Replacing the TFT7600 and Power Supply This section describes how to replace a TFT7600 and power supply.
Cables routed through the cable management arm Power supply attached to the cable management arm Cables are routed through the cable management arm and exit through the side rail (see callout 1 in Figure 2-23) To replace the TFT7600, follow these steps: Extend the TFT7600 forward to the locked position (see Figure 2-17).
Note: The TFT7600 is not mounted with screws to the inner rails (see callout 1 in Figure 2-22). It simply slides out from the inner rails once the slide locks are simultaneously pulled back toward the front of the unit releasing the TFT7600 from the inner rails. Figure 2-22 Tool-less Rails and Slides The following list corresponds to the callouts in Figure...
2.7.1.5 Replacing a TFT7600 Power Supply To replace a TFT7600 power supply, follow these steps: Warning! Ensure that the power to the unit is disconnected before removing the power supply. Failure to do so can result in damage to the unit or personal injury. Extend the TFT7600 to the locked position.
Note: The TFT5600 is installed in older models of HP Cluster Platform. Figure 2-25 TFT5600 Rack-Mount Keyboard and Monitor 2.7.3 Console Switch for Cluster Platform A 4-port USB switch (Figure 2-26) provides up to four outputs to the control node and utility nodes.
Note: An optional 16-port console switch is available for future expansion capability. The 16-port console switch also offers dual console support. Figure 2-27 shows an 8-port console switch only. Figure 2-27 8–port Console Switch The console switch has the following features: •...
• Compatibility with HP/Compaq video monitors and “hot keys” on all HP/Compaq keyboards, including Compaq TFT5600 and TFT7600. • Multiple language support through programmable ROM SoftPaq (English, Spanish, French, Dutch, German, Italian, and Japanese versions). 2.7.5 Expansion Module for SVA The expansion module connects to an 8-port switch, letting you manage a total of 16 computers.
Mouse Local I/O cable connected to the control node server blade Half-height server blade c-Class enclosure You will need a local I/O cable to connect to the server, as shown in Figure 2-30. Figure 2-30 Local I/O Cable The following list describes the callouts in Figure 2-30: Server blade connector...
is a single-phase unit that branches out into four single-phase branch circuits. The power strips mount directly to the rear of the rack frame and plug into the control units. Each HP Cluster Platform rack has up to four modular PDUs. For Cluster Platform racks with server blades there may be up to eight modular PDUs in the rack.
Warning! Some clusters use zero-U PDUs, which are mounted in the side of a rack and made accessible by removal of the rack's side panel. To ensure safety and serviceability, do not block access to the side of the rack. Warning! For installations in computer rooms with overhead power supplies, the maximum available power cable length when routed to the top of the rack is approximately 36 inches.
Figure 2-34 Power Strips – Single (New HP Cluster Platform Models) The following list describes the callouts in Figure 2-34: Power cable Power strip Mounting hardware Right rear rack column Right rear rack chassis Platform Core Components...
Figure 2-35 Power Strips – Double (New HP Cluster Platform Models) The following list describes the callouts in Figure 2-35: Power cable Power strip Mounting hardware Right rear rack column Right rear rack chassis Caution: Do not use the power strips to supply power to any components not already configured into the cluster.
Figure 2-36 PDU for SVA Fasteners — quantity three (3). Attach the PDU to the rack chassis, and not the column inside the rack. PDU status monitoring screen. Warning! For installations in computer rooms with overhead power supplies, the maximum available power cable length when routed to the top of the rack is approximately 36 inches.
Page 61
The MSA2000 SAN storage arrays are 2U units that are mounted in the HP Series 10000 rack. For more information on the MSA2000 line of storage arrays, go to: http://h18006.www1.hp.com/storage/disk_storage/msa_diskarrays/san_arrays/index.html 2.9 Storage...
3 Connecting and Powering Up the Core Cluster Components HP Cluster Platform cluster building blocks are connected, configured, and tested during the manufacturing process. After technicians complete the final diagnostic process at the factory, they disconnect the cables and package the cables separately from the cluster to prevent them from being damaged during shipment.
Note: The TFT7600 is installed in recent Scalable File Share (SFS) systems. Figure 3-1 TFT7600 Cabling The following list corresponds to the callouts shown in Figure 3-1. Utility node (a DL360 G4p is used in this example) Administrative (Control) node (a DL360 G4p is used in this example) PS/2 adapter Eight-port console switch TFT7600 rack-mount keyboard and monitor...
3.2 Connecting the KVM to the Control Node KVM cables are bundled in the rack that contains the TFT5600 rack-mount keyboard and monitor. When you receive the cluster, These cables are connected to the four-port KVM switch and to the TFT5600 console. You must complete the connections to the control node. Note: The four-port KVM switch is installed in older HP Cluster Platform models.
The VGA cable connector is connected to the server's video out port. The USB cable is connected to the topmost of the server's two stacked (generic) USB ports. Caution: The topmost port of the generic USB ports is the only supported connection. Do not use the dedicated keyboard and mouse ports or the lower generic (stacked) USB port.
Figure 3-3 Expansion Module Mounting The following list corresponds to the callouts in Figure 3-3 The mount location of the expansion module upper bracket is approximately U25 on the rack chassis. The KVM console switch connects to the connector labeled IN. Connectors numbered 1 through 8, connected to each individual workstation.
Figure 3-4 Console Cabling Control node (a DL380 in this example). Direct cabling from the KVM to the control node. Use this method when there is no console switch. PS/2 interface adapter. Console switch. Rack-mount KVM. KVM cables connected to the console switch. CAT5 cable from the console switch in the UVB to the expansion module in the VBB.
administrative connection to the array's storage controllers. You use this connection to open a command terminal and enter Array Configuration Utility (ACU) commands to the controllers. The HP XC implementation of the ACU provides a command-line interface (CLI) only. The controller management cable is a serial console cable. This cable has a DB-style female plug, which you connect to the port labeled console serial A on the control node.
4 Component Firmware Your HP Cluster Platform configuration arrives with the latest versions of the firmware installed on the components. However, the firmware for these components may be updated at a future date. This manual does not provide instructions on how to install or upgrade firmware for the components described in this document, but does provide pointers on where to obtain firmware upgrades.
5 Ethernet Hardware Troubleshooting This chapter provides some basic hardware troubleshooting information for the ProCurve Ethernet switches included in your cluster. The following topics are described in this chapter: • Basic Ethernet troubleshooting tips (Section 5.1) • An explanation of component LEDs (Section 5.2) •...
Front Panel HP Procurve Switch The front panel of the HP ProCurve switch usually has seven types of diagnostic LEDs. Figure 5-1 shows these LEDs for a ProCurve 2650. See “HP ProCurve Switches” (page 32) for the specific LEDs that are on your version of the HP ProCurve switch. Figure 5-1 HP ProCurve 2650 LEDs hp P roCurve Switch 2848...
Item Description State Meaning Port LED 4 green LEDs Link = Indicates that the port LEDs are displaying link information. If the View port LED is on, the port is enabled and receiving a link indication from the connected device. If the port LED is off, the port has no active network cable connected, or is not receiving link beat or sufficient light.
A Component Specifications This appendix provides operating and performance specifications for core HP Cluster Platform components. Specifications might change for individual components as products are improved. Consult the user information for a specific components to obtain the latest specification and operation restrictions.
• The HP Modular Cooling System overview website provides an overview of the MCS as well as the specifications and key benefits. For more information, go to: http://h10010.www1.hp.com/wwpc/us/en/en/WF05a/ 3447589-3447589-3446285-3446371-3461917-1155256.html • TheHP Modular Cooling QuickSpecs provides the latest specifications and other information on the HP Modular Cooling System.
A.4 HP ProCurve 2848 Switch and HP ProCurve 2824 Specifications The following list provides links to the latest specifications and other information for the HP ProCurve 2848 and HP ProCurve 2824 switches: • The ProCurve Networking Switch 2800 series overview website provides information on specifications, features, and support for the HP ProCurve 2848 and HP ProCurve 2824 switches.
NOTE: The TFT5600 rack-mount keyboard and monitor was discontinued on March 29, 2006 in North America. HP recommends upgrading to the HP TFT7600 rack-mount keyboard and monitor. • The HP TFT5600 Integrated Rack-Mount Keyboard and Monitor Manuals website provides links to setup and users manuals for the TFT5600. Go to: http://h20000.www2.hp.com/bizsupport/TechSupport/ DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64179&taskId=101&prodTypeId=329290&SeriesId=331826 •...
B Preventing Electrostatic Damage To prevent damaging the system, you must follow the configuration procedures and be aware of the standard precautions for handling electronic parts. A discharge of static electricity from a finger or other conductor may damage system boards or other static-sensitive devices. This type of damage can reduce the life expectancy of a component.
Index firmware, 71 Symbols groups, 21 10000 G2 series racks hot components, 16 specifications, 77 KVM, 21 10000 series racks power (PDU), 21 specifications, 77 rack, 21 storage, 21 components, 21 ACU (see Array Configuration Utility) configuration annotations, 12 dense, 23 Array Configuration Utility, 68 configuration options, 24 audience for this book, 9...
Page 84
federated cluster, 25 laser radiation, 15 feedback LED, 73 e-mail address for documentation, 12 power. fault, and locator, 35 firmware, 71 ProCurve switch, 73 upgrade, 71 LED mode select button and indicator LEDs firmware upgrade ProCurve 2610, 36 overview, 71 LEDs format ProcCurve 2610 RPS.
Page 85
for SVA examples, 26 specifications, 79 safety, 15 specifications, 79 stability, 15 port LED racks ProCurve 2610, 36 HP Rack 10000 G2, 29 power HP Rack 10000 Series, 28 components, 21 HP Rack 10000 Series wide, 28 distribution unit specifications, 77 for SVA, 59 reader skills, 9 power-up sequence, 69...
Need help?
Do you have a question about the Cluster Platform Introduction v2010 and is the answer not in the manual?
Questions and answers