Page 1 IBM Storage Scale System 6000 Hardware Planning and Installation Guide SC28-3485-03...
Page 2 IBM welcomes your comments; see the topic “How to submit your comments” on page xvi. When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.
Page 3: Table Of Contents
Conventions used in this information......................xv How to submit your comments......................... xvi Chapter 1. Summary of changes................1 Chapter 2. Notices....................3 Trademarks..............................4 Caution, danger, and warning notices for IBM Storage Scale System 6000..........5 Environmental notices..........................7 Electromagnetic compatibility notices......................8 Canada Notice............................8 European Community and Morocco Notice................... 8 Germany Notice............................
Page 4 Installing the management server in a rack..................49 Unpacking IBM Storage Scale System 6000 (IBM SSR task)..............50 Installing support rails for the IBM Storage Scale System 6000 (IBM SSR task)......50 Installing enclosures (IBM SSR task)....................54 Installing the cable management arm (IBM SSR task)............... 57 Chapter 6. Monitoring the system using LEDs............61...
Page 5 MTM 5149-S54 switch service......................124 Configuration and cabling........................126 Warranty and service..........................127 Reference documentation........................128 Accessibility features for the system..............129 Accessibility features..........................129 Keyboard navigation..........................129 IBM and accessibility..........................129 Glossary......................131 Index........................ 139 v...
Page 6 vi ...
Page 7: Figures
20. Rear view of an IBM Storage Scale System 6000..................34 21. Orientation of ports on IBM Storage Scale System 6000................ 37 22. SSR and management access ports in the IBM Storage Scale System 6000......... 40 23. SSR access port in the Utility Node......................41...
Page 8 40. IBM Storage Scale System 6000 server canister LEDs................62 41. Close view of server canister LEDs......................63 42. Close view of IBM Storage Scale System 6000 Ethernet and BMC port LEDs........63 43. Close view of theIBM Storage Scale System 6000 PSU status LEDs............66 44.
Page 9 49. Ports view of the MTM 5149-S05 switch....................73 50. PSUs view of the MTM 5149-S05 switch....................73 51. Ports views of the MTM 5149-S54 switch....................74 52. PSUs view of the MTM 5149-S54 switch....................74 53. 5149-N64 switch ports..........................74 54. 5149-N64 switch power........................... 74 55.
Page 10 74. Releasing the metal latches........................89 75. Pressing the lock button........................... 89 76. Releasing latch while pulling the handle....................90 77. Extracting a fan module..........................90 78. PSUs view of the MTM 5149-S05 switch....................91 79. Ports view of the MTM 5149-S05 switch....................91 80.
Page 11 99. Displaying one of the cables grommets of an air duct................105 100. Aligning the right edge tabs of the air duct top with the right edge of the air duct bottom....105 101. Sliding the air duct top into the air duct bottom.................. 105 102.
Page 12 124. Connecting an external AC power source to the PSUs of an MTM 5149-S54 switch......122 125. Connecting cables to the RJ-45 port of an MTM 5149-S54 switch............ 122 126. Connecting cables the SFP+ port of an MTM 5149-S54 switch............123 127.
Page 13 Tables 1. Conventions..............................xv 2. Latest updates made in the IBM Storage Scale System 6000 Service Guide..........1 3. Overview of the IBM Storage Scale System 6000 system................. 14 4. Storage capacities and feature codes of NVMe-attached drives...............14 5. System maximum measured power specifications................... 23 6.
Page 14 23. Summary of supported transceivers for host interface adapters............39 24. Description of the system status LEDs..................... 62 25. IBM Storage Scale System 6000 Ethernet and BMC port status LEDs........... 63 26. Description of the canister status LEDs....................64 27. IBM Storage Scale System 6000 PSU status LEDs.................. 66 28.
Page 15: About This Information
About this information Who should read this information This information is intended for administrators of IBM Storage Scale System 6000 that includes IBM Storage Scale RAID. Conventions used in this information Table 1 on page xv describes the typographic conventions used in this information. UNIX file name conventions are used throughout this information.
Page 16: How To Submit Your Comments
In the left margin of the document, vertical lines indicate technical changes to the information. How to submit your comments To contact the IBM Storage Scale development organization, send your comments to the following email address: scale@us.ibm.com xvi IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 17: Chapter 1. Summary Of Changes
Chapter 1. Summary of changes Summary of changes for IBM Storage Scale System 6000 documentation. The updates to the IBM Storage Scale System 6000 Hardware Planning and Installation Guide include the changes that are detailed in the following table. Table 2. Latest updates made in the IBM Storage Scale System 6000 Service Guide...
Page 18 2 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 19: Chapter 2. Notices
This information was developed for products and services offered in the US. This material might be available from IBM in other languages. However, you may be required to own a copy of the product or product version in that language in order to access it.
Page 20: Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at www.ibm.com/legal/copytrade.shtml.
Page 21: Caution, Danger, And Warning Notices For Ibm Storage Scale System 6000
Caution, danger, and warning notices for IBM Storage Scale System 6000 Ensure that you read and understand all notices for IBM Storage Scale System 6000. Attention: Only a trained person must install, remove, or replace the field replaceable units (FRU).
Page 22: Multiple Power Sources
Protective earthing DANGER: The equipment must be grounded properly to earth. Failure to use properly grounded outlets may result in electrical shocks. 6 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 23: Environmental Notices
Environmental notices This information contains all the required environmental notices for IBM Storage Scale System 6000. The IBM Systems Environmental Notices includes statements on limitations, product information, product recycling and disposal, battery information, flat panel display, refrigeration and water-cooling systems, external power supplies, and safety data sheets.
Page 24: Electromagnetic Compatibility Notices
EMC compliance requirements as offered, including auxiliary equipment and cables that may be ordered from IBM to be used in conjunction with IBM Storage Scale System 6000 (MTM 5149-F48). Canada Notice...
Page 25: Japan Electronics And Information Technology Industries Association (Jeita) Notice
Tel: 914-499-1900 Der verantwortliche Ansprechpartner des Herstellers in der EU ist: IBM Deutschland GmbH Technical Relations Europe, Abteilung M456 IBM-Allee 1, 71139 Ehningen, Germany Tel: +49 800 225 5426 e-mail: Halloibm@de.ibm.com Generelle Informationen: Das Gerät erfüllt die Schutzanforderungen nach EN 55024 und EN 55032 Klasse A.
Page 26: Japan Voluntary Control Council For Interference (Vcci) Notice
Operation of this equipment in a residential area is likely to cause harmful interference, in which case the user will be required to correct the interference at his own expense. 10 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 27: Homologation Statement
Properly shielded, grounded, and approved cables and connectors must be used in order to meet FCC emission limits. IBM is not responsible for any radio or television interference caused by using other than approved cables and connectors, or by unauthorized changes or modifications to this equipment.
Page 28 12 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 29: Chapter 3. Technical Overview
SAS-attached flash drives. IBM Storage Scale System 6000 can contain up to 48 NVMe-attached drives. You can either use 24 drives in a half-populated configuration or 48 drives in a fully populated configuration. These drives are accessible from the front of IBM Storage Scale System 6000, as shown in the following figure.
Page 30 Figure 2. Rear view of IBM Storage Scale System 6000 The IBM Storage Scale System 6000 uses two AMD EPYC 9454 SP5 per server canister. The following table provides an overview of IBM Storage Scale System 6000. Table 3. Overview of the IBM Storage Scale System 6000 system...
Page 31 – PN: 01LL984 - CX-7 VPI, dual port, 200 Gb. – PN: 01LL983 - CX-7 InfiniBand, single port, 400 Gb. The ports and LEDs details of IBM Storage Scale System 6000 are shown in the following figure. Chapter 3. Technical overview 15...
Page 32: Ibm Storage Scale System 6000 Ports And Leds
Dual CXL (not used) USB A Figure 3. IBM Storage Scale System 6000 ports and LEDs Networking details The networking details of IBM Storage Scale System 6000 are shown in the following figure. 16 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 33: Ibm Storage Scale System 6000 Networking
• FC: AJQQ - CX-7 InfiniBand, single port, 400 Gb NVMe transport protocol in IBM Storage Scale System 6000 The IBM Storage Scale System 6000 systems use the Non-Volatile Memory express (NVMe) drive transport protocol. • NVMe is designed specifically for flash technologies. It is a faster and less complicated storage drive transport protocol than SAS.
Page 34: System Topology
You need a minimum of one EMS node as a part of your IBM Storage Scale System cluster. The EMS node can be ordered as a part of IBM Storage Scale System 6000.
Page 35: Ibm Storage Scale System 6000 Configurations
Figure 7. Performance configuration with 48 drives installed For more information, see Table 3 on page 14 and Table 4 on page 14. Drives IBM Storage Scale System 6000 supports industry-standard 2.5-inch NVMe-attached drive options. The following storage capacities are supported for NVMe-attached drives: • 3.84 TB •...
Page 36: Drive Slots
Drives are hot swappable from the front of the enclosure. Self-encrypting drives On IBM Storage Scale System 6000, you can configure all the supported HDDs and NVMe drives as self-encrypting drives (SEDs). Drive slots A drive slot represents the location in an enclosure into which a drive can be inserted.
Page 37: Ssr Access Port
Note: An IBM intranet connection is required to access the guide. In each of the IBM Storage Scale System 6000 server canisters, the RJ45 10 GbE port is designated as the SSR access port, as shown in the following figure.
Page 38 For more information, see the Model 5149-23E server specifications of the IBM Storage Scale System Utility Node. 22 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 39: Chapter 4. Planning For Ibm Storage Scale System 6000 Hardware
Ensure that the system's physical configuration records are easily accessible to be used as a reference as needed. The customers must work with the IBM technical team to plan for installation and deployment of an IBM Storage Scale System 6000.
Page 40: Environmental Requirements
Measurements that are obtained in other operating environments might vary. Conduct your own testing to determine specific measurements for your environment. Each IBM Storage Scale System 6000 enclosure contains two power modules for redundancy. The total power consumption values represent the total power that is drawn by both power modules when operating together or a single power module when operating in a maintenance or failure mode.
Page 41 The weight of this part or unit is more than 46 kg (102 lb). It takes specially trained persons, a lifting device, or both to safely lift this part or unit. (C011) The following table shows the rack space requirements for IBM Storage Scale System 6000 in tabular form.
Page 42: Service Envelope Illustration (Dimensions In Millimeters)
The service clearance area is the area around the rack that is needed for the authorized service representatives to service the enclosure as shown in the sample illustration in the following figure. 26 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 43: Service Clearance Requirements
Figure 12. Sample illustration to show space around the rack For IBM Storage Scale System 6000, use the recommended measurements that are given in the following table. Table 10. Service clearance requirements Front Back 915 mm (36 in.) 915 mm (36 in.) Storage racks require larger service clearances in the front of the rack.
Page 44: Supported Drives
Supported drives The following table provides drive specifications for your IBM Storage Scale System 6000 system. Table 12. Drive specifications Model and type 2.5 in. drives 4U IBM Storage Scale System 6000, with 48 drive 12 or 24 dual-port NVMe drives slots of 2.5 in.
Page 45: Powering On Ibm Storage Scale System 6000
Powering on IBM Storage Scale System 6000 Powering on an IBM Storage Scale System 6000 by connecting cables The following devices are needed to power on an IBM Storage Scale System 6000: • IBM Storage Scale System 6000 • 4 power cables with C19 connectors •...
Page 46: Ibm Storage Scale System 6000 Ports
• A USB keyboard Complete the following steps to power on an IBM Storage Scale System 6000. 1. Keep the IBM Storage Scale System 6000 on a bench. Figure 14. IBM Storage Scale System 6000 2. Identify all the ports that are needed to power on the system.
Page 47: Planning For Site Preparation
Planning for site preparation This information is intended to help you prepare your physical site for the installation of an IBM Storage Scale System 6000. Marketing and installation planning representatives are also available to help you Chapter 4. Planning for IBM Storage Scale System 6000 hardware 31...
Page 48: Site Preparation And Physical Planning
IBM Storage Scale System 6000 contains two server canisters. Each IBM Storage Scale System 6000 cluster consists of the following components: • One or more IBM Storage Scale System 6000 systems – in a rack, each system requires a space of 4U (standard rack units).
Page 49: Ibm Storage Scale System 6000 Model
The following figure shows the CPUs and the location of the DIMM slots. It shows one of the two server canisters for an IBM Storage Scale System 6000 (MTM 5149-F48), which is open and oriented for DIMM removal and replacement.
Page 50: Planning For Power For Server Canister
Attention: The power cords are the main power disconnect. Ensure that the socket outlets are located near the equipment and are easily accessible. The following figure shows the rear view of an IBM Storage Scale System 6000. The power modules are located on the sides of the IBM Storage Scale System 6000.
Page 51: Physical Installation Planning
“Environmental requirements” on page 24. • Do you have adequate rack space for your hardware? The system requires two Electronic Industries Alliance (EIA) units for each IBM Storage Scale System 6000. For more information, see “Dimensions and weight requirements for rack installation” on page •...
Page 52: Operating Environment
As you plan your installation, you must consider IP address requirements and service access for the system. IBM Storage Scale System 6000 uses 100 GbE / 200 GbE / 100 Gb EDR / 200 Gb HDR network for cluster communication and data transport. For IP port usage requirements, see IBM Storage Scale Documentation.
Page 53: Planning For Cables
Planning for cables Connections for IBM Storage Scale System 6000 Care must be taken to note the orientation of each IBM Storage Scale System 6000 so that the interconnect cables are properly connected. The following figure shows the ports on IBM Storage Scale System 6000.
Page 54: Summary Of Supported Host Interface Adapters
Supported host interface adapters, cables, and transceivers Each IBM Storage Scale System 6000 server canister has eight Gen 5 x16 PCIe slots to support optional host interface adapters. The host interface adapters can be supported in any of the interface slots.
Page 55: Planning For Adapters
OSFP: octal small form factor pluggable. QSFP: quad small form factor pluggable. Planning for adapters Each IBM Storage Scale System 6000 server has two PCIe interface slots to support extra adapters. The following two adapters are supported for use in the IBM Storage Scale System 6000: •...
Page 56: Planning Worksheets (Customer Task)
(or update) the /etc/hosts file on the management server node during the software installation. For reference, see IBM ESS Install Complete Guide and IBM Storage Scale System Deployment Guide. The high-speed interfaces are on a separate switch and must be defined on a different subnet.
Page 57: Management Port In The Utility Node
Figure 23. SSR access port in the Utility Node Figure 24. Management port in the Utility Node The following figure shows the cabling for IBM Storage Scale Systems. Chapter 4. Planning for IBM Storage Scale System 6000 hardware 41...
Page 58 VLAN) to port (management port) on each canister as shown in the previous figures. • If a management server node is received with IBM Storage Scale System 6000, connect two Ethernet cables from the management switch (if there are VLANs, connect from the management VLAN) to port C11-T1 (top port) and to port C11-T2 (second from top).
Page 59 – Port 1 – Management port (set by the SSR) - Logical port: mgmt – The SSR connects a laptop to the SSR port when performing the IBM Storage Scale System 6000 tasks. You need to move the cable when you work on a different canister.
Page 60 For the IBM Storage Scale System Utility Node, it is highly recommended for the SSR to set a campus connection on the host. This is in addition to the BMC connection.
Page 61 (top node in building-block) IO node 2 FSP (HMC1 port) 10.0.0.102 255.255.255.0 interface (top node in building- block) IBM Storage Scale System 5000 building-block 2 (higher position in frame) IP address Netmask IO node 1 management interface 192.168.45.23 255.255.255.0 (bottom node in building-block) IO node 1 FSP (HMC1 port) 10.0.0.103...
Page 62 255.255.255.0 Protocol (host) management 192.168.45.81 255.255.255.0 interface Protocol (host) BMC interface 10.0.0.141 255.255.255.0 IBM Storage Scale System 6000 building-block 1 (lower position in frame) IP address Netmask Canister 1/A management 192.168.45.60 255.255.255.0 interface (top node in building block) Canister 1/A interface (top node 10.0.0.60...
Page 63 The VLAN tag is required to route traffic on the BMC interface to the service network. This interface is connected to the green trunk ports on the switch. This is required only on IBM Storage Scale System 3200 and IBM Storage Scale System 3500. The default is 101.
Page 64 48 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 65: Chapter 5. Installing
Note: An IBM intranet connection is required to access this content. Note: Much of the information in this section is intended only for IBM authorized service providers. Customers need to consult the terms of their warranty to determine the extent to which they must attempt any IBM Storage Scale System 6000 maintenance.
Page 66: Unpacking Ibm Storage Scale System 6000 (Ibm Ssr Task)
5. Remove the four corner reinforcement pieces from the carton. 6. Using the box knife, carefully cut the four corners of the carton from top to bottom. 7. Fold the sides and back of the carton down to uncover the rear of the IBM Storage Scale System 6000 system.
Page 67 Figure 26. Pulling the inner rail b) Slide and hold the release tab in the direction shown in a in the next figure. c) Remove the inner sliding rail from the outer mounting rail. See b in the following figure. Figure 27.
Page 68 Retract and hold the front retention bracket. See a in the following figure. b) Insert the pegs on the mounting flange into the rack holes. See b in the following figure. 52 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 69 c) Release the front retention bracket. See c in the following figure. Figure 30. Mounting the front left rail to the rack posts 4. Release the front retention bracket. a) Retract and hold the rear retention bracket. See a in the following figure. b) Insert the pegs on the mounting flange into the rack holes.
Page 70: Installing Enclosures (Ibm Ssr Task)
Following your enclosure location plan, install the IBM Storage Scale System 6000 system. CAUTION: • To lift an assembled IBM Storage Scale System 6000 system, you must use a mechanical lift or involve four people. Do NOT attempt to reduce the enclosure weight by temporarily removing server components from the enclosure.
Page 71 Figure 33. Lifting the system by using a mechanical lift 2. On both sides, engage the inner rail with the outer rail. 3. Slide the system until it stops. 4. Pull the release tab on the inner rail to release the lock. Then, push the system further into the rack. Chapter 5.
Page 72 Figure 34. Release tab of the inner rail 5. Use the slam latch of each ear to lock the system assembly on the rack. See the following figure for reference. 56 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 73: Installing The Cable Management Arm (Ibm Ssr Task)
Installing the cable management arm (IBM SSR task) Each canister of the IBM Storage Scale System 6000 requires a cable management arm (CMA); a total of two CMAs is required per enclosure. When working on replacing hot-swappable components, use the CMAs to smoothly pull and push the enclosure.
Page 74: Installing The Cma In The Bottom Canister
- Outer rail connector E - CMA body connector Figure 36. CMA parts and orientation 2. Install the CMA in the bottom canister: Figure 37. Installing the CMA in the bottom canister 58 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 75: Installing The Cma In The Top Canister
a) Insert the CMA body connector ( E ) into the outer rail connector ( D ). b) Insert the inner CMA connector ( A ) into the inner rail connector ( B ). c) Insert the outer CMA connector ( C ) into the outer rail connector ( D ). 3. Install the CMA in the top canister: Figure 38.
Page 76 60 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 77: Chapter 6. Monitoring The System Using Leds
• PCIe slots System status LEDs System status LEDs provide information about the status of the IBM Storage Scale System 6000. In the front of the system, three status LEDs are available per enclosure in the left ear, as shown in the following figure.
Page 78: Server Canister Status Leds
Identify. Server canister status LEDs These LEDs are used to monitor the status of server canisters and Ethernet ports of the IBM Storage Scale System 6000. The IBM Storage Scale System 6000 server has three status LEDs on the operator panel of each server canister.
Page 79: Ethernet And Bmc Ports Status Leds
Ethernet and BMC ports status LEDs These LEDs are used to monitor Ethernet and BMC ports of the IBM Storage Scale System 6000. The Ethernet ports and the BMC ports on the rear of server canister are for standard RJ45-style connectors.
Page 80: Server Canister Status Leds
Server canister status LEDs These LEDs are used to monitor server canisters of the IBM Storage Scale System 6000. The table describe the server canister LEDs included in the IBM Storage Scale System 6000. Table 26. Description of the canister status LEDs...
Page 81: Psu Status Leds
PSU status LEDs These LEDs are used to monitor power supply units (PSUs) of the IBM Storage Scale System 6000. The PSU status LEDs are visible from the rear of the enclosure. The following figure provides a close view of the power module status LED.
Page 82: Ibm Storage Scale System 6000 Psu Status Leds
Figure 43. Close view of theIBM Storage Scale System 6000 PSU status LEDs The following table describes bi-color PSU status LEDs. Table 27. IBM Storage Scale System 6000 PSU status LEDs Green color Amber color Description DC GOOD and FAULT...
Page 83: Drive Status Leds
These LEDs are used to monitor drives of the IBM Storage Scale System 6000. Each drive carrier assembly of IBM Storage Scale System 6000 includes a set of LEDs that are visible from the top of the drive carrier. On system startup, the green LED illuminates automatically.
Page 84 Table 28. IBM Storage Scale System 6000 drive status LEDs (continued) Color States Description Fault Amber No isolated FRU failures on the drive. Blink The drive is being identified by using an identify command. Solid The associated drive has been isolated and is requiring service or replacement.
Page 85: Appendix A. Ibm Storage Scale System 6000 - Slot Placement Summary
Appendix A. IBM Storage Scale System 6000 - Slot placement summary Each slot number and its usage, as well as VM usage when setup, are explained in the slot placement summary. The following table and illustration represent each slot and its usage: Figure 45.
Page 86 70 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 87: Appendix B. Pcie 5 X16 Dual-Port Ndr 200 Gb/S Infiniband And Ethernet Connectx-7 Vpi Adapter (Fc: Ajqs)
ConnectX-7 adapter is a PCI Express (PCIe) generation 5 (Gen 5) x16 adapter. This adapter can be used in a Gen 5 x16 PCIe slot in the IBM Storage Scale System 6000 system. The adapter enables higher HPC performance with new Message Passing Interface (MPI) offloads, such as MPI Tag Matching and MPI AlltoAll operations, advanced dynamic routing, and new capabilities to perform various data algorithms.
Page 88 Thermal requirement If you have a IBM Storage Scale System Utility Node, you might be required to set the thermal mode of the system to a setting other than the default setting, depending on your system, adapter, and cable type.
Page 89: Appendix C. Switches For Ibm Storage Scale System 6000
Appendix C. Switches for IBM Storage Scale System 6000 These switches can be installed in the same rack as the IBM Storage Scale System solution, in a similar way as other supported high-speed data switches. MTM 5149-N64 switch Figure 47. Ports view of the MTM 5149-N64 switch Figure 48.
Page 90: The Mtm 5149-N64 Switch
Figure 51. Ports views of the MTM 5149-S54 switch Figure 52. PSUs view of the MTM 5149-S54 switch The MTM 5149-N64 switch This topic describes the key features and specifications of the NVIDIA/Mellanox NDR switch, IBM MTM 5149-N64. The 5149-N64 switch introduction The NVIDIA/Mellanox NDR switch, IBM MTM 5149-N64 switch, is a managed 64-port switch that provides 400 Gb/s InfiniBand bandwidth per port in a 1U standard chassis design.
Page 91 The following image labels the 32 OSFP ports and highlights features on the port side of 5149-N64 switch. Port numbers are printed on the switch between the top and bottom ports. Appendix C. Switches for IBM Storage Scale System 6000 75...
Page 92: Power Side View
The chassis label is affixed to a pull out tab on the left of the system on the power side. The chassis label contains the part number, serial number, and GUID of the 5149-N64 switch. 76 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 93: Chassis Labels
Figure 57. Chassis labels Switch labels On the bottom of the switch you can find the regulatory label, and the MTM label, as shown in the following figure: Appendix C. Switches for IBM Storage Scale System 6000 77...
Page 94: Air Temperature
Air temperature Air flows from the power side of the switch to the port side (power-to-connector (P2C) air flow configuration). Temperature requirements for the switch are listed in the following table: 78 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 95: Service Indicators
System LEDs The following figure shows system LEDs on the port side of the 5149-N64 switch: Figure 59. LEDs on the port The following table lists the port side LED states: Appendix C. Switches for IBM Storage Scale System 6000 79...
Page 96: Fan Module Leds
Description Action required Solid green This fan unit is operating as expected. - Solid amber This fan unit is missing or not The fan unit should be replaced. operating properly. 80 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 97: Power Supply Unit Leds
PSUs might be faulty. the electrical outlet that the PSU is plugged into. Port LEDs Each OSFP port can be divided into 2 dual-lane ports, for a total of 4 ports. Appendix C. Switches for IBM Storage Scale System 6000 81...
Page 98: The 5149-N64 Switch Installation
Customers must complete a Technical and Delivery Assessment (TDA) that the assigned SSR then reviews with them. In conjunction with the TDA, planning information is available in IBM Docs. The SSR and the customer must also review and complete all of the planning information in IBM Docs. Before scheduling the trip onsite to perform the installation, the SSR should attend meetings with the customer to perform planning tasks.
Page 99: Installation Overview
TDAs are online forms, but can be exported in an .xlsx or .csv format, saved, and printed. TDA checklists are in the IMPACT website (this site requires an IBM ID). The IBM practitioner creates the pre-install checklist. Customers, their business partner, IBM quality practitioner or TAM, and the assigned SSR can access this site to look at and to work on the TDA.
Page 100: The 5149-N64 Switch And Rails Alignment
Figure 65. Inserting rack rails into the back of the rack A rack rail that is inserted into the back of the rack is shown in the following figure: 84 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 101: A Rack Rail That Is Inserted Into The Back Of The Rack
Figure 68. Fully inserted latch into the rack slots 3. Mount the 5149-N64 switch. a) Slide the switch into the channel on the rack rails. Push the switch until you hear the locking mechanism click on both sides. Appendix C. Switches for IBM Storage Scale System 6000 85...
Page 102: Mtm 5149-N64 Switch Service
Air duct and rack mount rail kit 01LL977 CRU Tier 1 NVIDIA800 Gb NDR twin port transceiver 01LL927 CRU Tier 1 PSU (P2C airflow) 01LL976 CRU Tier 1 Fan unit (P2C airflow) 01LL975 CRU Tier 1 86 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 103: Replaceable Parts
• Under the standard warranty – If an IBM representative installs a Tier 1 CRU at the customer’s request, the customer is charged for the installation. – If an IBM representative installs a Tier 1 CRU at IBM Support's request, the customer is not charged for the installation.
Page 104 Figure 72. Pulling out the system c) Press the spring latches on both sides of the system. Continue pulling out the system until the system rails are clear from the rack rails. 88 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 105 PSU is required for the switch to function. Before removing a PSU, ensure that the other PSU is connected and shows a solid green LED. 1. Remove a power cord from a PSU. Appendix C. Switches for IBM Storage Scale System 6000 89...
Page 106 Extract the fan module by pulling the left side of the handle. As the fan module unseats, the fan module status LED turns off. Figure 77. Extracting a fan module b) Remove the fan module from the switch chassis. 90 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 107: The Mtm 5149-S05 Switch
6.4 Tb/s, with port speeds spanning from 10 Gb/s to 200 Gb/s per port. The MTM 5149-S05 switch can be installed in the same rack as the IBM Storage Scale System solution, in a similar way as other supported high-speed data switches.
Page 108: Key Features And Components Of The Mtm 5149-S05 Switch
PSUs are placed on the contrary side of the enclosure. Ports view (front of the enclosure) The following figure shows the 32 200-GbE ports and the 3 management ports of an MTM 5149-S05 switch. 92 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 109: Labels Of The Mtm 5149-S05 Switch
Labels of the MTM 5149-S05 switch Different details are provided in the labels of the MTM 5149-S05 switch and its PSUs. PSUs labels Each PSU has its labels at its top side. Figure 83. PSUs label Appendix C. Switches for IBM Storage Scale System 6000 93...
Page 110: Switch Labels
Important: Regardless of the system specifications about absolute maximum temperature, the nonoperating environmental conditions of the system solid-state devices must adhere to the JEDEC JESD218 Enterprise-class SSD requirements for data retention endurance. These requirements must be 94 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 111: Service Indicators: Leds Of The Mtm 5149-S05 Switch
Solid green All fans are up and running. The faulty FRUs should be replaced. Solid amber Error, one or more fans are not operating properly. Fan status LED Appendix C. Switches for IBM Storage Scale System 6000 95...
Page 112 No identification in progress. Unit identifier LED Switch ports LEDs Figure 86. Switch port LEDs of an MTM 5149-S05 switch Figure 87. LED splitting options of an MTM 5149-S05 switch 96 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 113: Descriptions For The Led Splitting Options Of An Mtm 5149-S05 Switch
The link is down 4X/1XD link types Solid green The link is up with no traffic Flashing green The link is up with traffic Flashing amber A problem with the link Appendix C. Switches for IBM Storage Scale System 6000 97...
Page 114: Descriptions For The Fan Leds Of An Mtm 5149-S05 Switch
Table 36. Descriptions for the PSU LEDs of an MTM 5149-S05 switch State Description Action required Solid green All PSUs are connected and running normally. Flashing green AC present / 5VSB on (the PSU is off) Contact IBM support. 98 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 115: Mtm 5149-S05 Switch Installation
Customers must complete a technical and delivery assessment (TDA) that the assigned SSR then reviews with them. In conjunction with the TDA, planning information is available in IBM Docs. The SSR and the customer must also review and complete all of the planning information in IBM Documentation. Before scheduling the trip onsite to perform the installation, the SSR should attend meetings with the customer to perform planning tasks.
Page 116 TDAs are online forms, but can be exported in an .xlsx or .csv format, saved, and printed. TDA checklists are in the IMPACT website (this site requires an IBM ID). The IBM practitioner creates the preinstallation checklist. Customers, their business partner, IBM quality practitioner or TAM, and the assigned SSR can access this site to look at and to work on the TDA.
Page 117: Sliding The Switch Enclosure Into The Bottom Of The Air Duct
From the switches side of the enclosure, face the flat side of a rack mounting rail with one side of the enclosure. b) Align the enclosure pins with the key holes in the rack mounting rail. Appendix C. Switches for IBM Storage Scale System 6000 101...
Page 118: Securing The Rack Mounting Rail
3. Insert the eight cage nuts in the rack posts, in the position where the switch enclosure is meant to be installed in the rack. Figure 94. Inserting the cage nuts (example of two cage nuts that are inserted in position in a left rack post) 102 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 119 With the help of a second person who is at the rack side where the rack mounting blades were previously placed in position, insert the rack mounting blades in the rack mounting rails. Appendix C. Switches for IBM Storage Scale System 6000 103...
Page 120 Figure 98. Securing the rack mounting rail 7. Pass the PSUs cables through the cables grommet. Warning: Do not connect the PSUs cables to the external AC power source yet. 104 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 121 Insert the edges of the air duct bottom on the edge tabs of the air duct top. Figure 101. Sliding the air duct top into the air duct bottom c) Secure the air duct top by tightening its two thumbscrews. Appendix C. Switches for IBM Storage Scale System 6000 105...
Page 122: Mtm 5149-S05 Switch Service
01PG843 200 G QSFP56 Ethernet optical transceiver 03NK246 CRU Tier 1 Power cord 4.3 m (14 ft) 39M5510 CRU Tier 1 (drawer to IBM PDU, 250 V / 10 A) 106 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 123: Replaceable Parts
• Under the standard warranty – If an IBM representative installs a Tier 1 CRU at the customer’s request, the customer is charged for the installation. – If an IBM representative installs a Tier 1 CRU at IBM Support's request, the customer is not charged for the installation.
Page 124 2. Label all data and power cables that connect to the faulty MTM 5149-S05 switch. Take note of which cables are plugged into the specific ports before you remove any cable. This record is needed for the installation of the new MTM 5149-S05 switch. 108 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 125 7. Pull the faulty MTM 5149-S05 switch enclosure out of the rack and place it on a flat antistatic surface. 8. Install the new MTM 5149-S05 switch. To install the new MTM 5149-S05 switch, follow the steps from “Installing the MTM 5149-S05 switch” on page 100. Appendix C. Switches for IBM Storage Scale System 6000 109...
Page 126: Locating The Release Latch And Handle Of A Psu
Wait for the FRU PSU LED to be solid green. If the LED does not turn solid green, restart the procedure, and make sure that the PSU is correctly seated. 110 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 127: The Mtm 5149-S54 Switch
This Edgecore switch, IBM MTM 5149-S54 switch, is a 1U, Gigabit Ethernet L2, 54-ports switch that offers 108 Gbps of switching capacity and 61.6 Mbps in forwarding rate. The MTM 5149-S54 switch can be installed in the same rack as the IBM Storage Scale System solution, in a similar way as other supported high-speed data switches.
Page 128: Key Features And Components Of The Mtm 5149-S54 Switch
Figure 110. Ports views of the MTM 5149-S54 switch For more information about the MTM 5149-S54 switch, see EPS121 in Edgecore documentation. The following table lists the IBM MTM and the Edgecore system model. Table 37. IBM IBM and Edgecore system model...
Page 129 • Options to enable UEFI Secure Boot. • Preloaded ONIE for automated loading of compatible open source or commercial NOS oﬀerings. • SPI flash with capacity of 16 Mb x 2. Appendix C. Switches for IBM Storage Scale System 6000 113...
Page 130: Views Of The Mtm 5149-S54 Switch
The following figure highlights the placement of the 3 fixed fans and the 2 PSUs of an MTM 5149-S54 switch. Figure 113. PSUs and fixed fans of an MTM 5149-S54 switch 114 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 131: Labels Of The Mtm 5149-S54 Switch
Specifications of the MTM 5149-S54 switch The following specifications provide brief information about the MTM 5149-S54 switch. System dimensions and weight System weight and dimensions are listed in the following table. Appendix C. Switches for IBM Storage Scale System 6000 115...
Page 132: Service Indicators: Leds Of The Mtm 5149-S54 Switch
System LEDs (rear of the enclosure) The following figure shows system state LEDs, which are visible on the port side of an MTM 5149-S54 switch. 116 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 133: Led Indicators For System, Loc, Psus, Fans, And Poe Of An Mtm 5149-S54 Switch
The management port has two LEDs, one on each side. The left LED (see 1 in the next figure) indicates that a link is established, and the right LED (see 2 in the next figure) indicates that the port is active. Appendix C. Switches for IBM Storage Scale System 6000 117...
Page 134: Mtm 5149-S54 Switch Installation
Figure 118. LED indicators for the 6 SFP + uplink ports of an MTM 5149-S54 switch MTM 5149-S54 switch installation This switch is installed by the customer or by IBM, depending on the deployment. Installing a system requires planning to facilitate a smooth installation, initialization, and configuration.
Page 135 Customers must complete a technical and delivery assessment (TDA) that the assigned SSR then reviews with them. In conjunction with the TDA, planning information is available in IBM Docs. The SSR and the customer must also review and complete all of the planning information in IBM Documentation. Before scheduling the trip onsite to perform the installation, the SSR should attend meetings with the customer to perform planning tasks.
Page 136 Align the first set of four holes (front to rear of the front bracket) of the front bracket with the first set of four holes (ports side to PSUs side of the enclosure) of the enclosure. 120 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 137 Align the holes of the rack brackets with the holes in the rack posts. c) Use four cage nuts to secure the enclosure in its position. Figure 123. Securing the MTM 5149-S54 switch enclosure to the rack 3. Connect to power. Appendix C. Switches for IBM Storage Scale System 6000 121...
Page 138 Figure 125. Connecting cables to the RJ-45 port of an MTM 5149-S54 switch b) For the SFP+ ports, install transceivers and then connect fiber optic cabling to the transceiver ports. 122 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 139 Use the micro-USB cable that is included with your MTM 5149-S54 switch to connect it to an external storage device through the micro-USB port of the MTM 5149-S54 switch. b) For the RJ-45 management port, use a Category 5, 5e, or better twisted-pair cable. Appendix C. Switches for IBM Storage Scale System 6000 123...
Page 140: Mtm 5149-S54 Switch Service
Each replaceable part is assigned one of three designations. Most of the parts for this switch are classified as Tier 1 CRU. None of the parts for this switch are Tier 2 CRU. For more information, see the following table. 124 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 141 • Under the standard warranty – If an IBM representative installs a Tier 1 CRU at the customer’s request, the customer is charged for the installation. – If an IBM representative installs a Tier 1 CRU at IBM Support's request, the customer is not charged for the installation.
Page 142: Configuration And Cabling
• For MTM 5149-S05 switch, see Nvidia Spectrum SN3700 in Nvidia documentation. For detailed configuration and cabling information, see the corresponding Hardware Planning and Installation Guide in IBM Storage Scale System documentation. 126 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 143: Warranty And Service
(CRU) has failed, the CRU part will be provided as part of the machine's standard warranty service. If on-site service is required, a service call is scheduled. Installation These switches are specified as IBM set up; installation is performed by an IBM representative. Appendix C. Switches for IBM Storage Scale System 6000 127...
Page 144: Parts Replacement
CRU whether the defective CRU must be returned to IBM. When a return is required, return instructions are shipped with the replacement CRU. The customer might be charged for the replacement CRU if IBM does not receive the defective CRU within 15 days of receipt of the replacement.
Page 145: Accessibility Features For The System
IBM Documentation, and its related publications, are accessibility-enabled. Keyboard navigation This product uses standard Microsoft Windows navigation keys. IBM and accessibility See the IBM Human Ability and Accessibility Center (www.ibm.com/able) for more information about the commitment that IBM has to accessibility. © Copyright IBM Corp. 2023, 2025...
Page 146 130 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 147: Glossary
Glossary This glossary provides terms and definitions for the IBM Storage Scale System 6000 solution. The following cross-references are used in this glossary: • See refers you from a non-preferred term to the preferred term or from an abbreviation to the spelled- out form.
Page 148 A data structure containing key information about a file system. This information includes the disks assigned to the file system (stripe group), the current state of the file system, and pointers to key files such as quota files and log files. 132 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 149 Hardware Management Console (HMC) Standard interface for configuring and operating partitioned (LPAR) and SMP systems. See Hardware Management Console (HMC). IBM Security Key Lifecycle Manager (ISKLM) For GPFS encryption, the ISKLM is used as an RKM server to store MEKs. independent fileset A fileset that has its own inode space.
Page 150 From a system management perspective, it is the central coordinator of the cluster. It also serves as a client node in an ESS building block. master encryption key (MEK) A key that is used to encrypt other keys. See also encryption key. 134 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 151 maximum transmission unit (MTU) The largest packet or frame, specified in octets (eight-bit bytes), that can be sent in a packet- or frame-based network, such as the Internet. The TCP uses the MTU to determine the maximum size of each packet in any transmission. See master encryption key (MEK).
Page 152 A network protocol that ensures a loop-free topology for any bridged Ethernet local-area network. The basic function of STP is to prevent bridge loops and the broadcast radiation that results from them. 136 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 153 See secure shell (SSH). See Spanning Tree Protocol (STP). symmetric multiprocessing (SMP) A computer architecture that provides fast performance by making multiple processors available to complete individual processes simultaneously. See Transmission Control Protocol (TCP). Transmission Control Protocol (TCP) A core protocol of the Internet Protocol Suite that provides reliable, ordered, and error-checked delivery of a stream of octets between applications running on hosts communicating over an IP network.
Page 154 138 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 155: Index
23 MTM 5149-S54 73, 87, 107, 111, 112, 114–116, 118, 120, weight and dimensions 24 124–126 IBM Storage Scale Systems 73, 87, 91–95, 99, 100, MTM 5149-S54 air temperature 115 106–108, 110–112, 114–116, 118, 120, 124–126 MTM 5149-S54 components 112...
Page 156 23 preface xv requirements additional space 27 drives 28 electrical 23 power 23 space 24 shock specifications 29 site planning 32 site preparation 32 space requirements 140 IBM Storage Scale System 6000: Hardware Planning and Installation Guide...
Page 158 IBM® Product Number: 5765-DME 5765-DAE SC28-3485-03...

IBM Storage Scale System 6000 Hardware Planning And Installation Manual

Quick Links

Need help?

Questions and answers

Subscribe to Our Youtube Channel

Related Manuals for IBM Storage Scale System 6000

Summary of Contents for IBM Storage Scale System 6000

Table of Contents