Related Manuals for Nvidia Spectrum-4 SN5000 2U
Summary of Contents for Nvidia Spectrum-4 SN5000 2U
- Page 1 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual Rev. 1.0 Exported on Nov/02/2023 01:58 PM...
-
Page 2: Table Of Contents
Table of Contents Ordering Information ................5 SN5400 Ordering Part Numbers ..............5 SN5600 Ordering Part Numbers ..............5 Introduction..................6 Speed and Switching Capabilities ..............7 Management Interfaces, PSUs and Fans ............7 Features ....................8 Certifications ..................8 Installation ..................9 System Installation and Initialization .............9 Safety Warnings ..................9 Air Flow....................9 Package Contents .................. - Page 3 System Status LED ................29 Fan Status LED.................. 29 Power Supply Status LEDs..............30 Unit Identification LED................ 32 Port LEDs ..................32 Inventory Information ................34 Software Management ...............36 Software Upgrade.................. 36 Cumulus Linux Software Upgrade ............36 SONiC Software Upgrade..............36 Switch Firmware Update..............
- Page 4 Relevant for Models: SN5400 and SN5600. About this Manual This manual describes the installation and basic use of NVIDIA Ethernet switches based on the NVIDIA® Spectrum-4 ASIC. Ordering Information See Ordering Information. Intended Audience This manual is intended for IT managers and system administrators.
-
Page 5: Ordering Information
SN5400 Ordering Part Numbers NVIDIA Product Description Lifecyc Phase 920-9N4 NVIDIA Spectrum-4 based 400GbE 2U Open Ethernet switch with Cumulus Linux Enginee 2C-00RB Authentication, 64 QSFP56-DD ports and 1 2xSFP28 port, 2 power supplies (AC), x86 CPU, ring -7C0 Secure-boot, standard depth, C2P airflow, Tool-less Rail Kit samples ... -
Page 6: Introduction
NVIDIA EGXTM, NVIDIA DGX PODTM , and NVIDIA OVXTM SuperPODs, and AI solution stacks such as NVIDIA AI Enterprise and NVIDIA LaunchPad. As part of the Spectrum platform, the SN5000 systems are pre-tested and pre-validated with NVIDIA’s full portfolio of Ethernet networking technology, including BlueField DPUs, ConnectX SmartNICs, and LinkX interconnects. -
Page 7: Speed And Switching Capabilities
SN5600 Front View SN5400 and SN5600 Rear View Speed and Switching Capabilities The table below describes maximum throughput and interface speed per system model. System Interfaces Supported Rates Max Throughput Model SN5400 64 QSFP-DD 400GbE 64 x 400GbE 25.6Tb/s 2xSFP28 25GbE 128 x 200GbE 256 x 10/25/50/100GbE ... -
Page 8: Features
In the main menu, click on Products > Ethernet Switch Systems, and select the desired product family. Certifications For a list of certifications (such as EMC, Safety and others) per system for different regions of the world, please contact your NVIDIA representative. ... -
Page 9: Installation
Unless otherwise specified, NVIDIA products are designed to work in an environmentally controlled data center with low levels of gaseous and dust (particulate) contamination. The operation environment should meet severity level G1 as per ISA 71.04 for gaseous contamination and ISO 14644-1 class 8 for cleanliness level. -
Page 10: Package Contents
the images are provided for illustration purposes only. The design may slightly vary in different systems • Power (rear) side inlet to connector side outlet • Connector (front) side inlet to power side - marked with blue dots that are placed on the outlet - marked with red dots that are placed power inlet side. -
Page 11: Mounting Options
Prior to the installation procedure, inspect all rail-kit components and make sure none of them is missing or damaged. If anything is missing or damaged, contact your NVIDIA representative at Networking-support@nvidia.com. The following parts are included in the tool-less rail kit (see figure below): •... - Page 12 Prerequisites: Before mounting the system to the rack, select the way you wish to place the system. Pay attention to the airflow within the rack cooling, connector and cabling options. While planning how to place the system, review the two installation options shown in the table below, and consider the following points: •...
- Page 13 The following steps include illustrations that show front side (ports) installation, yet all instructions apply to all installation options. Attach the left and right system rails (A) to the switch. Attaching the System Rails (A) to the Switch Secure the assembly by gently pushing the system chassis’ pins through the slider key holes, until locking occurs...
- Page 14 Securing the System in the Switch Rails (A) Chassis' Pins in the Rails' Slots Locking them in a Fixed Position Mount both of the rack rails (B) into the rack by angularly inserting the brakes located at the rails edges into the designated slots in the rack unit, as shown in the following figure: ...
- Page 15 Inserting the Rack Rails (B) Align both rack rails (B) to sit horizontally in parallel to the rack assembly. By straightening the rails' angular position, their breaks will be caught and locked in the rack's slots. Aligning the Rack Rails (B) Angular Position The Breaks are Caught and Locked in the Rack's Slots...
- Page 16 Rack Rails Fully Inserted and Locked in the Rack Assembly Pull the rack rails' telescopic extensions all the way to the rack's opposite side, and insert the latches at the rails' free edges to the rack's slots. A click should be heard as the spring latches are fully inserted and locking occurs.
- Page 17 Slide the rails installed on the system into the channels in the rack rails. Push them forward until the locking mechanism is activated on both sides, and a click is heard. Tighten the captive screws on both sides to further secure the system to the rack's posts. Sliding the System's Rails (A) into the Rack Rails (B) Sliding the System Rails (A) into the Rack Rails Tightening the Captive Screws...
- Page 18 Pulling the System Out Press the spring latches on both sides of the rack, and continue to pull the system out until the rack rails are clear of the system's rails. Pressing the Spring Latches on Both Sides Remove the rails from the system. Release the metal latches and pull out the rails, so the system's pins will be removed out of the oval slots.
-
Page 19: Cable Installation
Removing the Rails from the System Remove the rails from the rack by pressing the lock button, and pull the rails outside of the rack assembly. Pressing the Lock Button to Remove the Rails from the Rack Cable Installation All cables can be inserted or removed with the unit powered on. -
Page 20: Splitter (Breakout) Cables And Adapters
For a list of Supported Cables and Transceivers, please refer to the SN5000 Systems Datasheet. For full cabling guidelines, ask your NVIDIA representative for a copy of NVIDIA Cable Management Guidelines and FAQs Application Note. For more information about port LEDs, refer to Port LEDs. - Page 21 Using Splitter (Breakout) Cables with Cumulus Linux If you are using splitter cables, edit the to enable support for these /etc/cumulus/ports.conf cables, and restart the switchd service using the command. For sudo systemctl restart switchd more details, see Switch Port Attributes in the Cumulus Linux User Guide.
-
Page 22: Initial Power On
If the Fan Status LED is not green, unplug the power connection and check that the fan module is inserted properly and that the mating connector of the fan unit is free of any dirt and/or obstacles. If no obstacles were found and the problem persists, call your NVIDIA representative for assistance. -
Page 23: System Bring-Up
For a list of the FRU replacements, see “Accessory and Replacement Parts”. Power Supplies NVIDIA systems that are equipped with two replaceable power supply units work in a redundant configuration. Either unit may be extracted without bringing down the system. -
Page 24: Fans
Make sure that the power supply unit that you are NOT replacing is showing green for the power supply unit LED without powering down the system. Power supply units have directional air flows similar to the fan module. The fan module airflow must coincide with the airflow of all of the power supply units (there are different power supply Part Numbers to indicate the different airflow types). - Page 25 In case the system is operated at 35°C ambient temperature at full capacity with all ports occupied, and one of the system fans becomes faulty, it is recommended to replace the fan as soon as possible to avoid the risk of having no fan redundancy. To remove a fan unit: ...
-
Page 26: Interfaces
Interfaces This document is preliminary and subject to change. The systems support the following interfaces: • 10/100/1000Mb Ethernet management interface (RJ45) • USB port (Type A) • RS232 Console port (RJ45) • Reset button • Status and Port LEDs •... -
Page 27: Rs232 (Console)
Model Family Ports Maximum High Power Support Ports #1,5,9,13,17,21,25,29,33,37,41,45,4 9,53,57,61 SN5600 with Dual 3KW PSU Port #1-64 Speed Ethernet speed must be set manually. The system’s ports can be manually configured to run at a wide range of speeds (for more details, see Specifications). To change the port speed configuration, refer to the Switch Port Attributes chapter in the... -
Page 28: Reset Button
USB 1.0 is not supported. Do not use excessive force when inserting or extracting the USB disk to and from the connector. Reset Button The reset button is located on the front side of the system. This reset button requires a tool to be pressed. -
Page 29: System Status Led
*The figure is provided for Illustration purposes only. The design may slightly vary in different systems. It may take up to five minutes to boot the system. If the System Status LED shows amber after five minutes, unplug the system and call your NVIDIA representative for assistance. System Status LED Assignments LED Behavior... -
Page 30: Power Supply Status Leds
Rear Panel LED: Fan Status Front LED Assignments LED Behavior Description Action Required Solid Green All fans are up and running. Solid Amber Error, one or more fans are not operating The faulty FRUs should be replaced. properly. Fan Status Rear LED Assignments (One LED per Fan) LED Behavior Description Action Required... - Page 31 Rear Panel LED: In the SN5400 and SN5600 switches there are two power supply inlets in each system (for redundancy). The system can operate with only one power supply connected. Each power supply unit has two single color LEDs on the right side of the unit, that indicate the status of the unit. When looking from the FRUs side, the first power supply (PS) unit is located on the left side of the system, and the second unit is located on the right side.
-
Page 32: Unit Identification Led
Behavior Flashing AC Present, Standby – On, Main Output - Refer to Troubleshooting. For further assistance, Green 1Hz call your NVIDIA representative. Flashing PSU warning - events where the PSU Amber 1Hz continues to operate Solid Amber PSU failure (voltage, current, temperature or fan related issue) No AC power to all power supplies. - Page 33 Each QSFP-DD/OSFP module can be used as two 4X ports or four 2X ports, and certain modules can be split to eight 1X ports as well. Each QSFP-DD/OSFP module has one dedicated bi-color LED. In order to provide link information for more than one port by using one LED, LED splitting control button is available.
-
Page 34: Inventory Information
State State Indication OSFP/QSFP-DD Module Comments LEDs LED Indication [/1 /2 /3 /4] 8x/1xF 8x/1xG 8x/1xH The port LED behavior indicates the port state, as follows: Port LEDs Indications Description Action Behavior Required No 8x/4x/2x/1x link was established on this OSFP/QSFP-DD module Solid Green At list one link was established: 8x/4x/2xA/2xB/1xA/1xB/1xC/1xD/1xE/ 1xF/1xG/1xH... - Page 35 SN5400 and SN5600 *The above label example is provided for illustration purposes only. ...
-
Page 36: Software Management
The SN5000 series switches are available out of the factory in two different flavors: • Pre-installed with NVIDIA Cumulus Linux, that is providing a rich routing functionality for large-scale applications. For Cumulus Linux software management instructions, refer to the Cumulus Linux User Guide. -
Page 37: Troubleshooting
Troubleshooting Problem Symptoms Cause and Solution Indicator LEDs System Status LED is Cause: The operating system did not boot properly, and only blinking for more firmware is running. than 5 minutes Solution: Connect to the system via the console port, and check the software status. -
Page 38: Specifications
Typical power with passive cables (ATIS): TBD Consumption: Main CPU: Intel x86 Xeon, Hexa-core Coffee Lake E-2276ME w/ secured-boot Devices PCIe: Gen3 x4 Switch: NVIDIA Spectrum®-4 Memory: 32GB RAM DDR4, 160GB SSD Throughput 25.6Tbps SN5600 Specifications Feature Value Mechanical Size: ... - Page 39 AC current rating 16A 50/60Hz Global Power Typical power with passive cables (ATIS): 940W Consumption: Main CPU: Intel x86 Xeon, Hexa-core Coffee Lake E-2276ME w/ secured-boot Devices PCIe: Gen3 x4 Switch: NVIDIA Spectrum®-4 Memory: 32GB RAM DDR4, 160GB SSD Throughput 51.2Tb/s...
-
Page 40: Appendixes
Ordering Part Numbers for Replacement Parts Part Type Part Number Description Supported Systems Rack Installation 930-9SKIT-00L0 NVIDIA Tool-Less Rail-Kit for 64x400G and 64x800G Switch SN5400, SN5600 -00A Fan Modules 930-9SFAN-00R NVIDIA FAN Unit, C2P Airflow, for 64x400G and 64x800G SN5400, SN5600 M-00A switch... -
Page 41: Interface Specifications
Networks Help Center. Interface Specifications Small Form Factors Specifications NVIDIA switch systems come in a flexible range of form factors - SFP/QSFP, SFP28/QSFP28, SFP56/ QSFP56, SFP-DD/QSFP-DD and OSFP. All form factors specification documents are available on the Storage Networking Industry Association (SNIA) Website. ... -
Page 42: Disposal
Unplug and remove all connectors. Unplug all power cords. Remove the ground wire. Unscrew the center bolts from the side of the system with the bracket. Support the weight of the system when you remove the screws so that the system does not fall. -
Page 43: Document Revision History
Document Revision History Date Revisio Description April 2023 First release... - Page 44 NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs.
- Page 45 Copyright © 2023 NVIDIA Corporation & affiliates. All Rights Reserved.