Nvidia Spectrum-4 SN5000 Hardware User Manual

2u switch systems

Hide thumbs Also See for Spectrum-4 SN5000:

Hardware user manual (45 pages)

Hardware user manual (68 pages)

Hardware user manual (66 pages)

page of 56

/ 56
Bookmarks

Quick Links

Download this manual

NVIDIA Spectrum-4 SN5000 2U Switch

Systems Hardware User Manual

Rev. 1.0

Need help?

Do you have a question about the Spectrum-4 SN5000 and is the answer not in the manual?

Questions and answers

Summary of Contents for Nvidia Spectrum-4 SN5000

Page 1 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual Rev. 1.0...
Page 2 Table of Contents About this Manual..............5 Related Documentation .............. 6 Revision History ............... 7 Ordering Information ..............8 SN5400 Part Numbers ..............8 SN5600 Part Numbers ..............8 SN5600D Part Numbers ..............8 SN5610 Part Numbers ..............8 Introduction................9 Speed and Switching Capabilities .............
Page 3 System Bring-Up................32 6.9.1 Operating Systems Support............32 6.9.1.1 Configuring Network Attributes Using Cumulus Linux ....... 33 6.9.1.2 Configuring Network Attributes Using SONiC ......... 33 6.10 FRU Replacements ............... 33 6.10.1 Power Supplies ................33 6.10.2 Fans ..................34 Interfaces ................36 RS232 (Console) ................36 Management ................
Page 4 11.3 Interface Specifications..............52 11.3.1 Small Form Factors Specifications ..........52 11.3.2 USB Standard Specifications............52 11.3.3 RJ-45 to DB-9 Harness Pinout ............52 11.4 Disassembly and Disposal ............... 53 11.4.1 Disassembly Procedure ...............53 11.4.2 Disposal.................53 11.4.2.1 Lithium Battery ..............53 Document Revision History ............54...
Page 5 This manual is relevant for the following models: SN5400, SN5600, SN5600D and SN5610. It describes the installation and basic use of NVIDIA Ethernet switches based on the NVIDIA Spectrum-4 ASIC. Its intended audience is IT managers and system administrators. Refer to Ordering Information product descriptions and part numbers.
Page 6 Cumulus Linux User Guide training course • SONiC User Manual and release notes: log in to the NVIDIA Enterprise Support Portal → Downloads → Switches and Gateways → Switch Software → SONiC → Documentation • Open Network Install Environment (ONIE) Quick Start Guide •...
Page 7 3 Revision History A list of the changes made to this document can be found in the Document Revision History.
Page 8 Product Description Lifecy Phase 920-9N42C-0 NVIDIA Spectrum-4 based 400GbE 2U Open Ethernet switch with Cumulus Linux P-Rel 0RB-7C0 authentication, 64 QSFP56-DD ports and 1 2xSFP28 port, 2 power supplies (AC), x86 CPU, Secure Boot, standard depth, C2P airflow, tool-less rail kit 4.2 SN5600 Part Numbers...
Page 9 5 Introduction The NVIDIA Spectrum SN5000 series switches are the 5th generation of Spectrum switches, purpose- built to accelerate data center fabrics. The SN5000 series provides port speeds spanning from 10GbE to 800GbE, and delivers accelerated Ethernet to every data center. The SN5000 series is ideal for enabling cloud-scale infrastructure for data centers of any size.
Page 10 SN5600 Front View SN5600D Front View SN5610 Front View SN5400 and SN5600 Rear View SN5600D Rear View SN5610 Rear View...
Page 11 5.1 Speed and Switching Capabilities The following table describes maximum throughput and interface speed per system model. The systems support different interfaces and speed rates using QSFP/QSDP-DD/OSFP to SFP adapters or breakout cables. For further information, see Splitter (Breakout) Cables and Adapters.
Page 12 5.4 Certifications For a list of certifications (such as EMC, safety, and others), contact your NVIDIA representative. ...
Page 13 Unless otherwise specified, NVIDIA products are designed to work in an environmentally controlled data center with low levels of gaseous and dust (particulate) contamination. The operation environment should meet severity level G1 as per ISA 71.04 for gaseous...
Page 14 Connector (front) side inlet to power side outlet: marked with red dots that are placed on the power inlet side. All servers and systems in the same rack should have the same airflow direction. Likewise, all FRU components must also have the same airflow direction. A mismatch in the airflow will affect the heat dissipation.
Page 15 For installation instructions, refer to the following table: System Model Rail Kit SN5400/SN5600/SN5610 SN5xxx Rail Kit SN5600D SN5600D Rail Kit 6.5 SN5xxx Rail Kit Kit Part Number Rack Size and Rack Depth Range 930-9SKIT-00L0-00A NVIDIA 600mm to 800mm tool-less rail kit for 64×400G and 64×800G switch...
Page 16 The illustrations in this section depict the SN5400 system. You can follow the same installation steps for other systems, but the physical design may vary slightly. 6.5.1 Rail Kit Parts Item Quantity Item Rail kit for 2× (of each part) 23.6-31.5”...
Page 17 6.5.2 Installation Instructions • The following steps include illustrations that show front side (ports) installation, yet all instructions apply to all installation options. • The illustrations show the SN5400 system, yet the installation procedures apply to SN5600 systems as well. Attach the left and right system rails (A) to the switch.
Page 18 4. Align both rack rails (B) to sit horizontally in parallel to the rack assembly. By straightening the rails' angular position, their breaks will be caught and locked in the rack's slots. 5. Pull the rack rails' telescopic extensions all the way to the rack's opposite side, and insert the latches at the rails' free edges to the rack's slots.
Page 19 6. While your installation partner is supporting the system’s weight, slide the rails installed on the system into the channels in the rack rails, and push them forward until the locking mechanism is activated on both sides and a click is heard. 7.
Page 21 6.5.2.1 Removing the Device from the Rack To remove the system from the rack: Turn off the system and disconnect it from peripherals and from the electrical outlet. While your installation partner is supporting the system’s weight, loosen the captive screws attaching the system's rails to the rack's posts. ...
Page 22 6. Remove the rails from the rack by pressing the lock button, and pull the rails outside of the rack assembly. 6.6 SN5600D Rail Kit The following instructions have been prepared and provided with permission from King Slide. 6.6.1 Package Contents ...
Page 23 6.6.2 Installation Instructions...
Page 25 6.6.3 Troubleshooting...
Page 26 6.7 Cable Installation All cables can be inserted or removed with the unit powered on. To insert a cable, press the connector into the port receptacle until the connector is firmly seated. The LED indicator that corresponds to its respective data port will turn orange when the physical connection is established.
Page 27 Do not push the cable connector into the receptacle using excessive force; this may damage the cable or the cage. QSFP Cable Orientation 6.7.1 Splitter (Breakout) Cables and Adapters When using an NVIDIA splitter cable, the following splitting options are available: • SN5400 (see Splitting Options): •...
Page 28 Examples of Splitter (Breakout or Fanout) Cables 6.7.1.2 Splitting Options The following diagram shows the systems' ports non-blocking splitting capability to 2 and 4 ports respectively. In addition, each of the ports can be split to 8, while disabling the adjacent port. For example, when splitting port no.
Page 29 The OSFP-based optical transceivers have two distinctive ports that can be used as OSFP transceivers inserted to the switch in a "belly-to-belly" configuration, meaning that the transceiver on the top will be right side up, and the transceiver below it will be upside down. OSFP Transceiver The following illustration depicts the numbering of the ports in 2x400G and 4x200G modes.
Page 30 Connector: 800G Port: Port Configuration Physical Connector NOS Description Examples 800G swp1, swp2 1. These descriptions may vary between network operating systems ...
Page 31 6.7.2.1 Breakout Ports Reflection in NOS The illustration below shows the first two rows of the system’s OSFP carriers: the image on the left identifies the OSFP cage number as it is presented by the OS; the image on the right identifies the corresponding subinterfaces. Connector # Connector # 2×400G 4×200G Port Configuration Physical Connector...
Page 32 Wait for the System Status LED to turn green. If after five minutes the System Status LED is still amber, unplug the system and contact your NVIDIA representative for assistance. Check the System Status LEDs and confirm that all of the LEDs show status lights consistent with normal operation as shown in the figure below.
Page 33 Parts. 6.10.1 Power Supplies NVIDIA systems with replaceable power supply units work in a redundant configuration. Either unit may be extracted without bringing down the system. If you are replacing one of the PSUs, make sure that the power supply unit LED for the unit you are NOT replacing displays a green status while the system is up and running.
Page 34 Remove the power cord from the power supply unit. Grasp the handle and push the latch release with your thumb while pulling the handle outward. As the power supply unit unseats, the power supply unit status LEDs will turn off. Remove the power supply unit.
Page 35 After two unsuccessful attempts to install the fan unit, power off the system before attempting any system debug. Each fan module contains 2 individual rotors. If one fan module becomes faulty or defective, NVIDIA recommends replacing it immediately to ensure continued redundancy. This is particular important when the system is operating at 35°C ambient temperature at full capacity with all ports occupied.
Page 36 7 Interfaces The systems support the following interfaces: • 10/100/1000Mb Ethernet management interface (RJ45) • USB port (Type A) • RS232 console port (RJ45) • Reset button • Status and port LEDs • Lane select LEDs and power button Refer to Management Interfaces, PSUs, and Fans for full configuration options.
Page 37 7.3 USB The USB interface is USB 3.0-compliant and can be used by the operating system to connect to an external disk for software upgrade or file management. The connector complies with the USB 3.0 type A standard. To view the full matrix of the USB configuration options, refer to Management Interfaces, PSUs, and Fans.
Page 38 Model Family Ports Maximum Speed 10GbE SN5600/SN5600D/SN5610 800GbE (100Gbps/lane) 400GbE 200GbE 100GbE 50GbE 25GbE 10GbE 7.6.2 High Power Transceivers Support The systems offer several high-power ports for optical transceivers that require such support. The following table specifies each system's ports max power capabilities: Model Family Ports Maximum High Power Support...
Page 39 Front (Ports Side): It can take up to five minutes for the booting process to complete. If the System Status LED displays amber after five minutes, unplug the system and contact your NVIDIA representative for assistance. System Status LED Descriptions...
Page 40 Fan Status Front LED Assignments Description Action Required Solid green All fans are up and running Solid amber One or more fans are not operating Replace the faulty FRUs properly Fan Status Rear LED Assignments (One LED per Fan) Description Action Required Solid green A specific fan unit is fully operational...
Page 41 • The SN5400 and SN5600/SN5600D switch models have two power supply inlets each. Each system can operate with only one power supply connected. • The SN5610 switch model has four power supply inlets for redundancy. The system can operate when at least two power supplies are connected. Each power supply unit has two single-color LEDs on the right side of the unit that indicate the status of the unit.
Page 42 The PSU is running normally. Flashing AC Present, Standby (On), Main Output Refer to Troubleshooting. For further assistance, green 1Hz (Off) contact your NVIDIA representative. Flashing PSU warning - events where the PSU amber 1Hz continues to operate Solid amber PSU failure (voltage, current, temperature, or fan issue) No AC power to all power supplies.
Page 43 Each QSFP-DD/OSFP module can be used as two 4× ports or four 2× ports; some modules can be split into eight 1× ports. Each QSFP-DD/OSFP module has one dedicated bi-color LED. The LED splitting control button can be used to display link information for more than one port using only one LED. The lane select button lets you choose between 8 indication states: press the button to cycle through all 8 states.
Page 44 State State Indication OSFP/QSFP-DD Module Comments LEDs LED Indication [/1 /2 /3 /4] 8×/4×A/2×A/1×A • See details in the Port LEDs Indications table below, states 1-8. • Only one of the link types can be up at a given time. 8×/4×B/2×B/1×B 8×/2×C/1×C 8×/2×D/1×D...
Page 45 State Description Action Required Behavior Flashing amber Link is down Solid green Link is up with no traffic Flashing Link is up with traffic green Flashing A problem with the link Refer amber to Troubleshooting 7.8 Inventory Information The system’s inventory information (such as serial number, part number, and GUID address) can be extracted from the inventory pull-out tab on the right side of the rear panel.
Page 46 Upgrading Cumulus Linux. ▪ For SONiC upgrade instructions, log in to the NVIDIA Enterprise Support Portal, and go to Downloads → Switches and Gateways → Switch Software → SONiC → Documentation. 8.2 Switch Firmware Update The switch systems do not require firmware updates. Firmware updates are handled through the...
Page 47 9 Troubleshooting Problem Symptoms Cause and Solution Indicator LEDs System Status LED is Cause: The operating system did not boot properly, and only blinking for more firmware is running. than 5 minutes Solution: Connect to the system via the console port, and check the software status.
Page 48 Typical power with passive cables (ATIS): 670W Consumption: Main CPU: Intel x86 Xeon, Hexa-core Coffee Lake E-2276ME w/secured-boot Devices PCIe: Gen3 x4 Switch: NVIDIA Spectrum®-4 Memory: 32GB RAM DDR4, 160GB SSD Throughput 25.6 Tbps 10.1 SN5600 Specifications Feature Value Mechanical Size: ...
Page 49 Typical power with passive cables (ATIS): 940W Consumption: Main CPU: Intel x86 Xeon, Hexa-core Coffee Lake E-2276ME w/secured-boot Devices PCIe: Gen3 x4 Switch: NVIDIA Spectrum®-4 Memory: 32GB RAM DDR4, 160GB SSD Throughput 51.2 Tbps 10.2 SN5600D Specifications Feature Value Mechanical Size: ...
Page 50 Typical power with passive cables (ATIS): 940W Consumption: Main CPU: Intel x86 Xeon, Hexa-core Coffee Lake E-2276ME w/secured-boot Devices PCIe: Gen3 x4 Switch: NVIDIA Spectrum®-4 Memory: 32GB RAM DDR4, 160GB SSD Throughput 51.2 Tbps 10.3 SN5610 Specifications Feature Value Mechanical Size: ...
Page 51 64×800G switch 930-9SFAN-00R NVIDIA fan unit, C2P airflow SN5610 M-00F Power supplies 930-9SPSU-00R NVIDIA power supply unit, 3KW AC, C2P airflow, for SN5400, SN5600 A-00B 64×800G switch, power cord included 930-9SPSU-00F NVIDIA power supply unit, 3KW AC, P2C airflow, for...
Page 52 USB Document Library. 11.3.3 RJ-45 to DB-9 Harness Pinout NVIDIA supplies an RS232 harness cable (DB-9 to RJ-45) to connect a host PC to the console RJ-45 system port. The RS-232 default baud rate is 115200 bps with an 8N1 configuration.
Page 53 11.4 Disassembly and Disposal 11.4.1 Disassembly Procedure To disassemble the system from the rack: Unplug and remove all connectors. Unplug all power cords. Remove the ground wire. Unscrew the center bolts from the side of the system with the bracket. Support the weight of the system when you remove the screws so that the system does not fall.
Page 54 12 Document Revision History Date Revisi Description April 2025 • Added the SN5610 system • Added the SN5600D system December 2024 • Added new airflow option for the SN5400 switch • Replaced fan module image • Made editorial and stylistic revisions November 2024 •...
Page 55 NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs.
Page 56 U.S. and in other countries. Other company and product names may be trademarks of the respective companies with which they are associated. Copyright © 2025 NVIDIA Corporation & affiliates. All Rights Reserved.

This manual is also suitable for:

Spectrum-4 sn5400Spectrum-4 sn5600Spectrum-4 sn5600dSpectrum-4 sn5610