Nvidia Spectrum-4 Hardware User Manual
  1. Manuals
  2. Brands
  3. Nvidia Manuals
  4. Switch
  5. Spectrum-4 SN5000 2U
  6. Hardware user manual

Nvidia Spectrum-4 Hardware User Manual

2u switch systems
Hide thumbs Also See for Spectrum-4:
Table of Contents

Advertisement

Quick Links

Download this manual
NVIDIA Spectrum-4 SN5000 2U Switch Systems
Hardware User Manual

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the Spectrum-4 and is the answer not in the manual?

Questions and answers

Related Manuals for Nvidia Spectrum-4

Summary of Contents for Nvidia Spectrum-4

  • Page 1 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 2: Table Of Contents

    Appendixes Accessory and Replacement Parts Accessory and Replacement Parts Thermal Threshold Definitions Thermal Threshold Definitions Interface Specifications Interface Specifications Disassembly and Disposal Disassembly and Disposal Document Revision History Document Revision History NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 3 1709076250100 Api V2 Figure 9. Image 2024 2 28 11 3 24 Version 1 Modificationdate 1709111004723 Api V2 Figure 10. Image 2024 2 28 11 4 40 Version 1 Modificationdate 1709111080667 Api V2 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 4 1681669252767 Api V2 Figure 21. Image2023 3 13 9 55 4 Version 1 Modificationdate 1681669256813 Api V2 Figure 22. Image2023 3 15 12 50 47 Version 1 Modificationdate 1681669257097 Api V2 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 5 1681669252977 Api V2 Figure 33. Image2019 3 5 17 35 21 Version 1 Modificationdate 1681669253147 Api V2 Figure 34. Image2019 3 5 17 36 30 Version 1 Modificationdate 1681669252923 Api V2 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 6 1681669252923 Api V2 Figure 44. Image2023 8 2 8 46 16 Version 1 Modificationdate 1690991177847 Api V2 Figure 45. Image2018 11 14 14 50 31 Version 1 Modificationdate 1693421586840 Api V2 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 7 Relevant for Models: SN5400 and SN5600. About this Manual This manual describes the installation and basic use of NVIDIA Ethernet switches based on the NVIDIA® Spectrum-4 ASIC. Ordering Information See Ordering Information. Intended Audience This manual is intended for IT managers and system administrators.
  • Page 8 Revision History A list of the changes made to this document are provided in Document Revision History. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 9: Ordering Information

    Lifecy NVIDI Product Description A SKU Phase 920- NVIDIA Spectrum-4 based 400GbE 2U Open Ethernet switch with Engin 9N42 Cumulus Linux Authentication, 64 QSFP56-DD ports and 1 2xSFP28 port, eering 2 power supplies (AC), x86 CPU, Secure-boot, standard depth, C2P...
  • Page 10 00RI- power supplies (AC), x86 CPU, Secure-boot, standard depth, C2P airflow, Tool-less Rail Kit NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 11: Introduction

    The SN5000 is built to accelerate NVIDIA platforms, including NVIDIA EGXTM, NVIDIA DGX PODTM , and NVIDIA OVXTM SuperPODs, and AI solution stacks such as NVIDIA AI Enterprise and NVIDIA LaunchPad. As part of the Spectrum platform, the SN5000 systems are pre-tested and pre-validated with NVIDIA’s full portfolio of Ethernet networking technology, including...
  • Page 12 The table below describes maximum throughput and interface speed per system model. System Interfaces Supported Rates Max Throughput Model SN5400 64 QSFP-DD 64 x 400GbE 25.6Tb/s 400GbE 128 x 200GbE 2xSFP28 25GbE 256 x 10/25/50/100GbE NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 13 The table below lists the various management interfaces, PSUs and fans per system model. RJ45 MGT RJ45 System Model USB-A (Management) Console SN5400 Front Front Front Yes, 2 Yes, 4 SN5600 Front Front Front Yes, 2 Yes, 4 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 14 Ethernet Switch Systems, and select the desired product family. Certifications For a list of certifications (such as EMC, Safety and others) per system for different regions of the world, please contact your NVIDIA representative. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 15: Installation

    Take precautions to guarantee proper ventilation in order to maintain good airflow at ambient temperature. Warning Unless otherwise specified, NVIDIA products are designed to work in an environmentally controlled data center with low levels of gaseous and dust (particulate) contamination.
  • Page 16 NVIDIA systems are offered with two air flow patterns: Warning The SN5400 and SN5600 systems are currently offered with one airflow type only - Connector side inlet to power side outlet. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 17 The table below provides an air flow color legend and respective OPN designation. Direction Description and OPN Designation Connector side inlet to power side outlet. Red dots are placed on the power inlet side. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 18 2 x Power cables for each power supply unit – Type C20 to C19 1 x Harness: HAR000028 – Harness RS232 2M cable – DB9 to RJ-45 2 x Cable retainers for each power supply unit Warning NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 19: Sn5400/Sn5600 Rail Kit

    Rack Size and Rack Depth Range 930-9SKIT-00L0- NVIDIA 600-800 mm Tool-Less Rail-Kit for 64x400G and 64x800G Switch Warning The illustrations show the SN5400 system, yet the installation procedures apply to SN5600 systems as well. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 20 Make sure the system air flow is compatible with your installation selection. It is important to keep the airflow within the rack in the same direction. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 21 The following steps include illustrations that show front side (ports) installation, yet all instructions apply to all installation options. 1. Attach the left and right system rails (A) to the switch. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 22 Attaching the System Rails (A) to the Switch 2. Secure the assembly by gently pushing the system chassis’ pins through the slider key holes, until locking occurs NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 23 3. Mount both of the rack rails (B) into the rack by angularly inserting the brakes located at the rails edges into the designated slots in the rack unit, as shown in the following figure: NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 24 Aligning the Rack Rails (B) Angular The Breaks are Caught and Locked in Position the Rack's Slots Rack Rails Fully Inserted and Locked in the Rack Assembly NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 25 A click should be heard as the spring latches are fully inserted and locking occurs. Inserting the Spring Latches to the Rack's Slots To mount the system into the rack: Important NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 26 1. Turn off the system and disconnect it from peripherals and from the electrical outlet. While your installation partner is supporting the system’s weight: 2. Loosen the captive screws attaching the system's rails to the rack's posts. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 27 Pressing the Spring Latches on Both Sides 5. Remove the rails from the system. Release the metal latches and pull out the rails, so the system's pins will be removed out of the oval slots. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 28: Cable Installation

    6. Remove the rails from the rack by pressing the lock button, and pull the rails outside of the rack assembly. Pressing the Lock Button to Remove the Rails from the Rack Cable Installation All cables can be inserted or removed with the unit powered on. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 29 For a list of Supported Cables and Transceivers, please refer to the SN5000 Systems Datasheet. For full cabling guidelines, ask your NVIDIA representative for a copy of NVIDIA Cable Management Guidelines and FAQs Application Note. For more information about port LEDs, refer to Port LEDs.
  • Page 30 For sudo systemctl restart switchd more details, see Switch Port Attributes in the Cumulus Linux User Guide. Examples of Splitter (Breakout or Fanout) Cables NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 31 “Belly-to-Belly" configuration, meaning that the transceiver on the top will be "right-side-up“, and the transceiver below it will be "right- side-down". OSFP Transceiver Illustration NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 32 The illustration below shows the first two rows of the system’s OSFP carriers , where the left part identifies the OSFP cage number , as it is presented by the OS, and the right part identifies the corresponding subinterafces. Connector # Connector # 2x400G 4x200G NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 33: Initial Power On

    8x100G swp55s5, swp55s6, swp55s7, s5, s6, s7, s8 swp55s8 Initial Power On Each system’s input voltage is specified in the Specifications chapter. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 34 4. Check the System Status LEDs and confirm that all of the LEDs show status lights consistent with normal operation as shown in the figure below. For more information, refer to “LEDs”. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 35 If no obstacles were found and the problem persists, call your NVIDIA representative for assistance. Two Power Inlets - Electric Caution Notifications: Important Risk of electric shock and energy hazard.

  • Page 36: System Bring-Up

    Remote Access" page in the Cumulus Linux User Guide. Configuring Network Attributes Using SONiC For initial configuration instructions using SONiC, see Configuring the Switch for the First Time in the SONiC User Manual available on the NVIDIA Enterprise Support Portal. FRU Replacements Warning For a list of the FRU replacements, see “Accessory and Replacement...
  • Page 37 Power Supplies NVIDIA systems that are equipped with two replaceable power supply units work in a redundant configuration. Either unit may be extracted without bringing down the system. Important Make sure that the power supply unit that you are NOT replacing is showing green for the power supply unit LED without powering down the system.
  • Page 38 The system has 3+1 fan module redundancy. Full operation is possible with 3 fans only. Important Make sure that the fans configuration matches the model number. Operation with incorrect fan configuration can cause over- NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 39 1. Grasping the handle and pull the unit outward. As the fan unit unseats, the fan unit status LEDs will turn off. 2. Remove the fan unit. SN5400/SN5600C Fan Module Latches SN5400/SN5600C Fan Module Pulled Out NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 40 Upon first installation, the green Fan Status LED should light. If not, extract the fan unit and reinsert it. After two unsuccessful attempts to install the fan unit, power off the system before attempting any system debug. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 41: Interfaces

    (for more details, see Specifications). To change the port speed configuration, refer to the Switch Port Attributes chapter in the Cumulus Linux User Guide. RS232 (Console) Warning The Console port is labeled images/download/thumbnails/2487194620/image2019-3-5_16-50-5- version-1-modificationdate-1680644493023-api-v2.png . NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 42 The USB interface is USB 3.0 compliant and can be used by the operating system to connect to an external disk for software upgrade or file management. The connector complies with the USB 3.0 type A standard. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 43 Use a flat object to push the reset button. For Cumulus Linux password reset instructions, please refer to the Single User Mode - Password Recovery section in the Cumulus Linux User Guide. Status and Port LEDs See LED Notifications. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 44: Data Interfaces

    100GbE 50GbE 25GbE 10GbE The systems offer several high-power ports for optical transceivers that require such support. The following table specifies each system's ports max power capabilities: High Power Transceivers Support NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 45: Led Notifications

    Lights up on command Off or blue when activated through the CLI. by the user System Status LED The LED in the red oval shows the system’s status. Front (Ports Side): NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 46 Fan Status LED - Front and Rear Sides Both of the LEDs in the red ovals show the fans’ status. *The figures are provided for Illustration purposes only. The design may slightly vary in different systems. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 47 Description Action Required Behavior Solid A specific fan unit is operating. Green Solid A specific fan unit is missing or not operating The fan unit should be Amber properly. replaced. Important NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 48 When looking from the FRUs side, the first power supply (PS) unit is located on the left side of the system, and the second unit is located on the right side. Rear Side Panel NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 49 Refer to Troubleshooting. For further AC Present, Standby – On, Main Green assistance, call your NVIDIA Output - Off representative. Flashing PSU warning - events where the Amber PSU continues to operate NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 50 Module LED Status -------------------------------------------------------------------------- MGMT UID Blues To deactivate the UID LED on a switch system, run: (config) # led MGMT uid off switch Port LEDs SN5400 Port LEDs SN5600 Port LEDs NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 51 The current state can be identified by the LED splitting state indication LEDs. The states and their indications are detailed in the below table. Lane Select LEDs LED Splitting Control LED Splitting Options NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 52 (States 1-8). Only one of the link types can be up at a given time. 8x/1xE 8x/1xF 8x/1xG 8x/1xH The port LED behavior indicates the port state, as follows: Port LEDs Indications NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 53: Inventory Information

    The system’s inventory parameters (such as serial number, part number and GUID address) can be extracted from the inventory pull-out tab on the right side of the rear panel. Pull-out Tab SN5400 and SN5600 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 54 *The above label example is provided for illustration purposes only. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 55: Software Management

    For Cumulus Linux software upgrade instructions, see Upgrading Cumulus Linux in the Cumulus Linux User Guide. SONiC Software Upgrade For SONiC software upgrade instructions, log into the NVIDIA Enterprise Support Portal, and go to Downloads Switches and Gateways Switch Software SONiC Documentation Tab.

  • Page 56: Troubleshooting

    Check that the fan is fully inserted and nothing blocks the airflow Replace the fan FRU if needed Cause: Possible PSU issue Solution: PSU Status LED is Amber Check/replace the power cable Replace the PSU if needed NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 57 Problem Symptoms Cause and Solution Indicator System boot Software failure while upgrade failed See Monitoring and Troubleshooting in Cumulus using Cumulus on x86 based Linux User Guide. Linux systems NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 58: Specifications

    Specifications Note This document is preliminary and subject to change. SN5400 Specifications Feature Value NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 59 Ports #1-64: QSFP56-DD Connector cage: Port #65-66: SFP28 Operational: 0° to 40°C Temperature: Non-Operational: -40° to 70°C Environme Operational: 10% - 85% non-condensing ntal Humidity: Non-Operational: 10% - 90% non-condensing Altitude: 3050m NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 60 Typical power with passive cables (ATIS): 670W Consumption: Intel x86 Xeon, Hexa-core Coffee Lake E-2276ME CPU: w/ secured-boot PCIe: Gen3 x4 Main Devices Switch: NVIDIA Spectrum®-4 Memory: 32GB RAM DDR4, 160GB SSD Throughput 25.6Tbps SN5600 Specifications NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 61 Typical power with passive cables (ATIS): 940W Consumption: Intel x86 Xeon, Hexa-core Coffee Lake E-2276ME w/ CPU: secured-boot Main PCIe: Gen3 x4 Devices Switch: NVIDIA Spectrum®-4 Memory: 32GB RAM DDR4, 160GB SSD Throughput 51.2Tb/s NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 62: Appendixes

    Accessory and Replacement Parts Thermal Threshold Definitions Interface Specifications Disassembly and Disposal Accessory and Replacement Parts Note This document is preliminary and subject to change. Ordering Part Numbers for Replacement Parts NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 63: Thermal Threshold Definitions

    Emergency – 130°C: In case the firmware fails to shut down the ASIC device upon crossing its Critical threshold, the device will auto-shutdown upon crossing the Emergency (130°C) threshold. For thermal threshold definitions in Cumulus Linux, see Configuring SNMP Traps in the Cumulus User Guide. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 64: Interface Specifications

    Interface Specifications Small Form Factors Specifications NVIDIA switch systems come in a flexible range of form factors - SFP/QSFP, SFP28/QSFP28, SFP56/QSFP56, SFP-DD/QSFP-DD and OSFP. All form factors specification documents are available on the Storage Networking Industry Association (SNIA) Website. USB Standard Specifications For the specification documents of all available USB types, please refer to the document...

  • Page 65: Disassembly And Disposal

    1. Unplug and remove all connectors. 2. Unplug all power cords. 3. Remove the ground wire. 4. Unscrew the center bolts from the side of the system with the bracket. Important NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 66 The battery can explode if not properly used, replaced, or disposed Dispose of the battery according to your local regulations. Do not attempt to recharge the battery, disassemble, puncture, or otherwise damage it. NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...
  • Page 67 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

  • Page 68: Document Revision History

    Document Revision History Date Description June 2024 Updated SN5400 power specifications table May 2023 Updated Ordering Information page April 2023 First release © Copyright 2024, NVIDIA. PDF Generated on 06/06/2024 NVIDIA Spectrum-4 SN5000 2U Switch Systems Hardware User Manual...

This manual is also suitable for:

Sn5000

Table of Contents