Related Manuals for Nvidia MetroX-3 XC TQ8400 Long-haul 1U
Summary of Contents for Nvidia MetroX-3 XC TQ8400 Long-haul 1U
- Page 1 NVIDIA MetroX-3 XC TQ8400 Long-haul 1U Appliance User Manual Exported on Nov/02/2023 01:48 PM...
-
Page 2: Table Of Contents
Table of Contents Introduction..................5 Product Overview ..................5 NVIDIA MetroX-3 XC Highlights ..............5 Main System Components................5 System Features..................6 Operating Systems ...................6 Certifications ..................7 System Layout and Interfaces..............8 NVIDIA MetroX-3 XC Front Panel ..............8 NVIDIA MetroX-3 XC Rear Panel ..............8 Interfaces Detailed Description..............9... - Page 3 Cable Installation .................. 25 Power Cable ..................25 ConnectX-7 Networking Cards Cables............25 Initial Power-On ..................26 System Maintenance................27 Power Supply Units ................27 Configuring the Gateway for the First Time ..........28 MetroX Initialization................28 Rerunning the Wizard ................. 32 Starting the Command Line Interface (CLI)..........
- Page 4 About this Manual This manual describes the installation and basic use of NVIDIA® MetroX®-3 XC long-haul 1U appliance. Ordering Part Numbers The table below provides the ordering part number (OPN) for the available NVIDIA MetroX-3 XC systems. NVIDIA SKU Lifecycle...
-
Page 5: Introduction
Introduction This is the user guide for NVIDIA® MetroX®-3 XC product family. This document contains the complete product overview, installation and initialization instructions, and product specifications. This document is preliminary and subject to change. Product Overview The NVIDIA® MetroX®-3 XC (Xternal Connect) long-haul system seamlessly and securely extends the reach of the NVIDIA Quantum InfiniBand networking platform, providing high data throughput, In- Network Computing, and native remote direct-memory access (RDMA) communications. -
Page 6: System Features
System Features For a full list of features, please refer to the system's datasheet. Operating Systems NVIDIA MetroX-3 XC includes the NVIDIA Gateway Operating System, MLNX-GW, which manages the appliance and handles the high availability and load balancing between the ConnectX cards and... -
Page 7: Certifications
For a detailed description of MLNX-GW, please contact your NVIDIA representative. Certifications The list of certifications per system for different regions of the world (such as EMC, safety, and others) is located on the NVIDIA Networking website at https://www.nvidia.com/en-us/networking/... -
Page 8: System Layout And Interfaces
System Layout and Interfaces The figures below show the front and rear sides of NVIDIA Metro3-2 XC. Each numbered interface that is referenced in the figures is described in the following table. For additional information on the monitoring interfaces in the front and rear panel, see System... -
Page 9: Interfaces Detailed Description
Item Interface Description PCIe expansion card riser (slot 1) The expansion card riser enables to connect PCIe expansion cards PCIe expansion card riser (slot 2) The expansion card riser enables to connect PCIe expansion cards USB 2.0 port USB 2.0-compliant Power supply unit (FRU) PSU 2 USB 3.0 port... -
Page 10: Pcie Gen 4.0 Slots
Redundant Power Module NVIDIA MetroX-3 XC is equipped with two redundant power supply units at the rear of the appliance. The PSUs are housed in a 2U canister containing the power supplies. Each PSU has an extraction handle, PSU status LED, and a power socket. -
Page 11: Fans
Fans Power Supply Fans NVIDIA MetroX-3 XC is equipped with one fan per power supply unit on the rear panel of the appliance. Internal Fans NVIDIA MetroX-3 XC system has an extensive collection of sensors that automatically track thermal activity, which helps regulate temperature, thereby reducing server noise and power consumption. -
Page 12: Hardware Installation
The rack mounting holes conform to the EIA-310 standard for 19-inch racks. Take precautions to guarantee proper ventilation in order to maintain good airflow at ambient temperature. MetroX-3 XC Installation The installation procedure of NVIDIA MetroX-3 XC systems involves the following steps. Step Procedure Direct Link Follow safety warning procedures. - Page 13 Bodily Injury Due to Weight Use enough people to lift this product safely. Heavy Equipment This heavy equipment should be moved using a mechanical lift to avoid injuries. Risk of Electric Shock! • With the fan module removed power pins are accessible within the module cavity.
- Page 14 Equipment Installation This equipment should be installed, replaced, and/or serviced only by trained and qualified personnel. Equipment Disposal Disposal of this equipment should be in accordance to all national laws and regulations. Local and National Electrical Codes This equipment should be installed in compliance with local and national electrical codes.
-
Page 15: Taiwan Rohs Declaration - Switch Systems
Country of Norway Power Restrictions This unit is intended for connection to a TN power system and an IT power system of Norway only. Taiwan RoHS Declaration - Switch Systems... -
Page 16: Taiwan Rohs Declaration - Gateway Systems
The operating environment should meet severity level G1 as per ISA 71.04 for gaseous contamination and ISO 14644-1 class 8 for cleanliness level. Airflow Requirements NVIDIA MetroX-3 XC is offered with one airflow pattern: from the front panel to the rear panel. Please refer to the Technical... -
Page 17: Unpacking The Package
Unpacking the Package Safety Precautions The NVIDIA MetroX-3 XC is installed in systems that operate with voltages that can be lethal. Before opening the case of the system, observe the following precautions to avoid injury and prevent damage to system components. - Page 18 Installing the Appliance in Rack Pull the inner rails out of the rack until they lock into place. Locate the rear rail standoff on each side of the system and lower them into the rear J-slots on the slide assemblies. Rotate the system downward until all the rail standoffs are seated in the J-slots.
- Page 19 Pull the blue side release lock tabs forward or backward on both rails and slide the system into the rack until the system is in the rack. Ground the appliance (see "Grounding the Appliance"). Plug in the power cables (see "Power Connections and Initial Power On").
-
Page 20: Connecting The Appliance To The Network/Fabric
InfiniBand ports should be connected to InfiniBand switches. They can be connected to the same switch, but NVIDIA recommends connecting to two separate switches to ensure SM connectivity to the fabric. - Page 21 Do not hot swap the power supply if your appliance has only one power supply. You must power down the system to replace the power supply unit there is only one PS unit in the appliance. Extracting and Inserting the Power Supply Unit Two Power Inlets - Electric Caution Notifications ...
- Page 22 Press and hold the PSU latch while sliding the PSU out: Slide the new PSU in: If you have unlatched the cable management arm, re-latch it. Connect the power cable to the PSU and plug the cable into a power outlet. ...
-
Page 23: Replacing The Ssd
Do not run the system with openings of missing parts. This may cause overheating due to improper air flow. Replacing the SSD Never pull out a working hard drive while the appliance is turned on. You can safely pull out a faulty hard drive indicated by a solid amber light. -
Page 24: Disassembly Of The System From The Rack
Remove the rail slides from the rack. Removing the Battery NVIDIA does not support battery replacement. Customer removal of the cover will void the warranty. Remove the cover only to comply with WEEE directives or to disassemble the appliance for environmentally approved disposal. -
Page 25: Cable Installation
Cable Installation Power Cable The NVIDIA MetroX-3 XC appliance is shipped with two power supply units. Each unit has a separate AC receptacle. The appliance accepts voltages of 100-127 VAC and 200-240 VAC for all possible power supply units. The power cords should be a standard 3-wire AC power cards, including a safety ground, and rated for 15A or higher. -
Page 26: Initial Power-On
The LED indicator for that port will turn off when the cable is unseated. For full cabling guidelines, ask your NVIDIA Networking representative for a copy of NVIDIA Cable Management Guidelines and FAQs Application Note. -
Page 27: System Maintenance
System Maintenance This chapter contains the installations and Un-installation instructions of the following customer replaceable units: Power Supply Units MetroX-3 XC is equipped with two replaceable power supply units that work in a redundant configuration. The below figure shows the power side of the system which includes a hot-swap power supply unit (PSU). -
Page 28: Configuring The Gateway For The First Time
Configuring the Gateway for the First Time MetroX Initialization To initialize the gateway, follow the steps below. Enable remote access to serial console via IPMI. The MAC address for the SOL port can be found in the BIOS or on the outside of the chassis is labeled with the port MAC address. - Page 29 Go to “iDRAC Settings” tab → “Network" Here the MAC can be found and various network configuration related to the SOL port. its IPV4 settings can also be viewed and configured (by default it will try to get IP via DHCP).
- Page 30 Go back to main BIOS menu shown in step b (press esc and follow prompts), go to "System BIOS" tab → "Boot Settings" and make sure "boot Mode" is "UEFI" Go back to previous screen ("System BIOS"), go to "Serial Communication" tab and make sure "Serial communication"...
- Page 31 Go through the MetroX Management configuration wizard (Using the IPMI connection from step 2) Wizard Session Display (Example) Comments Do you want to use the wizard for initial This configuration must be performed the first configuration? yes time the MetroX is operated or after resetting it to the factory defaults.
-
Page 32: Rerunning The Wizard
Wizard Session Display (Example) Comments You have entered the following information: The wizard displays a summary of choices and Hostname: <metroX name> then asks to confirm the choices or to re-edit Use DHCP on mgmt0 interface: yes them. Enable IPv6: yes •... -
Page 33: Starting The Command Line Interface (Cli)
5139846 bytes 28452 packets discards errors overruns carrier collisions queue len 1000 Starting the Command Line Interface (CLI) Set up an Ethernet connection between the metroX and a local network machine using a standard SOL connector. Start a remote secured shell (SSH) to the metroX using the command “ssh -l <username> <metroX ip address>”. -
Page 34: System Monitoring
System Monitoring Front Panel Monitoring Interfaces Right Control Panel Index Indicator or Button Icon Description Power button Indicates if the system is powered on or off. Press the power button to manually power on or off the system. Press the power button to shut down the ACPI-compliant operating system. -
Page 35: System Health And System Id Indicator Codes
Icon Description Condition Corrective Action Temperature The indicator turns solid amber if the Ensure that none of the following indicator system experiences a thermal error conditions exist: (for example, the ambient • A cooling fan has been temperature is out of range or there removed or has failed. -
Page 36: Rear Panel Monitoring Interfaces
Rear Panel Monitoring Interfaces RJ-45 Remote Management Port The remote management port is designed for secure local and remote server management and helps IT administrators deploy, update, and monitor the NVIDIA® MetroX-3 XC Appliance. -
Page 37: Rj-45 Management Ports Eth0-Eth1
RJ-45 Management Ports eth0-eth1 These four RJ-45 ports are found on the rear side of the appliance. The eth0-eth1 and remote management interfaces are pre-configured as DHCP and the initial host name is MetroX3xc-1 (the MAC address appears on the pull-tab label), so their IP addresses can be obtained from the DHCP server. -
Page 38: Usb Interface
NIC#1 Ethernet connector gets connected to Ethernet switches. This switch must be configured to 100M/1G auto-negotiation. USB Interface There are two USB connectors. These connectors can be used to install software and/or firmware upgrades using a memory device that has a USB connector. This connector is USB 2.0 compliant. Various upload/download operations are also supported through the USB using the CLI. -
Page 39: Nic Activity Led Indicators
Power Indicator Codes Condition Blinking green Indicates that the firmware of the PSU is being updated Do not disconnect the power cord or unplug the PSU when updating firmware. If firmware update is interrupted, the PSUs will not function. Blinking green and powers off When hot-plugging a PSU, it blinks green five times at a rate of 4 Hz and powers off. -
Page 40: Air Flow
Index Description Link LED indicator Activity LED indicator The following table lists the drive indicator codes: NIC Indicator Code Condition Link and activity indicators are off Indicates that the NIC is not connected to the network Link indicator is green, and activity indicator is Indicates that the NIC is connected to a valid network at its blinking green maximum port speed, and data is being sent or received... -
Page 41: Troubleshooting
General Troubleshooting Issue Resolution System Status LED is RED Unplug the appliance and call your NVIDIA representative. Power Supply Unit Status LED is not lit or is RED 1. Check that the power cable is plugged into a working outlet. -
Page 42: Technical Specifications
Hot-swappable: 1+1 power supplies Availability Redundancy N+N redundant Serviceabilit y Features The ConnectX-7 adapters supplement the IBTA auto-negotiation specification to get better bit error rates and longer cable reaches. This supplemental feature only initiates when connected to another NVIDIA InfiniBand product. -
Page 43: Thermal Threshold Definitions
Thermal Threshold Definitions There are two thermal threshold definitions for MetroX-3 XC which impact the overall system operation state: • Critical – When the device crosses this temperature, the firmware will automatically shut down the device. This temperature threshold is set from the BIOS (Advanced > IT8528 HW Monitor >... -
Page 44: Inventory Information
Inventory Information The system’s inventory parameters (such as Serial Number and Part Number) are found on the pull- tab label. The pull-tab can be extracted from the right bottom side of the system's front panel. -
Page 45: Field Replaceable Units
Field Replaceable Units Ordering Number Part Description MTQ84-PS NVIDIA power supply for MetroX-3 XC appliance MTQ84-RKIT NVIDIA rail kit for MetroX-3 XC appliance TQ8400-SD NVIDIA MetroX-3 XC appliance SSD FRU ... -
Page 46: Revision History
Revision History Date Revision Description of Changes January 2023 First Release... - Page 47 NVIDIA accepts no liability related to any default, damage, costs, or problem which may be based on or attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this document or (ii) customer product designs.
- Page 48 Copyright © 2023 NVIDIA Corporation & affiliates. All Rights Reserved.