Summary of Contents for Mellanox Technologies SwitchX-2 MSX6005F-1BFS
Page 1
SwitchX®-2 12 Port InfiniBand Switch System Hardware User Manual P/N: MSX6005F-1BFS, MSX6005T-1BFS, MSX6012F-1BFS, MSX6012F-2BFS, MSX6012T-1BFS, MSX6012T-2BFS, MSX6005F-1BRS, MSX6005T-1BRS, MSX6012F-1BRS, MSX6012T-1BRS Rev 1.0 www.mellanox.com...
Page 2
KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY.
SwitchX®-2 12 Port InfiniBand Switch System Hardware User Manual Rev 1.0 Revision History Table 1 - Revision History Table Date Revision Description July 2013 Initial Draft Mellanox Technologies...
This requires a customer support login. Look up the relevant SwitchX®- based switch system/series release note file. This document contains information regarding configuring and manag- Mellanox MLNX-OS® Software User ing Mellanox Technologies Switch Platforms. Manual Command Reference Guide for MLNX-OS® listing all of the commands MLNX-OS® Software Command Ref- available through MLNX-OS®...
This icon indicates a situation that can potentially cause damage to hardware or soft- ware. BEWARE! This icon indicates a situation that can potentially cause personal injury or damage to hardware or software. Abbreviations FDR — Fourteen Data Rate – Used to indicate a 4X port running at 56 Gb/s. Mellanox Technologies...
Rev 1.0 Mellanox Part Numbering Legend Place Field Decoder Mellanox Technologies System Type SwitchX® switch Family Data Transfer Protocol 6 = InfiniBand Size of box 0 = 1U 1 = 1.5U 2 = 2U Management Capabilities 05 = 12 Ports externally managed...
• Congestion control** • Adaptive routing** • Port mirroring** • Forward Error Correction -(FEC) • Link Layer Retransmission -(LLR) • Multiple SWIDs ** * License required. ** This feature will be available to customers in the near future. Mellanox Technologies...
Both FDR and FDR10 support Forward Error Correction (FEC), as described in IEEE 802.3ap chapter 74. FDR and FDR10 are only guaranteed to work with approved Mellanox Cables. FDR10 is only guaranteed to work with approved Mellanox ConnectX®-3 adapters. Mellanox Technologies...
RS232 cable DB9 to RJ-45 included in the box to connect to host PC for ini- Managed tial configuration of the switch. After initial configuration, the switch can be managed through the Ethernet port using a remote connection. Mellanox Technologies...
Figure 5: Switch System Connector Side Panels Mellanox SX6012 Status LEDs The System Status LEDs are located to the left of the connectors on the connector side panel. 2.2.1 System Status LEDs Figure 6: Externally Managed System Status LEDS Mellanox SX6005 Mellanox Technologies...
For more information See page 53. If the PS LEDs are not green, this indicates a problem with the power supplies. The switch is operational only if at least one of the PS LEDs is green. Mellanox Technologies...
All fans must be operating while the power supply is plugged in. switch switch If the shuts down due to over temperature, unplug the , wait 5 minutes switch See page 53 and replug in the . For more information Mellanox Technologies...
Error – One or more fans is not operating properly. The system should be powered down and troubleshoot the fan module. For the SX6005 replace the switch. Off – The fan unit is not receiving any power. Check that the fan unit is properly and completely inserted. Mellanox Technologies...
Error – The PS unit is not operational. Off – There is no power to the system (neither PS unit is receiving power). If one PS unit is showing green and the second PS unit is unplugged it will show a red indication. Mellanox Technologies...
The UID LED is a debug feature that lights a blue LED on the switch connector side panel for ease in finding a particular switch within a cluster. 1. This feature will be available in a coming software release. Mellanox Technologies...
Table 11 - Port Connector Physical and Logical Link Assignments for Ethernet Mode LED Configuration LED Description Physical link is down / Default. Solid Green Physical link is up with no traffic. Flashing Green Physical link is up with traffic. Flashing Orange Physical errors. Mellanox Technologies...
• 1 – USB port that is labeled This interface can be used to update software or firmware. This is only on the managed switch. • 1 – connector that is labeled Use this connector to connect to the host PC. Mellanox Technologies...
The Ethernet connection labeled provides access for remote management. The switch can be connected to any Ethernet port. The Ethernet port on the connector side of the switch is mgmt0. These connector(s) are not found in unmanaged (externally managed) switches. Mellanox Technologies...
Page 25
This connector can be used to install firmware upgrades, should the and Troubleshooting only. firmware image be damaged and cannot be upgraded through a host PC or remotely. This inter- face is for support personnel and advanced users only. Mellanox Technologies...
On the connector side panel under the system LEDs is a reset button . This reset button requires a tool to be pressed. DO NOT use a sharp pointed object such as needle or push pin for pressing the Reset button. Sharp objects can cause damage, use a flat object to push the reset button. Mellanox Technologies...
The ports are labelled as shown in Figure 17. The bottom row ports are flipped from the top row. See Figure 18 for bottom row - top row port orientation. See page 48 for more information regarding breakout (fanout) cables. Figure 17: Port Numbering Figure 18: Top and Bottom QSFP Port Orientation Mellanox Technologies...
“Avertismente privind siguranţa la instalare (Romanian)” on page 83, and for Chinese see page 88. 1. Installation Instructions Read all installation instructions before connecting the equipment to the power source. 2. Installation in Restricted Access Location This unit is intended for installation in a Restricted Access Location. Mellanox Technologies...
Page 29
When this product is mounted or serviced in a rack, special precautions must be taken to ensure that the system remains stable. In general you should fill the rack with equipment starting from the bottom to the top. Mellanox Technologies...
Page 30
“<HAR>”, 3 - conductor, minimum 1.0 mm wire, rated at 300 V, with a PVC insulated jacket. The cord must have a molded plug rated at 250 V, 10 A. 16. High Leakage Current Warning: High leakage current; Earth connection essential before connecting supply. Mellanox Technologies...
Page 31
Caution: Slide/rail mounted equipment is not to be used as a shelf or a work space. The rails are not intended for sliding the unit away from the rack. It is for permanent installation at final resting place only, not used for service and maintenance Mellanox Technologies...
Norway only. 3.1.1 Battery Replacement Mellanox Technologies does not support battery replacement. Customer removal of the switch cover will void the warrantee. Only Remove the cover to comply with WEEE directives or to dis- assemble for environmentally approved disposal. Customer removal of the switch cover will void the warrantee.
1. Peel and stick the four rubber bumpers into the bottom of the switch. Place them in the round circles. Figure 19: Placing the Bumpers Bottom of the switch 2. Place on a flat surface. Make sure the switch sits solid on the surface. Mellanox Technologies...
1. Place the ESD mat on the floor where you will be working and put on the ESD strap. Make sure the ESD strap is touching your skin and that the other end is connected to a verified ground. 2. Screw the two switch mounted rails to the switch using six flat head screws per rail. Mellanox Technologies...
SwitchX®-2 12 Port InfiniBand Switch System Hardware User Manual Rev 1.0 Figure 21: Screw on the Switch Mounted Rails Install the two switch frame into the rack. 1. Slide the Frame slides into the frame. Mellanox Technologies...
2. Place the frame in the rack. Both sides of the frame need to be on the inside of the rack. Figure 23: Placement of Frame in the Rack 3. Take the 10 spacer bushings and use them to adapt the square openings in the vertical support. Mellanox Technologies...
4. Use the 10 M5 pan head screws. Do not tighten the screws for the frame. Figure 25: Using the Spacer Bushings 5. Slide the switch into the frame, and screw in the capture bolts into the frame. Mellanox Technologies...
Anytime the switch is powered down (by choice or by power failure) you must unplug the 2 PSUs before powering up the system. 12. You can start connecting all of the cables to the switch. Mellanox Technologies...
15A or higher. Caution: The switch platform will automatically power on when AC power is applied. There is no power switch. Check all boards, power supplies, and fan tray modules for proper insertion before plugging in a power cable. Mellanox Technologies...
The externally managed switches are Plug and Play and all firmware updates should be done in- band. The I2C connection should only be used if the firmware image was corrupted to the point that the regular firmware tools cannot successfully reburn the correct image. Mellanox Technologies...
This must be done before any remote management is available. Hook up the supplied harness cable (HAR00028) from the connector labelled to the DB9 connector of the local host PC. Figure 29: Console Port Connect the host PC to this port. Mellanox Technologies...
Type ‘y’ for initial configuration? yes and then press <Enter>. Step 1: Hostname? [switch] If you wish to accept the default hostname, then press <Enter>. Otherwise, type a different hostname and press <Enter>. Mellanox Technologies...
Page 43
Note: <enter> to save changes and exit. To re-run the configuration wizard run the command Choice: <Enter> “configuration jump-start” in Config mode. Configuration changes saved. The table below shows an example of static IP configuration for mgmt0 interface. Mellanox Technologies...
To change an answer, enter the step number to return to. Otherwise hit <enter> to save changes and exit. Choice: Configuration changes saved. To return to the wizard from the CLI, enter the “configuration jump-start” command from configure mode. Launching CLI... switch> Mellanox Technologies...
The following is an example of the output: Interface mgmt0 state Admin up: Link up: IP address: 192.168.10.43 Netmask: 255.255.255.0 Speed: 1000Mb/s (auto) Duplex: full (auto) Interface type: ethernet Interface source: physical MTU: 1500 HW address: 00:02:C9:11:2A:AE Comment: Mellanox Technologies...
Page 46
To Start an SSH Connection to the Switch perform the following steps: 1. Set up an Ethernet connection between the switch and a local network machine (“the remote machine” henceforth) using a standard RJ-45 connector. 2. Connect to the remote machine (rem_mach1 is used as an example). Mellanox Technologies...
Page 47
<switch_IP_address> is the IP address of the switch. rem_mach1 > ssh -l <username> <switch ip address> Mellanox MLNX-OS® Switch Management Password: Last login: March 13, 2013 from 192.168.10.1 Mellanox Switch switch > 4. You can enter any supported command now. Mellanox Technologies...
40 GbE port into 4 lanes (4 SFP+ connectors 10 GbE each). All ports can also split into 2 10 GbE ports, using lanes 1 and 2 only. When a QSFP port is split into 2 10 GbE ports then only SFP+ connectors #1 and #2 are used. Connectors #3 and #4 are left unconnected. Mellanox Technologies...
Figure 31: Breakout or Fanout Cable When using this feature you MUST go into the MLNX-OS® chassis management system and reconfigure the individual ports to split-4. See the MLNX-OS® Software Command Reference Guide for instructions on port configuration. Mellanox Technologies...
Fabric Inspector includes a complete set of InfiniBand tools for fabric wide diagnostics to check node-node and node-switch connectivity and to verify routes within the fabric. Advanced filtering allows creating filtering rules on a system wide basis, between nodes or port connections based on traffic patterns and user assigned system names (GUIDs). Mellanox Technologies...
5.5.1 Current Firmware Revision For Externally Managed Switches In order to obtain the Firmware version of the externally managed switch: 1. Run the following command from a host: flint -d lid-[number] -qq Mellanox Technologies...
Page 52
Mellanox Firmware Tools (MFT) and documentation are available for download via http://www.mellanox.com > Downloads > Firmware & Documents. The MFT kit includes: • mlxburn • flint • spark • debug utilities See “Related Documentation” on page 8. Mellanox Technologies...
Plug in the switch. If the switch does not come on, check the power supplies. If the switch comes on, Use the MLNX-OS® management CLI to determine the cause of the Shutdown. Check the temperature. Check the Fan status. Mellanox Technologies...
Page 54
Issue 7. Reset the switch. If resetting the switch does not work: Unplug the switch. Wait 5 minutes. Plug in the switch. If the switch comes on, use the MLNX-OS® management CLI to determine the cause of the shutdown. Mellanox Technologies...
Temperature: Cable power / MAX 2.0W Operating 0° to 45° Celsius Temperature TYP: 1.5W Non-operating -40° to 70° Celsius Shock and Humidity: Shock and Vibration: ETSI EN 300 019-2-2: 1999-09 Operating 5% - 90% non-condensing Vibration/ Humidity Mellanox Technologies...
For a list of all approved cables see: http://www.mellanox.com/related-docs/user_manuals/Mellanox_approved_cables.pdf EMC Certifications The list of approved certifications per switch in different regions of the world is located on the Mellanox Website at: http://www.mellanox.com/related-docs/user_manuals/Regulatory_and_Compliance_Guide.pdf EMC statements are also in the Regulatory and Compliance Guide. Mellanox Technologies...
SwitchX®-2 12 Port InfiniBand Switch System Hardware User Manual Rev 1.0 China CCC Warning Statement * Will be available in Q4 2013. Mellanox Technologies...
When the SwitchX® device crosses this temperature, the firmware will automatically shut down the device. 3. Emergency – 130C In case the firmware fails to shut down the SwitchX® device upon crossing the Critical thresh- old, the SwitchX® device will auto-shutdown upon crossing the Emergency (130C) threshold. Mellanox Technologies...
+3.3 V Power supply transmitter Vcc 1 +3.3 V Power Supply LPMode Low Power Mode Ground Tx3p Transmitter Non-Inverted Data Input Tx3n Transmitter Inverted Data Input Ground Tx1p Transmitter Non-Inverted Data Input Tx1n Transmitter Inverted Data Input Ground Mellanox Technologies...
Rev 1.0 Figure 32: QSFP Connector Male and Female Views 18.35 View into Rear of Connector 8.50 18.35 View into Front of Cage 8.50 Mellanox Technologies...
Table 20 - OPNs for Replacement Parts Part Description I2C DB9 or RJ-45 to USB Adapter MTUSB-1 Harness RS232 2M cable – DB9 to RJ-45 (for managed switches only) HAR000028 Power cord Type C13-C14 ACC000500 Side by side installation kit MSX60-DKIT Mellanox Technologies...
Dispose of this product and all of its parts in a responsible and environmentally friendly way. Follow the instructions found in the Mellanox Web site at mellanox.com for proper instructions to disassemble and dispose of the Switch according to the WEEE directive. Mellanox Technologies...
Cet équipement ne doit pas être en service dans un local dont la température dépasse le maximum recommandé de 45°C (113°F). En outre et pour garantir une circulation d'air correcte, laisser un espace d'au moins 8 cm (3") autour des orifices de ventilation. Mellanox Technologies...
Page 65
Lors du montage ou de la maintenance de ce produit dans un rack, il faut faire spéciale- ment attention pour s'assurer que l'ensemble reste stable. En règle générale, le rack doit être rempli en commençant par le bas. Mellanox Technologies...
Page 66
300 V, avec une gaine isolante en PVC. Le cordon doit avoir une prise moulée 250 V 10 A. 20. Courant de fuite élevé Avertissement : courant de fuite élevé, une connexion à la terre est indispensable avant de brancher l'alimentation. Mellanox Technologies...
Page 67
: IEC 60 825-1:1993 + A1:1997 + A2:2001 et EN 60825-1:1994+A1:1996+ A2:2001 Pour les produits disposant de ports optiques Armoire de protection appropriée Une enceinte appropriée aux niveaux électrique, mécanique et incendie doit être four- nie par le constructeur du produit fini ou par l'utilisateur. Mellanox Technologies...
Page 68
électroniques (EEE) doivent être collectés séparément et ne pas être mis au rebut avec les déchets ménagers habituels. Ce produit et toutes ses pièces doivent être mis au rebut d'une manière responsable, respectant l'environnement. 28. Restrictions concernant l'alimentation pour la Norvège Mellanox Technologies...
Temperatur von 45°C (113°F) betrieben werden. Es ist ein Luft- strom von 200 LFM bei maximaler Umgebungstemperatur erforderlich. Außerdem sollten mindestens 8 cm (3 in.) Freiraum um die Belüftungsöffnungen sein, um einen einwandfreien Luftstrom zu gewährleisten. Mellanox Technologies...
Page 70
Stabilität des Systems zu gewährleisten. Im Allgemeinen sollten Sie das Gestell von unten nach oben mit Geräten füllen. 14. Geräteinstallation Diese Gerät sollte nur von geschultem und qualifiziertem Personal installiert, ausge- tauscht oder gewartet werden. Mellanox Technologies...
Page 71
20. Hoher Ableitstrom WARNUNG: Hohe Ableitstrom; Earth Verbindung, bevor Sie die Verbindung von wesentlicher Bedeutung werden. 21. Infomationen zum Erdungsanschluss Bevor Sie dieses Gerät an das Stromnetz, die Schutzerde Terminal Schrauben dieses Gerät muss an den Schutzleiter in der Gebäudeinstallation. Mellanox Technologies...
Page 72
Gemäß WEEE Directive 2002/96/EC müssen alle elektrischen und elektronischen Abfallgeräte (EEE) separat gesammelt und nichit mit normalem Haushaltsmüll ensorgt werden. Dieses Produkt und alle seine Teile in verantwortungsvoller und umweltfreundlicher Art und Weise entsorgen. 28. Netzbeschränkungen für das Land Norwegen Mellanox Technologies...
No se debe utilizar el equipo en un área con una temperatura ambiente superior a la máxima recomendada: 45°C. Además, para garantizar una circulación de aire adec- uada, se debe dejar como mínimo un espacio de 8 cm (3 pulgadas) alrededor de las aberturas de ventilación. Mellanox Technologies...
Page 74
Al instalar o realizar el mantenimiento de este aparato en un bastidor, es preciso adop- tar precauciones especiales para garantizar que el sistema se mantenga estable. En gen- eral, en un bastidor, los equipos se deben instalar comenzando desde abajo hacia arriba. Mellanox Technologies...
Page 75
"<HAR>", de tres conductores, hilo de 1,0 mm2 como mínimo, 300 voltios nominal, con cobertura protectora aislante de PVC. El cable debe tener un enchufe moldeado con capuchón de 250 voltios nominal, 10 A. 20. Alta corriente de fuga Mellanox Technologies...
Page 76
PRODUCTO LÁSER DE CLASE 1 y referencia a las normas de láser más recientes: IEC 60825-1:2007/03 y EN 60825-1:2007 Recinto adecuado El fabricante del producto final o el usuario final deberán suministrar un confinamiento adecuado para componentes eléctricos y mecánicos y contra incen- dio. Mellanox Technologies...
Page 77
(EEE) se deben recolectar por separado y no se deben eliminar junto con residuos domésticos. Al deshacerse de este producto y de todas sus partes, hágalo de una manera respon- sable y respetuosa con el medio ambiente. 28. Limitaciones de potencia en Noruega Mellanox Technologies...
НЕ вставлять инструменты или части тела в углубление вентиляторного модуля. 6. Перегрев Не эксплуатировать это оборудование в помещении с температурой окружающей среды, превышающей максимально рекомендуемое значение: 45 °C (113 °F). Более того, для надлежащей вентиляции следует обеспечить зазор вокруг вентиляционных отверстий не менее 8 см (3 дюйма). Mellanox Technologies...
Page 79
подсоединять и отсоединять. За особыми предупреждениями и указаниями следует обратиться к производителю кабеля. 13. Установка или обслуживание в стойке При установке или обслуживании этого изделия в стойке следует обеспечить устойчивость системы. Как правило, стойка заполняется оборудованием снизу вверх. Mellanox Technologies...
Page 80
Подключение к электропитанию в Европе выполняется с помощью гармонизированного шнура питания с маркировкой <HAR>, 3-жильного, с сечением жилы не менее 1,0 мм², рассчитанного на номинальное напряжение 300 В, с ПВХ оболочкой. Шнур должен иметь литую вилку, рассчитанную на 250 В, 10 А. Mellanox Technologies...
Page 81
ЛАЗЕРНОЕ ИЗДЕЛИЕ КЛАССА 1 и ссылка на последние стандарты по безопасности лазерных изделий IEC 60 825-1:1993 + A1:1997 + A2:2001 и EN 60825-1:1994 + A1:1996 + A2:2001. Соответствующий кожух Производитель конечного изделия и/или конечный пользователь должны обеспечить соответствующий электрический, механический и противопожарный кожух. Mellanox Technologies...
Page 82
Следует утилизировать это изделие и все его части ответственным и экологически безопасным способом. 28. Ограничения для сетей электропитания Норвегии This unit is intended for connection to a TN power system and an IT power system of Norway only. Mellanox Technologies...
Acest echipament nu trebuie să fie acţionat într-o zonă unde temperatura ambiantă depăşeşte valoarea maximă recomandată: 45°C (113°F). În plus, pentru a asigura un flux de aer adecvat, lăsaţi un spaţiu liber de cel puţin 8 cm (3 inchi) în jurul fantelor de ventilare. Mellanox Technologies...
Page 84
Când acest produs este montat sau depanat într-un rack, trebuie să fie luate măsuri de precauţie speciale pentru a se asigura că sistemul rămâne stabil. În general, trebuie să umpleţi rack-ul cu echipamente începând de jos în sus. Mellanox Technologies...
Page 85
PVC. Cordonul de alimentare trebuie să fie prevăzut cu o fişă turnată cu putere nominală egală cu 250 V, 10 A. 20. Curent de scurgere de înaltă frecvenţă Avertisment: Curent de scurgere de înaltă frecvenţă; Împământarea este esenţială înainte de a conecta sursa de alimentare. Mellanox Technologies...
Page 86
Atenţie: Echipamentul montat pe o linie de alunecare/şină nu va fi utilizat ca raft sau spaţiu de lucru. Scopul şinelor nu este de a glisa unitatea de pe rack. Acestea sunt des- tinate instalării permanente numai la punctul final de oprire şi nu vor fi folosite pentru depanare şi întreţinere Mellanox Technologies...
Page 87
În conformitate cu Directiva DEEE 2002/96/CE, toate deşeurile de echipamente elec- trice şi electronice (EEE) trebuie colectate separat şi nu trebuie eliminate împreună cu deşeurile menajere obişnuite. Eliminaţi acest produs şi toate componentele sale în mod responsabil şi ecologic. 28. Restricţii electrice pentru Norvegia Mellanox Technologies...