Table of Contents
Advertisement
Quick Links
Download this manual
See also:
User Manual
Advertisement
Table of Contents
Related Manuals for Mellanox Technologies InfiniScale MTS3600Q-1BNC
Summary of Contents for Mellanox Technologies InfiniScale MTS3600Q-1BNC
- Page 1 ® InfiniScale IV 36-Port QSFP InfiniBand Switch User Manual P/N: MTS3600Q-1BNC, MTS3600Q-2BNC, MTS3600Q-1UNC, MTS3600Q-2UNC, MTS3600R-1BNC, MTS3600R-2BNC, MTS3600R-1UNC, MTS3600R-2UNC Rev 2.4 www.mellanox.com...
- Page 2 THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT.
-
Page 3: Table Of Contents
Chapter 4 Switch Management Tools 4.1 Chassis Management 4.2 FabricIT Management Chapter 5 Troubleshooting Appendix A Specification Appendix B Switch Hardware Components Appendix C QSFP Interface Appendix D Replacement Parts Ordering Numbers Appendix E Avertissements de sécurité d’installation (French) Mellanox Technologies... - Page 4 Rev 2.4 Appendix F Installation - Sicherheitshinweise (German) Appendix G Special Regulations Regarding Finland, Sweden, Denmark, and Norway Mellanox Technologies...
- Page 5 Figure 10: Screwing in the Rails Figure 11: Two Power Inlets - Electric Caution Notifications Figure 12: Power Supply Unit Extraction Figure 13: Cable connections to the Switch Figure 14: MTUSB-1 with Cables Figure 15: I2C Cable Connected to MTS3600 Mellanox Technologies...
- Page 6 Fans, System Status, LED Configurations Table 6: Fan Status LED Configurations Table 7: MTS3600Q Specification Data Table 8: MTS3600R Specification Data Table 9: Switch Certification Status Table 10: InfiniBand QSFP Connector Pinout Table 11: Replacement Parts Ordering Numbers Mellanox Technologies...
-
Page 7: Revision History
September 2008 Rev 1.2 Added Reprogramming the MT3600 Through The I2C Port section. Added power cord OPNs to table 11 in Appendix D. September 2008 Rev 1.1 Fixed link to Software package page. September 2008 Rev 1.0 Initial release Mellanox Technologies... -
Page 8: About This Manual
Tools (MFT) The manual supplied with this package provides an overview of the firm- User’s Manual ware its installation and replacement. The MFT can be downloaded with its Document # 2329 documentation at: http://www.mellanox.com > Downloads > Firmware Tools. Mellanox Technologies... -
Page 9: Conventions
Switch Products Covered in this User’s Manual Table 3 - Switch Products Covered by this User Manual Product Number Description MTS3600Q-1BNC 36-port Managed QSFP 40Gb/s InfiniBand switch with one power supply MTS3600R-1BNC 36-port Managed QSFP 20Gb/s InfiniBand switch with one power supply... -
Page 10: Chapter 1 Overview
The serial number and product version information are found on the label seen in the figure below. Figure 1: Generic Product label Also on this label is the GUID identifier for the switch. There is also a label with the MAC for the management module. Mellanox Technologies... - Page 11 Rev 2.4 Figure 2: Management Module Label Mellanox Technologies...
-
Page 12: Chapter 2 Installation And Basic Operation
If the STATUS LED shows red unplug the switch and call your Mellanox representative for assis- tance. If the FAN LED shows yellow, troubleshoot the fan module. Note: Do not run the switch for more than 2 minutes should the FAN LED show yellow. Mellanox Technologies... -
Page 13: Table 5: Fans, System Status, Led Configurations
PSU Status Indicators Table 5 - Fans, System Status, LED Configurations LED Configuration Status OK – The system is up and running. Error –A fault in the system. Off – The system has no power. Mellanox Technologies... -
Page 14: Table 6: Fan Status Led Configurations
OK – All fans are operating. Error – One or more fans is not operating. Repair or replace the fan unit. Off – The fan unit is not receiving any power. Check that the fan unit is properly and com- pletely inserted. Mellanox Technologies... -
Page 15: I2C Connector
PSU 2 2.1.4.1 Serial CONSOLE Interface The port labelled “CONSOLE” is for local host connection to the management module. This is used the first time the switch is connected. Connecting to a local host PC and following the Mellanox Technologies... -
Page 16: Switch Platform Installation And Operation
49, and for German see Section F,“Installation - Sicherheitshinweise (German),” on page For Special Regulations Regarding Finland, Sweden, Denmark, and Norway see Section G,“Special Regulations Regarding Finland, Sweden, Denmark, and Norway,” on page 53. 1. Installation Instructions Read all installation instructions before connecting the equipment to the power source. Mellanox Technologies... - Page 17 In general you should fill the rack with equipment starting from the bottom to the top. 8. Equipment Installation This equipment should be installed, replaced, or serviced only by trained and qualified personnel. Mellanox Technologies...
-
Page 18: Mechanical Installation
• ESD Strap • Grounding wire sufficient to reach a valid ground. • ESD mat Parts included in the installation kit: • 2 rails • 8 recessed flat head screws • 2 rail slides • 4 caged nuts Mellanox Technologies... - Page 19 IB connectors can be even with the vertical rack support. The other side of the switch will be further inside of the rack. Things to consider before choosing where to mount the rails and rail slides. Mellanox Technologies...
- Page 20 Tighten the screws to 3Nm or 26.5 pound inches. Figure 9: Screwing the Rail Slide onto the switch This side of the switch will be next to the ver- Mellanox Technologies...
-
Page 21: Grounding The Switch
2.2.3 Grounding the Switch You must install an external ground to all IT components. Connect a ground wire to one of the cas- ing screws and connect the other end to a valid ground. Do not rely on the power connection Mellanox Technologies... -
Page 22: Power Connections And Initial Power On
Einhieten spannungs- ensure a powered down state “Alimentation Stabilise’e”. frei zu machen sind die inside of the switch platform. Pour isoler completement le Netzstecker aller Netzteile module en cause, Il faut zu entfernen de’brancher toutes les ali- mentations stabilise’es. Mellanox Technologies... -
Page 23: Extracting And Inserting The Power Supply Unit
3. Continue pressing the PSU until it seats completely. The latch will snap into place confirming the proper installation. 4. Insert the power cord into the supply connector. Note: The green PSU indicator should light and the yellow indicator should remain off. If not, extract the PSU and re-insert it. Mellanox Technologies... -
Page 24: Infiniband Cable Installation
Care should be taken not to impede the air exhaust flow through the ventilation holes next to the InfiniBand ports. Cable lengths should be used which allow for routing horizontally around to the side of the chassis before bending upward or downward in the rack. Figure 13: Cable connections to the Switch Mellanox Technologies... -
Page 25: Extracting And Inserting The Fan Unit
Note: Support the weight of the switch when you remove the screws so that the switch does not fall. 5. Slide the switch from the rack. 6. Remove the rails from the rack. 7. Remove the eight caged nuts. Mellanox Technologies... -
Page 26: Disposal
According to the WEEE Directive 2002/96/EC, all waste electrical and electronic equipment (EEE) should be collected separately and not dis- posed of with regular household waste. Dispose of this product and all of its parts in a responsible and environ- mentally friendly way. Mellanox Technologies... -
Page 27: Chapter 3 Management And Tools Overview
The Subnet management features include: • upgrading Drivers • upgrading software • monitoring of: • AC power to the PSUs • DC power out from the PSUs Mellanox Technologies... -
Page 28: Initializing The Switch For The First Time
2. Follow the instructions in the Installation guide Rev 1.3 “Configuring the Switch for the First Time”. 3.2.3 Starting a Remote Connection to the Switch 3.2.3.1 Accessing the CPU via the Ethernet Connector Once the initial configuration is completed the management tools can be accessed through: • SSH • Telnet • the WEB Mellanox Technologies... -
Page 29: Downloading Firmware
/dev/mst/ directory. These devices can be used for access by other MFT tools. [root@mymach]> mst ib add -I- Running ibdiagnet to discover the fabric ... Loading IBDIAGNET from: /usr/local/lib/ibdiagnet1.2 -W- Topology file is not specified. Reports regarding cluster links will use direct routes. Mellanox Technologies... - Page 30 -I- Fabric Partitions Report (see ibdiagnet.pkey for a full hosts list) -I--------------------------------------------------- -I- skip option set. no report will be issued -I--------------------------------------------------- -I- IPoIB Subnets Check -I---------------------------------------------------32 -I- skip option set. no report will be issued -I--------------------------------------------------- -I- Bad Links Info -I--------------------------------------------------- -I- No bad link were found Mellanox Technologies...
- Page 31 -I- Added 3 in-band devices [root@mymach]> To list the discovered mst inband devices run “mst status”. [root@mymach]> mst status MST modules: ------------ MST PCI module loaded MST PCI configuration module loaded Inband devices: ------------------- /dev/mst/CA_MT25418_sw005_HCA-1_lid-0x0001 /dev/mst/SW_MT47396_lid-0x0011 /dev/mst/SW_MT48438_lid-0x0003 [root@mymach]> Mellanox Technologies...
-
Page 32: Reprogramming The Switch Through The I2C Port
Run: [root@mymach]# ibnetdiscover | grep 00000006660abcd0 | grep -w Switch Switch 24 "S-00000006660abcd0" "MT47396 Infiniscale-III Mellanox Technologies" base port 0 lid 17 lmc 0 Note: The resulting LID is given as a decimal number. 4. Run mlxburn with the LID retrieved in step #3 above to perform the In-Band burning operation. - Page 33 • "mtusb-1" mst device appears in "mst status" 2. Set the switch I2C switch to allow access by running: isw -d /dev/mst/mtusb-1 -s shark -p L1 3. Run the burn command: flint -d /dev/mst/mtusb-1 -i ./fw-IS4-rel-7_0_142-MTS3600Q_A1.bin -qq b Mellanox Technologies...
-
Page 34: Externally Managed Switch System
MTS36140 via IB. This is done by connecting a cable from the QSFP port of a leaf of the MTS3610 switch chassis to the IB connector on the Host. Mellanox Technologies... - Page 35 -I- skip option set. no report will be issued -I--------------------------------------------------- -I- Links With Logical State = INIT -I--------------------------------------------------- -I- skip option set. no report will be issued -I--------------------------------------------------- -I- PM Counters Info -I--------------------------------------------------- -I- skip option set. no report will be issued Mellanox Technologies...
- Page 36 Performance Counters Report Partitions Check IPoIB Subnets Check Please see /tmp/ibdiagnet.log for complete log ---------------------------------------------------------------- -I- Done. Run time was 1 seconds. -I- Added 3 in-band devices [root@mymach]> To list the discovered mst inband devices run “mst status”. Mellanox Technologies...
- Page 37 Run: [root@mymach]# ibnetdiscover | grep 00000006660abcd0 | grep -w Switch Switch 24 "S-00000006660abcd0" "MT47396 Infiniscale-III Mellanox Technologies" base port 0 lid 17 lmc 0 Note: The resulting LID is given as a decimal number. 4. Run mlxburn with the LID retrieved in step #3 above to perform the In-Band burning operation.
- Page 38 *** WARNING *** Running quick query - Skipping full image integrity checks. Current FW version on flash: 7.0.135 New FW version: 7.0.138 Burning second FW image without signatures - OK Restoring second signature - OK -I- Image burn completed successfully. Mellanox Technologies...
-
Page 39: Chapter 4 Switch Management Tools
Web interface or the command line interface and software supplied on the management chip. Make sure to register your FabricIT license to enable all of the available commands and functions. See the FabricIT EFM User Manual for instructions and commands available to manage the switches, and fabric. Mellanox Technologies... -
Page 40: Chapter 5 Troubleshooting
The link LED for the InfiniBand connector does not come on: 1. Check that both ends of the cable are connected. 2. Check that the locks on the ends are secured. 3. Make sure that the latest FW version is installed on both the HCA and the switch. Mellanox Technologies... - Page 41 Rev 2.4 4. If media adapters are used, check that the all connections are good, tight, and secure. The activity LED does not come on: Check that the Subnet Manager has been started. Mellanox Technologies...
-
Page 42: Appendix A Specification
Scalability and Performance Features Switching Perfor- Simultaneous wire-speed any port to any mance: port Hot-Swappable: Fan Module and Power Sup- 1+1 Redundant: plies Addressing: 48K Unicast Addresses Max. per Subnet Power Supplies 16K Multicast Addresses per Subnet Switching Capacity: 2.88Tb/s Mellanox Technologies... -
Page 43: Table 8: Mts3600R Specification Data
Scalability and Performance Features Switching Perfor- Simultaneous wire-speed any port to any mance: port Hot-Swappable: Fan Module and Power Sup- 1+1 Redundant: plies Addressing: 48K Unicast Addresses Max. per Subnet Power Supplies 16K Multicast Addresses per Subnet Switching Capacity: 1.44Tb/s Mellanox Technologies... -
Page 44: Table 9: Switch Certification Status
Statement Warning! Changes or modifications to this equipment not expressly approved by the party responsi- ble for compliance (Mellanox Technologies) could void the user's authority to operate the equipment. §15.105(a) Statement NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. - Page 45 A.0.1 (Translation - "This is a Class A product based on the standard of the Voluntary Control Council for Interference by Information Technology Equipment (VCCI). If this equipment is used in a domestic environment, radio interference may occur, in which case the user may be required to take corrective actions.") Mellanox Technologies...
-
Page 46: Appendix B Switch Hardware Components
Rev 2.4 Appendix B: Switch Hardware Components Place Optional Primary Power Supply Unit Fan Unit Secondary Mellanox Technologies... -
Page 47: Appendix C Qsfp Interface
Receiver Non-Inverted Data Output 3 Ground ModPrsL Module Present IntL Interrupt Vcc Tx +3.3 V Power supply transmitter Vcc 1 +3.3 V Power Supply LPMode Low Power Mode Ground Tx3p Transmitter Non-Inverted Data Input Tx3n Transmitter Inverted Data Input Mellanox Technologies... -
Page 48: Appendix D Replacement Parts Ordering Numbers
Power cord Type G for UK ACC000208 Power cord Type D for India ACC000209 Power cord Type I for China ACC000210 Power cord Type J for Switzerland ACC000211 Power cord Type B for Japan, ACC000212 Power cord Type I for Australia ACC000213 Mellanox Technologies... -
Page 49: Appendix E Avertissements De Sécurité D'installation (French)
7. Montage et entretien sur baie Lorsque ce produit est monté ou entretenu sur baie, il faut prendre des précau- tions spéciales pour s’assurer que le système reste stable. En général, il faut remplir la baie avec du matériel de bas en haut. Mellanox Technologies... - Page 50 L’élimination de ce matériel doit s’effectuer dans le respect de toutes les légis- lations et réglementations nationales en vigueur. 10. Codes électriques locaux et nationaux Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux. Mellanox Technologies...
- Page 51 Rack-Montage und Wartung Wenn dieses Produkt in einem Rack montiert oder gewartet wird, sind beson- dere Vorsichtsmaßnahmen zu ergreifen, um die Stabilität des Systems zu gewährleisten. Im Allgemeinen sollten Sie das Gestell von unten nach oben mit Geräten füllen. Mellanox Technologies...
- Page 52 Diese Gerät sollte nur von geschultem und qualifiziertem Personal installiert, ausgetauscht oder gewartet werden. 9. Geräteentsorgung Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestimmungen erfolgen. 10. Regionale und nationale elektrische Bestimmungen Dieses Gerät sollte unter Beachtung der regionalen und nationalen elektrischen Bestimmungen installiert werden. Mellanox Technologies...
- Page 53 Do not connect this unit to any outlet that is not fully grounded! • Finland “Laite on liitettävä suojamaadoituskoskettimilla varustettuun pistorasiaan” • Norway “Apparatet må tilkoples jordet stikkontakt” Unit is intended for connection to IT power systems for Norway only. • Sweden “Apparaten skall anslutas till jordat uttag.” Mellanox Technologies...