Page 2
THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT.
4.2.4 Starting a Remote Connection to the Switch 4.2.5 Downloading Firmware 4.3 Unmanaged (Externally Managed) Switch 4.3.1 I2C Connector 4.3.2 Current Firmware Revision 4.3.3 How to Get Mellanox Firmware Tools (MFT) 4.3.4 Open SM 4.4 Updating Firmware on Managed Switches Chapter 5 Troubleshooting Mellanox Technologies...
Page 4
Appendix E Replacement Parts Ordering Numbers Appendix F Avertissements de sécurité d’installation (French) Appendix G Installation - Sicherheitshinweise (German) Appendix H Advertencias de seguridad para la instalación (Spanish) Appendix I Special Regulations Regarding Finland, Sweden, Denmark, and Norway Mellanox Technologies...
Figure 30: Rack Installation Kit Parts Figure 31: Rack Rail Kit Parts MIS00083/85 Figure 32: Which Side of the Rack Do You Want the Connectors? Figure 33: Screwing the Rail Slide onto the switch Figure 34: Caged Nut Spacing Mellanox Technologies...
Page 6
Figure 43: Screwing on the Bracket Figure 44: Making Room for the Power Cord Figure 45: Screwing on the Rail Figure 46: Power Cord Through the Bracket Figure 47: Power Side View Figure 48: Connect Bracket to Rack Vertical support Mellanox Technologies...
Fan Status LED Configurations Table 9: Air Flow Direction Table 10: IS5025 Specification Data Table 11: IS5030 Specification Data Table 12: IS5035 Specification Data Table 13: InfiniBand QSFP Connector Pinout Table 14: RJ45 CONSOLE Pinout Table 15: Replacement Parts Ordering Numbers Mellanox Technologies...
December 2009 Rev 1.4 Added Chapter 6 regarding FabricIT-EFM licensing information. Added Chapter 2Management modules and CPU connections Section 4.3 on unmanaged switches including The I2C connector should only be used when the FW cannot be updated in-band. Mellanox Technologies...
Page 9
Fixed Status Fan LED info on page 13. Added to Troubleshooting section. Updated power numbers Added IS5025Q-2SRC, IS5025Q-2BRC model numbers September 2009 Rev 1.2 Added Spec sheets for IS5025 and IS5035 updated revision table September 2009 Rev 1.1 Updated power numbers September 2009 Rev 1.0 Initial Release Mellanox Technologies...
The manual supplied with this package provides an overview of the firm- User’s Manual ware its installation and replacement. The MFT can be downloaded with its Document # 2329 documentation at: http://www.mellanox.com > Support > Download Firmware Tools Mellanox Technologies...
Throughout this manual, the name IS50XX and the term switch are used to describe both the 36- port QSFP 40Gb/s InfiniBand Switch and the 36-port QSFP 20Gb/s InfiniBand Switch, unless explicitly indicated otherwise. Mellanox Part Numbering Legend Place Field Decoder Mellanox Technologies System Type InfiniScale Switch Model Family Form factor 25 = 36 Ports Unmanaged...
Please see section xxx, on how to install the license key. IS5030 and IS5035 manage- ment stack license is located on the bottom side of the pull out tab. Figure 1: Pull Out Tab IS5030 S/N: MT0924X00266 Rev: X1 CONSOLE P/N: MIS5030Q-1BFC GUID: 0002C9020040DDB0 Made in IL Pull out tab Mellanox Technologies...
IS5035 Managed RS232 cable DB9 to RJ45 included in the box to connect to host PC for initial configuration of the switch. After initial configuration the switch can be managed through the ethernet using a remote connec- tion. Mellanox Technologies...
• The PSU LED for the second PSU should light up green only if a second PSU is installed in the switch for redundancy and Hot-Swap ability and it is connected to a power source. If two PSUs Mellanox Technologies...
If the PSU LEDs are not green, this indicates a problem with the power supplies. Only run the switch if at least one of the PSU LEDs is green. Figure 3: Power, Fan, and System LEDs STATUS PSU 1 PSU 2 Mellanox Technologies...
No power to the port. Solid Green Logical link up Flashing Green Data activity flashing speed ≈ data transfer speed Orange Physical link up Flashing Orange A problem with the physical link Usually the SM is down or not enabling the port. Mellanox Technologies...
Off – The fan unit is not receiving any power. Check that the fan unit is properly and completely inserted. Figure 3 on shows the explanation of the PSU Status LED colors. page 15 Mellanox Technologies...
On the connector side panel under the system LEDs is a reset button. This reset button requires a tool to be pressed. DO NOT use a sharp pointed object such as needle or push pin for pressing the Reset button. Sharp objects can cause damage, use a flat object to press this reset button. Mellanox Technologies...
• Power side inlet to connector side outlet The air flow is specified in the product model number. See “Mellanox Part Numbering Legend” on page 11.. On the switch and fan modules the air flow direction can be seen on the power side panel. Mellanox Technologies...
The Connector side of the switch has 36 QSFP ports. These are placed in two rows, 18 ports to a row. The ports are labelled as shown in Figure 8. The bottom row ports are flipped from the top row. See Figure 9 for bottom row - top row port orientation. Mellanox Technologies...
An HAR 000028 harness is included in the package to connect to a DB9 connection on a host PC. Connecting to a local host PC and following the instructions in the Installation Guide, “Configuring the Switch for the First Time”, must be done Mellanox Technologies...
The switch with the FabricIt management module has an extra Ethernet connector below the MGT connector shown in Figure 10. Note: These connector(s) are not found in unmanaged (externally managed) switches. Each Ethernet connector gets connected to Ethernet switches. These switches must be configured to 10/100M auto-negotiation. Mellanox Technologies...
MIS000085 – kit for a standard switch in standard rack 50cm to 80cm Note: Kit # MIS000085 is for standard depth switches only. • 1) harness; HAR000028 – Harness for IS5025, IS5030 and IS5035 switches • 1) CD Mellanox Technologies...
8cm (3 inches) of clearance around the ventilation openings. 3. Stacking the Chassis The chassis should not be stacked on any other equipment. If the chassis falls, it can cause bodily injury and equipment damage. Mellanox Technologies...
Page 25
9. Equipment Disposal Disposal of this equipment should be in accordance to all national laws and regula- tions. 10. Local and National Electrical Codes This equipment should be installed in compliance with local and national electrical codes. Mellanox Technologies...
19” rack whose vertical supports are between 500mm and 800mm apart. The standard depth switch uses the MIS000085 rail kit for installation in a 19” rack whose vertical supports are between 500mm and 800mm apart. Mellanox Technologies...
For standard depth switches rail kit # MIS00085 is to be used for racks from 60cm to 80cm deep. Parts included in the rail kit: • 2 rails • 18 recessed flat head screws • 2 rail slides • 8 caged nuts • 2 brackets • 8 pan head screws M6 Mellanox Technologies...
Choose which side of the switch you want even with the rack vertical support. Either the side with the power supply units or the side with the IB connectors can be even with one of the vertical rack supports. Things to consider before choosing where to mount the rails and rail slides. Mellanox Technologies...
Will the connector side be recessed past other equipment in the rack and will this be prob- lematic? Screw the brackets onto the switch. Use the flat head screws to connect the bracket. There are two options for mounting the bracket. One option will place the switch even with the ver- Mellanox Technologies...
The side of the switch with these brackets will be the side that is even with or very close to the vertical rack support. Screw the rails onto the switch. Use 5 flat head screws to connect each rail to the switch. Mellanox Technologies...
3-wire AC power cords including a safety ground and rated for 15A or higher. Caution: The switch platform will automatically power on when AC power is applied. There is no power switch. Check all boards, power supplies, and fan tray modules for proper insertion before plugging in a power cable. Mellanox Technologies...
Power supply units have directional air flows similar to the fan module. The Fan module airflow must coincide with the airflow of all of the PSUs. If the PSU airflow direction is different from the fan module airflow direction the system will shut down. Mellanox Technologies...
Insert the power cord into the supply connector. Insert the other end of the power cord into an outlet of the correct voltage. The green PSU indicator should light. If not, repeat the whole procedure to extract the PSU and re-insert it. Mellanox Technologies...
This switch can operate indefinitely with one of the three fans in the fan module inoperable so long as the ambient temperature is below 45° Celsius. Operation without a fan unit should not exceed two minutes. During fan hot-swap, if both indicators are OFF then the fan unit is discon- nected. Mellanox Technologies...
The green fan status indicator should light. If not, extract the fan unit and reinsert it. After two unsuccessful attempts to install the fan unit, power off the switch before attempting any system debug. To disassemble the switch from the rack: Mellanox Technologies...
According to the WEEE Directive 2002/96/EC, all waste electrical and electronic equipment (EEE) should be collected separately and not dis- posed of with regular household waste. Dispose of this product and all of its parts in a responsible and environ- mentally friendly way. Mellanox Technologies...
• chassis manager and system BIST • SNMP agent, 3rd party tool integration • GUI • Subnet Manager (SM) The chassis manager will give the user access to: • switch temperatures • power supply voltages • fan unit information • power unit information Mellanox Technologies...
=> Downloads => Firmware. Be sure to read and follow all of the instructions regarding the updating of the firmware on your switch system. Firmware for the HCA cards connected to this switch system can be downloaded from the same site. Mellanox Technologies...
Make sure all subnet ports are in the active state. One way to check this is to run opensm, the Subnet Manager. [root@mymach]> /etc/init.d/opensmd start opensm start [ OK ] Make sure the local ports are active by running ‘ibv_devinfo’. Mellanox Technologies...
Page 42
MT_0D00110012 Burn the Infiniscale IV switch: flint -d /dev/mst/SW_MT48438_lid-0x0003 -i ./fw-IS4-rel-7_4_0-Q_A1.bin -qq b 4.3.2.2 Reprogramming the Switch Through The I2C Port This interface is for FAEs and advanced users This interface is for Debug and Troubleshooting only. only. Mellanox Technologies...
With the switch connected to a computer through the MTUSB, it is now possible to reprogram the SPI Flash memory. 4.3.2.3 Instructions for Reprogramming Through the I2C Port: Make sure that: the MTUSB I2C cable is connected to the switch and USB cable is connected to a computer Mellanox Technologies...
A typical InfiniBand installation using the OFED package will run the OpenSM subnet manager at system start up after the OpenIB drivers are loaded. This automatic OpenSM is resident in mem- ory, and sweeps the InfiniBand fabric approximately every 5 seconds for new InfiniBand adapters to add to the subnet routing tables. Mellanox Technologies...
You will also need to download and unzip the firmware binary image. This is provided in the Mel- lanox Web site at: http://www.mellanox.com => Support > Download Center and go to the InfiniS- cale IV Switch systems. Click in the Table for the firmware image that you need. Mellanox Technologies...
Make sure that the latest FW version is installed on both the HCA and the switch. If media adapters are used, check that the all connections are good, tight, and secure. The activity LED does not come on: Check that the Subnet Manager has been started. Mellanox Technologies...
Page 47
FLASH: 16 MB NAND: 1024 MiB PCI: Bus Dev VenId DevId Class Int PCIE0: link is not up. PCIE1: successfully set as root-complex 01 00 15b3 bd34 0c06 00 Net: ppc_4xx_eth0, ppc_4xx_eth1 Hit Ctrl+B to stop autoboot: 0 Mellanox Technologies...
EU: EN61000-3-3, Class A Japan: VCCI, Class A Environmental: EU: IEC 60068-2-64: Ran- dom Vibration EU: IEC 60068-2-29: Shocks, Type I / II EU: IEC 60068-2-32: Fall Test Acoustic: ISO 7779 Sound power level: ETS 300 753 76 dB(A) Mellanox Technologies...
Page 50
Simultaneous wire-speed any port to any Hot-Swappable: Fan Module and port 1+1 Redundant: Power Supplies Switching Capacity: 1.44Tb/s for 36 ports QDR 720 Gb/s for 36 ports DDR 720 Gb/s for 18 ports QDR 360 Gb/s for 18 ports DDR Mellanox Technologies...
EU: EN61000-3-3, Class A Japan: VCCI, Class A Environmental: EU: IEC 60068-2-64: Ran- dom Vibration EU: IEC 60068-2-29: Shocks, Type I / II EU: IEC 60068-2-32: Fall Test Acoustic: ISO 7779 ETS 300 753 Sound power level: 76 dB(A) Mellanox Technologies...
Page 52
Simultaneous wire-speed any port to any Hot-Swappable: Fan Module and port Power Supplies 1+1 Redundant: Switching Capacity: 1.44Tb/s for 36 ports QDR 720 Gb/s for 36 ports DDR 720 Gb/s for 18 ports QDR 360 Gb/s for 18 ports DDR Mellanox Technologies...
EU: EN61000-3-3, Class A Japan: VCCI, Class A Environmental: EU: IEC 60068-2-64: Ran- dom Vibration EU: IEC 60068-2-29: Shocks, Type I / II EU: IEC 60068-2-32: Fall Test Acoustic: ISO 7779 ETS 300 753 Sound power level: 76 dB(A) Mellanox Technologies...
Page 54
§ 15.21 Statement Warning! Changes or modifications to this equipment not expressly approved by the party responsible for compliance (Mellanox Technologies) could void the user's authority to operate the equipment. §15.105(a) Statement NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules.
Page 55
A.2.3 ICES Statements (Canada) Class A Statement: “This Class A digital apparatus complies with Canadian ICES-003. Cet appareil numérique de la classe A est conforme à la norme NMB-003 du Canada.” Mellanox Technologies...
Page 56
Class A Device This device is registered for EMC requirements for industrial use. The seller or buyer should be aware of this. If this type was sold or purchased by mistake, it should be replaced with a residential-use type. Mellanox Technologies...
Parts included in the installation kit: • 2 rails • 8 recessed flat head screws • 2 rail slides • 4 caged nuts • 8 pan head screws M6 • 2 metal washers Figure 30: Rack Installation Kit Parts rail rail slide Mellanox Technologies...
IB connectors can be even with the vertical rack support. The other side of the switch will be further inside of the rack. Things to consider before choosing where to mount the rails and rail slides. Mellanox Technologies...
Clip the nuts into the holes in the rack you will be using to connect the rail slides. Check that both sides of the switch are in the same position number on the rack. Mellanox Technologies...
# MIS00079 is to be used for racks from 38cm to 50cm deep including the iDataPlex rack. For standard depth switches rail kit # MIS00085 is to be used for racks from 60cm to 80cm deep. Mellanox Technologies...
Choose which side of the switch you want close to the rack vertical support. Either the side with the power supply units or the side with the IB connectors can be close to one of the ver- tical rack supports. Things to consider before choosing where to mount the rails and rail slides. Mellanox Technologies...
Do you want the connector side recessed in the rack to allow for a larger cable bending radius? Will the connector side be recessed past other equipment in the rack and will this be prob- lematic? Mellanox Technologies...
Clip the 4 caged nuts into the holes in the rack you will be using to connect the rail slides. Check that both sides of the switch, left and right, are the same level in the rack. Figure 39: Inserting the Caged Nuts The caged nuts are separated by a single space Mellanox Technologies...
Rev 2.9 Figure 40: Put the Cables Through the 1U Space Install the rail slides placing the power cords through the slots. Figure 41: Placing the Power Cords Mellanox Technologies...
Place to put the power cable. In this position you will use 3 In this position you will use 4 flat head screws flat head screws This configuration also allows you to put the power cord through the bracket. Mellanox Technologies...
Slide the switch into the rack and catch the rail slides into the rails. Note: Before putting the switch in place guide the power cords through the slots in the brackets. Figure 46: Power Cord Through the Bracket Mellanox Technologies...
Warning: Any yellow status LEDs is cause for concern and must be dealt with immediately. It can take up to 3 minutes to boot up, during which time the status LED may indicate red. You can start connecting all of the cables to the switch. Mellanox Technologies...
+3.3 V Power supply transmitter Vcc 1 +3.3 V Power Supply LPMode Low Power Mode Ground Tx3p Transmitter Non-Inverted Data Input Tx3n Transmitter Inverted Data Input Ground Tx1p Transmitter Non-Inverted Data Input Tx1n Transmitter Inverted Data Input Ground Mellanox Technologies...
Page 69
QDR InfiniBand Switch Platform User Manual Rev 2.9 18.35 View into Rear of Connector 8.50 18.35 View into Front of Cage 8.50 Mellanox Technologies...
Power cord Type G for UK ACC000208 Power cord Type D for India ACC000209 Power cord Type I for China ACC000210 Power cord Type J for Switzerland ACC000211 Power cord Type B for Japan, ACC000212 Power cord Type I for Australia ACC000213 Mellanox Technologies...
7. Montage et entretien sur baie Lorsque ce produit est monté ou entretenu sur baie, il faut prendre des précau- tions spéciales pour s’assurer que le système reste stable. En général, il faut remplir la baie avec du matériel de bas en haut. Mellanox Technologies...
Page 73
L’élimination de ce matériel doit s’effectuer dans le respect de toutes les légis- lations et réglementations nationales en vigueur. 10. Codes électriques locaux et nationaux Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux. Mellanox Technologies...
Vorsichtsmaßnahmen zu ergreifen, um die Stabilität des Systems zu gewährleisten. Im Allgemeinen sollten Sie das Gestell von unten nach oben mit Geräten füllen. 8. Geräteinstallation Diese Gerät sollte nur von geschultem und qualifiziertem Personal installiert, ausgetauscht oder gewartet werden. Mellanox Technologies...
Page 75
QDR InfiniBand Switch Platform User Manual Rev 2.9 9. Geräteentsorgung Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestimmungen erfolgen. 10. Regionale und nationale elektrische Bestimmungen Dieses Gerät sollte unter Beachtung der regionalen und nationalen elektrischen Bestimmungen installiert werden. Mellanox Technologies...
Al instalar o realizar el mantenimiento de este aparato en un bastidor, es preciso adoptar precauciones especiales para garantizar que el sistema se mantenga estable. En general, en un bastidor, los equipos se deben insta- lar comenzando desde abajo hacia arriba. Mellanox Technologies...
Page 77
300 voltios nominal, con cobertura protectora aislante de PVC. El cable debe tener un enchufe moldeado con capuchón de 250 voltios nomi- nal, 10 A. ADVERTENCIA: Alta corriente de fuga. Es esencial efectuar la conexión a tierra antes de conectar la alimentación. Mellanox Technologies...
Page 78
(EEE) se deben recolectar por separado y no se deben eliminar junto con residuos domésticos. Al deshacerse de este producto y de todas sus partes, hágalo de una manera res- ponsable y respetuosa con el medio ambiente. Mellanox Technologies...
Page 79
Do not connect this unit to any outlet that is not fully grounded! • Finland “Laite on liitettävä suojamaadoituskoskettimilla var- ustettuun pistorasiaan” • Norway “Apparatet må tilkoples jordet stikkontakt” Unit is intended for connection to IT power systems for Norway only. • Sweden “Apparaten skall anslutas till jordat uttag.” Mellanox Technologies...