Mellanox Technologies BridgeX BX4010 User Manual
Mellanox Technologies BridgeX BX4010 User Manual

Mellanox Technologies BridgeX BX4010 User Manual

Gateway platform ib to ethernet/fibre channel
Table of Contents

Advertisement

Quick Links

®
BridgeX
BX4010 Gateway Platform IB to
Ethernet/Fibre Channel User Manual
P/N:MTB4010B-PC
Rev 1.5
www.mellanox.com

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the BridgeX BX4010 and is the answer not in the manual?

Questions and answers

Summary of Contents for Mellanox Technologies BridgeX BX4010

  • Page 1 ® BridgeX BX4010 Gateway Platform IB to Ethernet/Fibre Channel User Manual P/N:MTB4010B-PC Rev 1.5 www.mellanox.com...
  • Page 2 © Copyright 2010. Mellanox Technologies. All rights reserved. Mellanox, BridgeX, ConnectX, Virtual Protocol Interconnect, InfiniBlast, InfiniBridge, InfiniHost, InfiniRISC, InfiniScale, and InfiniPCI are registered trademarks of Mellanox Technologies, Ltd. CORE-Direct, FabricIT, and PhyX are trademarks of Mellanox Technologies, Ltd. All other trademarks are property of their respective owners.
  • Page 3: Table Of Contents

    Chapter 4 Troubleshooting Appendix A Specifications Appendix B EMC Certification Statements B.1 FCC Statements (USA) Appendix C Interface Connector Pinouts C.1 CX4 Interface C.2 SFP+ Interface C.3 RS232 to RJ45 Connector Interface Appendix D Replacement Parts Ordering Numbers Mellanox Technologies...
  • Page 4 Appendix E Avertissements de sécurité d’installation (French) Appendix F Installation - Sicherheitshinweise (German) Appendix G Advertencias de seguridad para la instalación (Spanish) Appendix H SFP+ Modules H.1 Product Features H.2 Applications H.3 Dimensions H.4 Pin Descriptions H.5 Power Dissipation Mellanox Technologies...
  • Page 5: List Of Tables

    Installation Kit Parts Table 10: MTB4010B-NC Specification Data Table 11: MTB4010B-PC Specification Data Table 12: Bridge Certification Status Table 13: InfiniBand 4X Connector Pinout Table 14: SFP+ Pinout Table 15: Replacement Parts Ordering Numbers Table 16: SFP+ Pinout Mellanox Technologies...
  • Page 6: List Of Figures

    Figure 17: Fan Unit Pulled Out Figure 18: CX4 Connector Pinout Figure 19: SFP+ Connector Pinout - Rear View of Module With Pin Placement Figure 20: RJ45 Connector Pinout Figure 21: Rear View of Module With Pin Placement Mellanox Technologies...
  • Page 7: Revision History

    1.3.3 Removed redundant section regarding Management and Software Sept. 2009 1.3.2 Replaced references from SW UM to FabricIT-BXM Sept. 2009 1.3.1 Updated revision table July 2009 Implemented new template July 2009 Verified installation kit July 2009 Initial Release Mellanox Technologies...
  • Page 8: About This Manual

    InfiniBand (IB) is used. Downlink refers to the Local Area Network (LAN) or Storage Area Network (SAN) switch facing ports, where Fibre Channel (FC) and/or Ethernet (EN) are used. Caution: This symbol indicates the possibility of physical injury to the user or installer. Mellanox Technologies...
  • Page 9: Gateway Products Covered In This User Manual

    SFP+ downlink ports with CPU, RoHS5 Mellanox Part Numbering Legend Table 4 - Part Numbering Legend Place Field Decoder Mellanox Technologies System Type = BridgeX based Bridge System Gateway protocols 4 = VPI Uplinks For future use # of BridgeX devices...
  • Page 10: Chapter 1 Overview

    Mellanox to enable FCoIB bridging. Contact your Mellanox representative for more details. Each gateway port group is assigned a color, in the figures in this document, to easily associate the uplink ports to the corresponding downlink ports of the gateway port group. Mellanox Technologies...
  • Page 11: Features

    • Down link ports within a group must run the same protocol (Ethernet or Fibre Channel) • A down link port group running Ethernet must run the same speed on all ports within the group (10Gbps or 1Gbps) Mellanox Technologies...
  • Page 12 Indicators • Per port status LEDs; Link Activity • System status LEDs: system, fans, power supply Management Options • 1 – 1000BASE-T Ethernet port Power Supply • Dual redundant slots • 1 PSU is required for device functioning Mellanox Technologies...
  • Page 13: Serial Number And Product Version Information

    0 0 : 0 2 :C 9 : 0 2 :3 C :A 6 Gateway Port Groups Figure 3: Example of a Setup Showing a Port Group Uplink MTS3600 InfiniBand Switch Hybrid QSFP to CX4 cable Gateway IOIO ® Mellanox BX4010 PSU 1 PSU 2 Downlink Mellanox Technologies...
  • Page 14: Chapter 2 Installation And Basic Operation

    SR or LR modules. The gateways with single controllers have eight downlink ports while gateways with dual controllers have 16 downlink ports. The downlink ports are labelled alpha-numerically and the diagrams in this man- Mellanox Technologies...
  • Page 15: Uplink Ports

    Mellanox. The OPNs for the approved Mellanox modules are MFM1T02A-SR and MFM1T02A-LR. The figure below shows the Mellanox approved SFP+ module. SR and LR modules not recommended by Mellanox may not work with the adapter. Figure 5: SFP+ Transceiver Module Mellanox Technologies...
  • Page 16: Figure 6: Module With Locking Mechanism Closed

    3. Insert the module into the adapter card module cage. 4. Close the locking Mechanism. To remove the module from the cage: 1. Unlock the locking mechanism by opening the handle. 2. Pull the module out of the cage. Mellanox Technologies...
  • Page 17: Status Leds

    System Status Indicators Sys. Fans Gateway Fan Status Indicators PSU 1 PSU 2 PSU Status Indicators 2.2.6.1 System Status Indicators The System Status Indicators are located next to the Ethernet ports. The following status condi- tions are possible. Mellanox Technologies...
  • Page 18: Table 6: System Status Led Configurations

    Error – One or more fans is not operating. Repair or replace the fan unit. See Section 4, “Troubleshooting,” on page 37. yellow-on Off – The fan unit is not receiving any power. Check that the fan unit is properly and com- pletely inserted. Mellanox Technologies...
  • Page 19: Table 8: Psu Status Led Configurations

    DC supply. Green-on off Error – This power supply unit is NOT deliver- ing the correct DC supply. yellow-on Off – This power supply unit is NOT receiving any input power, or this PSU is not installed. Mellanox Technologies...
  • Page 20: I2C Connector

    This RS232 connection provides access for local management through connection to a host. The BX series gateways can be connected to the RJ45 connector of a computer. This connection is needed for the initial configuration of the gateway. For the initial configuration procedure for the gateway see the Installation Guide. Mellanox Technologies...
  • Page 21: Package Contents

    If the Flash memory is corrupted the Gateway may fail to boot. Caution: The gateway platform will automatically power on when AC power is applied. There is no power switch. Check all boards, power supplies, and fan tray mod- ules for proper insertion before plugging in a power cable. Mellanox Technologies...
  • Page 22: Installation Safety Warnings

    6. Copper InfiniBand Cable Connecting/Disconnecting Copper InfiniBand cables are heavy and not flexible, as such they should be carefully attached to or detached from the connectors. Refer to the cable manu- facturer for special warnings and instructions. Mellanox Technologies...
  • Page 23 20 A must be incorporated in the building wiring. 12. Equipment Installation This equipment should be installed, replaced, or serviced only by trained and qualified personnel. 13. Equipment Disposal Disposal of this equipment should be in accordance to all national laws and reg- ulations. Mellanox Technologies...
  • Page 24: Mechanical Installation

    The gateway platform can be rack mounted for installation in a standard 19” rack. Front and back are arbitrary, the rack kit can be mounted so that one side is even with the vertical rack support and the other side is recessed into the rack. Mellanox Technologies...
  • Page 25: Table 9: Installation Kit Parts

    16 recessed flat head screws 4 pan head screws 2 metal washers 4 sets of caged nuts and bolts for the mounting plates Note: There may be more screws in the kit than are required for installing the gateway. Mellanox Technologies...
  • Page 26: Figure 10: Installation Kit Parts

    • Do you want the connector side recessed in the rack to allow for a larger cable bending radius? • Will the connector side be recessed past other equipment in the rack and will this be problem- atic? Mellanox Technologies...
  • Page 27: Figure 11: Which Side Of The Rack Do You Want The Connectors

    Note: If you need room to bring the power cord from the other side of the rack, recess the gate- way and run the power cord through the bracket.The installation kit can be reversed so that the bracket can be installed on either side of the gateway the power side or the connector side. Mellanox Technologies...
  • Page 28: Figure 12: Making Room For The Power Cord

    5. Clip 4 caged nuts into the holes in the rack you will be using to connect the rail slides. Check that both sides of the gateway, left and right, are the same level in the rack. Mellanox Technologies...
  • Page 29: Grounding The Gateway

    4mm diameter is recommended for grounding this device. The chassis is concurrently grounded through each of the PSUs. Only connect the PSU cords to properly grounded outlets. Do not rely on the PSU grounds. It is absolutely necessary to physically ground the casing. Make sure the con- Mellanox Technologies...
  • Page 30: Disassembling The Gateway From The Rack

    Warning: Any yellow status LEDs is cause for concern and must be dealt with immediately. It can take up to 2 minutes to boot up, during which time the status LED may indicate red. Mellanox Technologies...
  • Page 31: Shut Down Procedure

    Mellanox representative for assistance. To run the gateway through a power cycle 1. Run: (config) # reload halt 2. Wait long enough to let the gateway halt, and verify there is no ping to it. 3. Unplug the power cord. Mellanox Technologies...
  • Page 32: Extracting And Inserting The Power Supply Unit

    3. Remove the PSU. 2.4.7.2 To insert a PSU 1. Make sure the mating connector of the new unit is free of any dirt and/or obstacles. Do not attempt to insert the PSU with a power cord connected to it. Mellanox Technologies...
  • Page 33: Cable Installation

    Cables for cable recommendations regarding Mellanox approved cables and recommended maxi- mum cable lengths. Cables in the bottom row should be inserted up side down in relation to the how the cables are inserted in the top row. Mellanox Technologies...
  • Page 34: Extracting And Inserting The Fan Unit

    2. Pull the fan unit out by pulling on the thumbscrews. As the fan unseats, the fan status indicator will turn off. To insert a FAN Unit: 1. Make sure the mating connector of the new unit is free of any dirt and/or obstacles. Mellanox Technologies...
  • Page 35 The green fan status indicator should light. If not, extract the fan unit and rein- sert it. After two unsuccessful attempts to install the fan unit, power off the gateway before attempting any system debug. 3. Turn the two thumb screws to lock the fan unit in place. Mellanox Technologies...
  • Page 36: Chapter 3 Management And Software

    GUI interface. The GUI interface can be run through the Web. The standard management module allows for managing the gateway using the Web interface or the command line interface and software supplied on the management chip. See the FabricIT BXM Management Software CLI User’s Manual for instructions and commands available. Mellanox Technologies...
  • Page 37: Chapter 4 Troubleshooting

    3. Make sure that the latest FW version is installed on both the HCA/ NIC cards and the gateway. 4. If media adapters are used, check that the all connections are good, tight, and secure. The activity LED does not come on: t the Subnet Manager has been started. Check tha Mellanox Technologies...
  • Page 38 2. Wait long enough to let the gateway halt, and verify there is no ping to it. 3. Unplug the power cord. 4. Plug in the power cord. Gateway Fails to Boot: 1. Reburn the firmware to the Flash memory, using the MFT tools. Mellanox Technologies...
  • Page 39: Appendix A Specifications

    ICES-003 Class A VCCI Class A AS/NZS CISPR 22 Class A CISPR 22 Class A EN 55024 EN 300386 Environmental: EU: IEC 60068-2-64: Ran- dom Vibration EU: IEC 60068-2-29: Shocks, Type I / II EU: IEC 60068-2-32: Fall Test Mellanox Technologies...
  • Page 40: Table 11: Mtb4010B-Pc Specification Data

    Ethernet 1, 10 Gb/s Passive Cables 2/4/8G Fibre Channel Max Power: 93.39 W Connectors and Typ Power: 90.17 W Cabling: Uplink: CX4 Cable Temperature: 0° to 55° Celsius Downlink: SFP+ Humidity: 10% - 90% non-condensing Fibre Channel Protocol Support Regulatory Compliance Mellanox Technologies...
  • Page 41 Scalability and Performance Features Switching Perfor- Simultaneous wire-speed any port to Hot-Swappable: Fan Module and Power mance: any port 1+1 Redundant: Supplies 48K Unicast Addresses Max. per Power Supplies Addressing: Subnet Bridging Capacity 16K Multicast Addresses per Subnet Mellanox Technologies...
  • Page 42: Appendix B Emc Certification Statements

    Statement Warning! Changes or modifications to this equipment not expressly approved by the party responsible for compliance (Mellanox Technologies) could void the user's authority to operate the equipment. §15.105(a) Statement NOTE: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules.
  • Page 43 (Translation - "This is a Class A product based on the standard of the Voluntary Control Council for Interference by Information Technology Equipment (VCCI). If this equipment is used in a domestic environment, radio interference may occur, in which case the user may be required to take corrective actions.") Mellanox Technologies...
  • Page 44: Appendix C Interface Connector Pinouts

    Rx_B7 IBtxIn(3) Rx_A6 Rx_B6 IBtxOn(3) Tx_A6 Tx_B6 IBtxOp(3) Tx_A7 Tx_B7 IBtxOn(2) Tx_A4 Tx_B4 IBtxOp(2) Tx_A5 Tx_B5 IBtxOn(1) Tx_A2 Tx_B2 IBtxOp(1) Tx_A3 Tx_B3 IBtxOn(0) Tx_A0 Tx_B0 IBtxOp(0) Tx_A1 Tx_B1 G1-G6, G9, Signal Ground H1-H2 Sense-3.3V SENSE_P1 SENSE_P2 MC_POWER MC_POWER Mellanox Technologies...
  • Page 45: C.2 Sfp+ Interface

    Receiver Ground (Common with Transmitter Ground) VeeR Receiver Ground (Common with Transmitter Ground) Receiver Inverted DATA out. AC Coupled Receiver Non-inverted DATA out. AC Coupled VeeR Receiver Ground (Common with Transmitter Ground) VccR Receiver Power Supply VccT Transmitter Power Supply Mellanox Technologies...
  • Page 46 MOD_ABS pulls line low to indicate module is plugged in. e. LOS is open collector output. Should be pulled up with 4.7kΩ – 10kΩ on host board to a voltage between 2.0V and 3.6V. Logic 0 indicates normal operation; logic 1 indicates loss of signal. Mellanox Technologies...
  • Page 47: C.3 Rs232 To Rj45 Connector Interface

    C.3 RS232 to RJ45 Connector Interface Figure 20: RJ45 Connector Pinout DSR Data Set Ready / Ring indicator Data Carrier Detect DTR Data Terminal Ready Signal Ground RXD Receive Data TXD Transmit Data CTS Clear to Send RTS Request to Send Mellanox Technologies...
  • Page 48: Appendix D Replacement Parts Ordering Numbers

    Power cord Type G for UK ACC000208 Power cord Type D for India ACC000209 Power cord Type I for China ACC000210 Power cord Type J for Switzerland ACC000211 Power cord Type B for Japan ACC000212 Power cord Type I for Australia ACC000213 Mellanox Technologies...
  • Page 49: Appendix E Avertissements De Sécurité D'installation (French)

    Les câbles InfiniBand en cuivre sont lourds et ne sont pas flexibles, il faut donc faire très attention en les branchant et en les débranchant des connecteurs. Consultez le fab- ricant des câbles pour connaître les mises en garde et les instructions spéciales. Mellanox Technologies...
  • Page 50 L’élimination de ce matériel doit s’effectuer dans le respect de toutes les législations et réglementations nationales en vigueur. 13. Codes électriques locaux et nationaux Ce matériel doit être installé dans le respect des codes électriques locaux et nationaux. Mellanox Technologies...
  • Page 51 à 3 conducteurs, d’un diamètre de fil minimum de 0,75 mm2, d’une tension nominale de 300 V, avec une gaine isolée en PVC. Le cordon doit avoir une fiche moulée d’une tension nominale de 250 V et d’une intensité nominale de 10 A. Mellanox Technologies...
  • Page 52: Appendix F Installation - Sicherheitshinweise (German)

    Rack-Montage und Wartung Wenn dieses Produkt in einem Rack montiert oder gewartet wird, sind besondere Vor- sichtsmaßnahmen zu ergreifen, um die Stabilität des Systems zu gewährleisten. Im Allgemeinen sollten Sie das Gestell von unten nach oben mit Geräten füllen. Mellanox Technologies...
  • Page 53 Für die europäischen Zusammenhang, wählen Sie ein Netzkabel, das international harmonisiert und der Aufschrift "<HAR>", 3 - Leiter, mindestens 0,75 mm2 Draht, bewertet mit 300 V, mit einem PVC-Mantel isoliert. Das Kabel muss eine angespritztem Stecker bewertet bei 250 V, 10 A. " Mellanox Technologies...
  • Page 54 Kabel für den Anschluss an das Gerät RS232-und Ethernet-Schnittstellen müssen UL zertifiziert Typ DP-1 oder DP-2. (Hinweis-, wenn nicht mit Wohnsitz in LPS-Schal- tung) Überstromschutz: Eine leicht zugängliche Auflistung Abzweigleitung Überstrom- Schutzeinrichtung 20 A bewertet werden müssen in dem Gebäude Verkabelung. Mellanox Technologies...
  • Page 55: Appendix G Advertencias De Seguridad Para La Instalación (Spanish)

    Si se dispone de una fuente de alimentación vacía, no utilizar el producto si su tapa está quitada o no está bien cerrada. 6. Cuando hay rayos: peligro de descarga eléctrica No utilizar el equipo ni conectar o desconectar cables durante períodos de activ- idad de rayos. Mellanox Technologies...
  • Page 56 La eliminación definitiva de este equipo se debe efectuar conforme a todas las leyes y reglamentaciones nacionales. 12. Códigos eléctricos locales y nacionales Este equipo se debe instalar conforme a los códigos eléctricos locales y nacionales. 13. Cable de alimentación homologado por UL y con certificación CSA Fuga > 3,5 mA Mellanox Technologies...
  • Page 57 UL tipo DP-1 o DP-2. (Nota: cuando residen en circuito no de tipo LPS) Protección contra sobrecargas: Al cableado del edificio se debe incorporar un disposi- tivo de protección contra sobrecargas de circuito derivado, de fácil acceso, con una cor- riente nominal de 20 A. Mellanox Technologies...
  • Page 58 (EEE) se deben recolectar por separado y no se deben eliminar junto con residuos domésticos. Al deshacerse de este producto y de todas sus partes, hágalo de una manera responsable y respetuosa con el medio ambiente. Mellanox Technologies...
  • Page 59: Appendix H Sfp+ Modules

    • Uncooled 850nm VCSEL laser • Receiver limiting electrical interface • Duplex LC connector • Built-in digital diagnostic functions • RoHS-6 compliant (lead-free) H.2 Applications • 10GBASE-SR/SW 10G Ethernet H.3 Dimensions 56.50 Mellanox Technologies 13.60 14.80 13.70 MTSFPP-SR Module 1.55 0.50 13.90 8.50 11.85...
  • Page 60: H.4 Pin Descriptions

    2-wire Serial Interface Clock Line MOD_ABS Module Absent. Grounded within the module No connection required RX_LOS Loss of Signal indication. Logic 0 indicates normal operation. No connection required VeeR Receiver Ground (Common with Transmitter Ground) VeeR Receiver Ground (Common with Transmitter Ground) Mellanox Technologies...
  • Page 61: H.5 Power Dissipation

    LOS is open collector output. Should be pulled up with 4.7kΩ – 10kΩ on host board to a voltage between 2.0V and 3.6V. Logic 0 indicates normal operation; logic 1 indicates loss of signal. H.5 Power Dissipation Max power dissipation 1.00 W Mellanox Technologies...

Table of Contents