Mellanox Technologies InfiniBand FDR SwitchX SX6518-NR Hardware User Manual
  1. Manuals
  2. Brands
  3. Mellanox Technologies Manuals
  4. Switch
  5. SX6518-NR
  6. Hardware user manual
Mellanox Technologies InfiniBand FDR SwitchX SX6518-NR Hardware User Manual

Mellanox Technologies InfiniBand FDR SwitchX SX6518-NR Hardware User Manual

324-port switch platform
Hide thumbs Also See for InfiniBand FDR SwitchX SX6518-NR:
Table of Contents

Advertisement

Quick Links

Download this manual
324-Port InfiniBand FDR SwitchX®
Switch Platform Hardware User
Manual
PN:SX6518-NR, SX6518-6R
Rev 2.9
www.mellanox.com
Mellanox Technologies Confidential

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the InfiniBand FDR SwitchX SX6518-NR and is the answer not in the manual?

Questions and answers

Related Manuals for Mellanox Technologies InfiniBand FDR SwitchX SX6518-NR

Summary of Contents for Mellanox Technologies InfiniBand FDR SwitchX SX6518-NR

  • Page 1 324-Port InfiniBand FDR SwitchX® Switch Platform Hardware User Manual PN:SX6518-NR, SX6518-6R Rev 2.9 Mellanox Technologies Confidential www.mellanox.com...
  • Page 2 ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX TECHNOLOGIES TO FULLY QUALIFY THE PRODUCT (S) AND /OR THE SYSTEM USING IT . THEREFORE , MELLANOX TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE HIGHEST QUALITY .

  • Page 3: Table Of Contents

    2.10.4 Fan Modules ..........65 Mellanox Technologies Confidential...
  • Page 4 Appendix A Specification Data ........96 Mellanox Technologies Confidential...
  • Page 5 G.13 Japan VCCI Class A Statement ....... 144 Mellanox Technologies Confidential...
  • Page 6 Table 23: Replacement Parts Ordering Numbers ....... . 106 Mellanox Technologies Confidential...

  • Page 7: List Of Figures

    Figure 34: Ground Connection ............51 Mellanox Technologies Confidential...
  • Page 8 Figure 69: Spine Side Panel Display Status Indications ........84 Figure 70: Management Module Status Indications for Normal Operation ....84 Mellanox Technologies Confidential...
  • Page 9 Figure 74: QSFP Connector Male and Female Views ........105 Mellanox Technologies Confidential...

  • Page 10: Revision History

    Appendix A, “Specification Data,” on page 96 Oct 27, 2014 Updated: • Max. Air Flow numbers in Appendix A, “Specification Data,” on page 96 • N+1 configuration in Section 1.4.2, “1670W Power Supply Units,” on page 18 Mellanox Technologies Confidential...
  • Page 11 2012 Fixed height of the shelf from 1U to 1.5 U in Spec Sheet and on page 35 October, New power numbers 2012 Changed Part numbering Legend Fixed height of the shelf from 1U to 1.5 U Mellanox Technologies Confidential...

  • Page 12: About This Manual

    Look up the relevant SwitchX®-based switch system/ series release note file. Mellanox MLNX-OS® User This document contains information regarding config- Manual for VPI uring and managing Mellanox Technologies SwitchX® switch platforms listing all of the commands available through MLNX-OS with explanations and examples. Mellanox Technologies Confidential...

  • Page 13: Mellanox Part Numbering Legend

    Data Rate B = 40Gb/s Ethernet F = FDR, T = FDR10, Q = QDR, D = DDR Separator # Power Supplies N = N+N redundant 1=1PSU, 2=2PSUs, ... Chip Generation R – SwitchX® S – SwitchX®-2 Mellanox Technologies Confidential...

  • Page 14: Chapter 1 Overview

    The serial number, GUID identifier and product version information are found on the label attached to the pull-out tab below the Mellanox logo on the spine side of the chassis. Figure 1: Product Label S/N: MT0806X01504 P/N: MSX6506-3R GUID: 0002C9020027051C Rev:A1 MAC: 0002C9020040 Made in Israel Mellanox Technologies Confidential...

  • Page 15: Management Module Mac

    The system power interface is comprised of 6 C14 plugs, arranged in a row. The switch ships in a minimum base configuration plus additional modules depending on the chosen customer configuration. Optional modules included: • Leaf boards • Management modules • Spine boards Mellanox Technologies Confidential...

  • Page 16: Figure 4: Sx6518 Switch Views

    The following figure shows the connector and spine sides fully populated. Figure 4: SX6518 Switch Views Leaf Side Spine Side Up to 6 Power Supply Units 9 spine modules Two leaf fan modules on each side of the chassis. 18 Leaf modules 2 management modules Mellanox Technologies Confidential...

  • Page 17: Features List

    802.3ap-2007 (Amendment to IEEE Std 802.3-2005) chapter 74. FDR and FDR10 are only guaranteed to work with approved Mellanox cables. FDR10 is only guaranteed to work with approved Mellanox ConnectX-3 adapters. 1. Features for a future release. Mellanox Technologies Confidential...

  • Page 18: Power Supply Redundancy

    Connecting 3 power supplies to one power supply grid and the remaining 3 power supplies to a secondary power supply grid will create N+N redundancy. This is High Availability. Under these conditions should a power grid fail (an electric company power failure or blackout for Mellanox Technologies Confidential...

  • Page 19: Table 3: Opns For Power Supply Units

    The form is identical between the two PSU types while the 1000W PSU weighs 0.3kg more than the 1670W PSU. Table 3 - OPNs for Power Supply Units Description Wattage MTP005001 1000W Supplies N+1 redundancy MTP006002 1670W Supplies N+N redundancy for all switch chassis at 220 Volts Mellanox Technologies Confidential...

  • Page 20: Chapter 2 Installation

    2. Bodily Injury Due to Weight Use enough people to safely lift this product. <40 lbs 70 - 121 lbs >121 lbs 40 - 70 lbs <18 kgs 32 - 55 kgs >55 kgs 18 - 32 kgs Mellanox Technologies Confidential...

  • Page 21: Over-Temperature

    10. Multiple Power Inlets , Risk of electric shock and energy hazard. The PSUs are all independent. Disconnect all power supplies to ensure a powered down state inside of the switch platform. Mellanox Technologies Confidential...

  • Page 22: Equipment Installation

    This device must be installed according to the latest version of the country national electrical codes. For North America, equipment must be installed in accordance to the applicable requirements in the US National Electrical Code and the Canadian Electrical Code. Mellanox Technologies Confidential...
  • Page 23 This device must be installed according to the latest version of the country national electrical codes. For North America, equipment must be installed in accordance to the applicable requirements in the US National Electrical Code and the Canadian Electri- cal Code. Mellanox Technologies Confidential...

  • Page 24: Interconnection Of Units

    According to the WEEE Directive 2002/96/EC, all waste electrical and electronic equipment (EEE) should be collected separately and not disposed of with regular household waste. Dispose of this product and all of its parts in a responsible and environmentally friendly way. Mellanox Technologies Confidential...

  • Page 25: Environmental And Safety Recommendations

    • 6 PSUs • 1 installation guide • 1 installation kit box • 1 box containing 6 power cords 250v 15a 2.0M, C14 to C13, USA UL Standard • 1 cable management kit • 6 cable supporter boxes Mellanox Technologies Confidential...

  • Page 26: Leaf Package Contents

    The switch platform uses 18U of rack space in a standard 19” rack, 16U for the chassis and 2U for the shelf. The switch ships from the factory with mounting holes on the spine side. There are Mellanox Technologies Confidential...

  • Page 27: Starting With The Rack

    The rack may need to be modified to accommodate the SX6518 chassis. The distance between the vertical supports must be between 65 and 80cm. Figure 5: Distance Between the Vertical Supports Vertical Supports This edge of the shelf must be Between 65 and 80 cm. as close as possible to the Mellanox Technologies Confidential...

  • Page 28: Esd Connection

    - 10 hex head bolts – 8 for the upper - 26 caged nuts 12 for the shelf and 8 for the brackets and 2 for the shelf extension chassis faceplate and 6 for the filler panels Mellanox Technologies Confidential...
  • Page 29 - Top filler panel • Parts included in the cable management installation kit: - 1 cable management rack RH - 1 cable management rack LH - 14 caged nuts M6 - 14 M6 bolts - 9 cable management shelves Mellanox Technologies Confidential...

  • Page 30: Figure 6: Installation Kit Parts

    If a shock sticker or more are red, notify the ship- per and Mellanox. This on its own does not indicate damage to the contents. But, be sure to care- fully inspect the contents if any of the shock stickers have tripped. Mellanox Technologies Confidential...

  • Page 31: Figure 7: Shock Stickers

    Installation Figure 7: Shock Stickers Stickers unaf- fected by ship- Stickers showing excessive shock Mellanox Technologies Confidential...

  • Page 32: Opening The Container

    • 6 PSUs are installed for the chassis • all 4 fans are installed 5. Remove all protective plastic film from all sides and top of the chassis. The leafs, spines, and management modules are shipped separately. Mellanox Technologies Confidential...

  • Page 33: Installing The Shelf

    The side of the chassis with the spine units will sit flush with the vertical supports of the rack. The side of the chassis with the QSFP connectors will sit at the edge of the shelf closer to the center of the rack. Mellanox Technologies Confidential...

  • Page 34: Figure 9: Vertical Rack Supports Dimension

    3. Decide which side of the rack you want the data connectors. Figure 10: Placement of Chassis in Rack Doors Cable with recommended bending radius Shelf Connector side Chassis Spine side Insufficient room for recommended bending radius Doors Mellanox Technologies Confidential...

  • Page 35: Figure 11: Adjustable Shelf Brackets

    They may need to be removed and repositioned to meet the distance between the vertical sup- ports of the rack. Figure 11: Adjustable Shelf Brackets These two brackets will need to be adjusted. They may even need to be The holes in the bracket are offset to one side. Mellanox Technologies Confidential...

  • Page 36: Figure 12: Top Of Shelf Bracket

    Do not tighten the hex head bolts. You may need to remove the brackets and reinstall them. Figure 13: Placing the Shelf Bracket Adjusting Mellanox Technologies Confidential...

  • Page 37: Figure 14: Caged Nut Locations For The Shelf Brackets

    9. Put the Shelf into the rack and twist the shelf into place. Make sure the spine side of the shelf is closest to the rack doors. See Figure 15 to identify the spine side of the shelf. 10. Install and tighten the 12 M6 bolts into the caged nuts. Mellanox Technologies Confidential...

  • Page 38: Figure 15: Shelf Brackets

    12. Place 8 caged nuts into the vertical rack support at the locations corresponding to the holes in the chassis faceplate. The holes in the faceplate start at 2U above the shelf and are spaced every 4U centered as you go up the vertical rack support. There are 4 holes on each side. Mellanox Technologies Confidential...

  • Page 39: Figure 17: Putting The Chassis In The Rack Using A Fork Lift

    1. Put the container on the fork lift. Figure 17: Putting the Chassis in the Rack Using a Fork Lift The chassis is screwed down to the pallet on both the spine side and the leaf side. Mellanox Technologies Confidential...

  • Page 40: Figure 18: Putting The Chassis In The Rack Using A Mechanical Lift

    Figure 18: Putting the Chassis in the Rack Using a Mechanical Lift Figure 19: Place the Upper Bracket Offset 2. Screw on the two Upper Bracket offsets onto the chassis using six flat head screws for each offset. Mellanox Technologies Confidential...

  • Page 41: Figure 20: Attach The Shelf Extension

    5. Place caged nuts in all of the locations specified in Figure 15. The black holes show where to place caged nuts; the red holes show the location of the upper brackets on the leaf side. Mellanox Technologies Confidential...

  • Page 42: Figure 21: Caged Nut Placement By U Numbers

    Figure 21: Caged Nut Placement by U Numbers Top filler panel For the 324 port chassis switches the upper brack- These are for the cable holder. Place 5 caged nuts here. Bottom filler panel Spine side Leaf Side Mellanox Technologies Confidential...

  • Page 43: Figure 22: Location For The Lock-Down Bar

    With both lock-down bars removed the chassis can roll! Be very careful that the chassis does not roll off of the pallet. 7. Place the chassis in front of the rack and as close as possible to the rack. Figure 23: Fork Lift Container Hinged exten- Figure 24: Mechanical Lift Mellanox Technologies Confidential...

  • Page 44: Figure 25: Face Plate Mounting Bolt Locations

    Do not move the forklift away from the rack just yet. 13. Confirm that the caged nuts for the faceplate are in the correct location. 14. Slide the chassis out just far enough to install the upper brackets. See Figure 26. Mellanox Technologies Confidential...

  • Page 45: Figure 26: Upper Bracket Installation

    16. Slide the chassis onto the shelf until the face plate of the chassis touches the vertical rack sup- port. 17. Make sure the upper brackets are installed in their correct location. 18. Screw the 8 bolts through the faceplate into the caged nuts, DO NOT TIGHTEN. Mellanox Technologies Confidential...

  • Page 46: Figure 28: Offset To Upper Bracket Bolts

    20. Place the front lock-down bar at the edge of the chassis and align the holes in the lock-down bar with the holes in the shelf. Screw down the lock-down bar.You will need the 10 hex head screws and washers for each lock down bar. DO NOT TIGHTEN 21. Screw down the second lock-down bar. Mellanox Technologies Confidential...

  • Page 47: Figure 31: Lock-Down Bars Bolt And Washer Order

    25. Tighten the upper bracket to upper bracket offset screws. See Figure 28. 26. Tighten the lock down bars. See Figure 31. 27. Connect the bottom filler panel below the shelf, bend facing out. 28. Connect the top filler panel directly above the top of the chassis. Mellanox Technologies Confidential...

  • Page 48: Figure 32: Spine Side With Filler Panels

    Fille r p an e l Sh e lf Fille r p an e l 29. Go to Chapter 4, “Chassis Power Up,” on page 83 to power up the switch system. 30. Insert 2 caged nuts for the filler panel. Mellanox Technologies Confidential...

  • Page 49: Installing The Cable Holder

    1. Place the cable holder next to the rack, on the connector side of the chassis, and identify the holes where the caged nuts were placed in Step 4 The cable holders should go to the outside of the vertical supports. Mellanox Technologies Confidential...

  • Page 50: Ground Connections

    The chassis is concurrently grounded through each of the PSUs. Only con- nect the PSU cords to properly grounded outlets. Do not rely on the PSU grounds. It is absolutely necessary to connect the grounding post. Make sure the connections are solid and permanent. If Mellanox Technologies Confidential...

  • Page 51: Power Connections

    PSU has a dedicated AC inlet. This design enables the optional use of separate main and backup AC feeds. The input voltage is Autorange, 100-240 VAC, 50Hz or 60Hz. The output voltage for the PSUs is 48V. The power cords should be standard 3-wire AC power cords including a safety ground. Mellanox Technologies Confidential...

  • Page 52: Connecting To An Ac Power Source

    2. Plug the other end of the power cords into grounded outlets. Make sure that the outlets and circuits will not be overloaded. Spread out the load over at least two or three circuits or use a 3 phase circuit. Mellanox Technologies Confidential...

  • Page 53: Infiniband Qsfp Cable Installation

    Technologies. The Mellanox proprietary QSFP cables support full 56+56Gb/s (FDR), 40+40Gb/ s (FDR10), 40+40Gb/s (QDR), 20+20Gb/s (DDR) and 10+10Gb/s (SDR) bidirectional wire speed of the switch ports. All InfiniBand QSFP connections are made to the leaf boards. Each Mellanox Technologies Confidential...

  • Page 54: Supported Approved Cables

    Chassis and switches need to be able to dissipate the heat generated by high power I/O cables and modules. The Mellanox SX65xx series chassis are rated for cables up to class 2 as per the SFF committee classification (SFF-8436.PDF). http://www.mellanox.com/content/pages.php?pg=cables for the cable class rating of Mella- nox cables. Mellanox Technologies Confidential...

  • Page 55: Hot Swap Insertion And Extraction

    1. Determine which AC connector on the connector side of the chassis corresponds to the defec- tive PSU. 2. Remove the power cord from the power supply unit. Note which power cord it is according to the AC numbering. Mellanox Technologies Confidential...

  • Page 56: Leaf Boards

    When hot-swapping any of the units, it is necessary to wait 1 minute after removing the defective part before inserting the new part. This is necessary so that the manage- ment module will start a new cycle checking through the leafs and spines for the FW versions. Mellanox Technologies Confidential...

  • Page 57: Figure 39: Leaf Board Numbering

    4. Open the ejector until it is 45 degrees from the leaf. Figure 40: Ejector Handle Pull the leaf out of the Push the ejector han- Ejector handle 5. Pull out the module half-way through the guiding rails using the ejector handle. 6. Lock the ejector handle. Mellanox Technologies Confidential...

  • Page 58: Figure 41: Leaf Release

    2. Check leaf mechanics for any noticeable damage. 3. Check back signal connectors’ integrity. Look for any broken signal dividers or any devia- tions from the pass criterion shown in Figure 42. Figure 42: Intact vs Defected Signal Connectors Pass Fail Mellanox Technologies Confidential...

  • Page 59: Figure 43: Intact Vs Defected Power Pin Holders

    8. Catch the hook onto the vertical bar of the chassis and push the ejector handle shut. 9. Lock the ejector handle onto the board. Mellanox Technologies Confidential...

  • Page 60: Spine Boards

    #2, as the management modules are connected to the chassis components through these spines. Extracting Spine Board #1 or #2 Spine board #1 is connected to management board #1 and spine board #2 is connected to man- agement board #2. Mellanox Technologies Confidential...

  • Page 61: Figure 44: Management Module Numbering

    2. Follow the steps in Section below. Extracting Spine Boards Except #1 or #2 1. Run the shut down command For example to shut down “no power enable <module>”. spine 06 run the command below. switch [master] (config) # no power enable S06 Mellanox Technologies Confidential...

  • Page 62: Inserting A Spine Board

    1. Check for foreign objects in or mechanical damage to the chassis spine slots. 2. Check spine mechanics for any noticeable damage or deviations from the pass criterion shown in Figure 46. Figure 46: Intact vs Defected Mechanics Fail Pass Mellanox Technologies Confidential...

  • Page 63: Figure 47: Intact Vs Defected Side Signal Connectors

    3. Check back signal connectors’ integrity. Look for any broken signal dividers or any devia- tions from the pass criteria shown in Figure 47 and Figure 48. Figure 47: Intact vs Defected Side Signal Connectors Fail Pass Figure 48: Intact vs Defected Middle Signal Connectors Pass Fail Mellanox Technologies Confidential...

  • Page 64: Figure 49: Intact Vs Defected Power Pin Holders

    Do not apply excessive force to slide in the spine board. If you feel resistance, remove the spine and double check both the chassis and spine for any damage. 9. Catch the hooks onto the vertical bar of the chassis and push the ejector handle shut. Mellanox Technologies Confidential...

  • Page 65: Fan Modules

    MNG2 MASTER S.Fans L.Fans Master CONSOLE Mellanox ® SX6518 SX6518 Mellanox Mellanox Extracting the Leaf Fan Module 1. Push and hold the blue latch release. See Figure 52. 2. Slowly pull out the fan module using the handle. Mellanox Technologies Confidential...

  • Page 66: Figure 52: Leaf Fan Module Extraction

    Each spine module has a fan module with two individual fans built in. When a fan module is not functioning the Fan Status LED on the spine will light up. Figure 53: Spine Fan Module 21 23 25 29 31 15 17 20 22 Fan mod- Mellanox Technologies Confidential...

  • Page 67: Figure 54: Fan Status Led On The Spine Module

    3. Push the fan module as far as it will go, make sure the locking latches engage. If the Fan LED continues to show red remove the fan module and check the pins on the connector inside of the spine to make sure that none of them are bent. Mellanox Technologies Confidential...

  • Page 68: Management Module

    7. Hold the body of the board on both sides and remove it from the chassis. The board is short, therefore do not let go of it while sliding it out. Figure 55: Management Module S.FANS L.FANS Master CONSOLE Ejectors Mellanox Technologies Confidential...

  • Page 69: Inserting A Management Module

    4. Check power connector’s integrity. Look for any damage on the power connector casing or blades damage, or any deviations from the pass criterion shown in Figure 57. Figure 57: Intact vs Defected Power Pin Holders Pass Fail Mellanox Technologies Confidential...

  • Page 70: Switch Shut-Down Procedures

    To shut down a leaf run the following command: no power enable <module> To shut down a spine run the following command: no power enable <module> To shut down a management module run the following command: no power enable <module> Mellanox Technologies Confidential...

  • Page 71: Chapter 3 Interfaces

    DC OK – When lit this LED indicates that the output from the power supply is +48 VDC. The PSUs are on the spine side of the chassis behind two cover panels. The plugs for these PSUs are on the leaf side of the chassis. Mellanox Technologies Confidential...

  • Page 72: Leaf Board Led Indicators

    3.1.2 Leaf Board LED Indicators Figure 60: Leaf Board Led Indicators 3.1.2.1 Status LED Each leaf board has a Status LED on the far left of the leaf. Table 4 shows the leaf status accord- ing to the LED condition. Mellanox Technologies Confidential...

  • Page 73: Table 4: Leaf Status Led

    3.1.2.3 UID LED Switch Identifier The UID LED is a debug feature that will become available to customers in the near future. For details please contact Mellanox Technologies support. 3.1.2.4 Leaf Board Port Connector LED Assignment Above the ports are two LEDs one for the upper port and one for the lower port .

  • Page 74: Spine Board Led Indicators

    2. If the (number of leafs) x (the maximum number of connections per leaf) is less than 36 then some of the leaf to spine connection LEDs may be OFF. The status LEDs for the spine and their descriptions are shown in Table 7. The LEDs indicate as follows. Mellanox Technologies Confidential...

  • Page 75: Table 7: Spine Status Led

    LED Description Solid Green Spine Fan is OK Flashing Green Spine Fan needs replacing Solid Yellow One or more of the fans in this spine is not working Each spine has two fans in the fan module Mellanox Technologies Confidential...

  • Page 76: Spine Side Panel Display Led Indicators

    3.1.3.5 UID LED Switch Identifier The UID LED is a debug feature that will become available to customers in the near future. For details please contact Mellanox Technologies support. 3.1.4 Spine Side Panel Display LED Indicators The spine side panel display has LEDs that show the chassis condition.

  • Page 77: Table 11: Spine Side Led Display For Normal Operation

    Green – Management module is operating as a master MNG2 Off – MASTER - no power STATUS - this management module is not installed - this management module is not the master Green – Management module is operating as a master Mellanox Technologies Confidential...

  • Page 78: Management Module Led Indicators

    Red – One or more of the leaf fans is bad. User should check individual leaf fan LEDs for fault indications. MASTER Off – Green if - no power master Off if slave - this management module is not the master Green – Management module is operating as a master Mellanox Technologies Confidential...

  • Page 79: Table 13: Management Module Psu Led Configurations

    LED Condition Description Green OK – All Chassis Leaf fans are working at configured speed. Error – One or more of the leaf fans for the chassis is bad. Check each chassis fan for a red LED. Mellanox Technologies Confidential...

  • Page 80: Port Connector Interfaces

    SX6518. They are: • 1 I2C port • 1 CONSOLE port – this is an RS232 connector for connecting to a host machine • 1 MGT – this is an Ethernet connector • 1 USB port Mellanox Technologies Confidential...

  • Page 81: I2C

    Initial configuration must be done on all of the management modules. The first man- agement module you configure will be the master. 3.4.4 USB The USB port can be used to upload new SW using any storage device that has a USB connector. This interface is USB 2.0. Mellanox Technologies Confidential...

  • Page 82: Reset - Rst

    Insure the PDU used has the required number of C13 sockets and the number of PDUs reflect the desired power redun- dancy level, refer to “Power Supply Redundancy” on page 18. for more details. Mellanox Technologies Confidential...

  • Page 83: Chapter 4 Chassis Power Up

    6. Tighten all leaf and spine mounting screws. 7. Connect the power cords to the PSUs. 8. Connect the power cords to grounded electrical outlets. With N+1 PSU redundancy do not power up the chassis with less than all PSUs installed. Mellanox Technologies Confidential...

  • Page 84: Power Supply And Spine Board Indicator Status At Power On

    4. Check the Spine LEDs and make sure they coincide with Figure 69. Figure 69: Spine Side Panel Display Status Indications 5. Check the Management Module LEDs and make sure they coincide with Figure 70. Figure 70: Management Module Status Indications for Normal Operation S.FANS L.FANS Master CONSOLE Mellanox Technologies Confidential...

  • Page 85: Chapter 5 Switch Management Tools

    • Fan unit information • Power unit information • Flash memory • Monitoring of: • AC power to the PSUs • DC power out from the PSUs • chassis failures • querying for: • switch serial numbers • revisions Mellanox Technologies Confidential...

  • Page 86: Infiniband Subnet Manager

    • SNMP • XML Upgrading and Downgrading Software The new software and firmware images are available to the user from the MyMellanox website. Copy an image to a known location on a Remote server within your LAN. Mellanox Technologies Confidential...

  • Page 87: Figure 71: Sx65Xx Downgrade Attention Sticker

    If in possession of an FDR director switch with the notice presented in Figure 71, the lowest MLNX-OS version you can downgrade to is 3.3.5006; otherwise, the switch system will mal- function. Figure 71: SX65xx Downgrade Attention Sticker Mellanox Technologies Confidential...

  • Page 88: Chapter 6 Troubleshooting

    3. Make sure that the latest FW version is installed on both the HCA card and the switch. 4. If media adapters are used check that the all connections are good, tight, and secure. 5. Replace the cable. Issue 4. The Activity indication does not come on: Mellanox Technologies Confidential...

  • Page 89: Management Module

    There are two non-interchangeable types of leaf fan modules. If the new leaf fan does not go into the chassis confirm that it is the correct fan module. Replace defective leaf fan modules as soon as they are identified. Mellanox Technologies Confidential...

  • Page 90: Spine Board

    2. Push the reset button on the switch or management module. 3. You will have ~ 5 seconds to stop the U-Boot by pressing Control-B. 4. Choose the image to upload. Only use image 1 or image 2. Mellanox Technologies Confidential...
  • Page 91 18 :00:03 ppc 3. U-Boot prompt Choice: 5. Select the image to boot. For more detailed instructions concerning MLNX-OS® software see the Mellanox MLNX-OS® SwitchX® Software WebUI User’s Manual or the Mellanox MLNX- OS® SwitchX® Software User Manual. Mellanox Technologies Confidential...

  • Page 92: Chapter 7 Disassembly And Disposal

    2. Remove the screws holding the chassis to the rack. These screws are located in the faceplate on the spine side of the chassis. 3. Remove the upper brackets from the chassis. 4. Remove the lock-down bars. Mellanox Technologies Confidential...

  • Page 93: Removing The Bottom Shelf

    WEEE Directive. 7.1.2 Removing the Bottom Shelf 1. Remove all of the bolts that are holding the shelf to the rack. 2. Remove the filler panels. 3. Remove the shelf. 4. Remove all of the caged nuts. Mellanox Technologies Confidential...

  • Page 94: Disposal

    (EEE) should be collected separately and not disposed of with regular household waste. Dispose of this product and all of its parts in a responsible and environmentally friendly way. Go to the Mellanox website for detailed instructions for disassembly according to the WEEE Directive. Mellanox Technologies Confidential...

  • Page 95: Appendix A: Specification Data

    Input Voltage Typical Passive: 3038W Active: 3777W (includes QSFP at 1.5W) Optical: 3876W (includes QSFP at 1.7W) Total Power Consumption FDR 10 Passive: 3522W Maximum Active: 4359W (includes QSFP at 1.7W) Optical: 4507W (includes QSFP at 2.0W) Mellanox Technologies Confidential...
  • Page 96 VCCI (Emissions) For more compliance information please refer to the following page on the Mellanox webpage: http://www.mellanox.com/page/switch_certification_ma- trix. EU: IEC 60068-2-64: Random Vibration EU: IEC 60068-2-29: Shocks, Type I / II Environmental EU: IEC 60068-2-32: Fall Test Mellanox Technologies Confidential...

  • Page 97: Emi Certification

    Spine fans: 9 EMI Certification EMI certification on the fully populated chassis was performed with the chassis installed in a closed two-door rack using the chassis installation kit supplied by Mellanox Technologies. Approved Cables For a list of all approved cables see: http://www.mellanox.com/related-docs/user_manuals/Mellanox_approved_cables.pdf...

  • Page 98: Appendix B: Thermal Threshold Definitions

    When the SwitchX® device crosses this temperature, the firmware will automatically shut down the device. 3. Emergency – 130ºC In case the firmware fails to shut down the SwitchX® device upon crossing the Critical thresh- old, the SwitchX® device will auto-shutdown upon crossing the Emergency (130ºC) threshold. Mellanox Technologies Confidential...

  • Page 99: Appendix C: Calculating The Weight Of A Customized Chassis

    # of leaf blanks * 0.764 = # of management mod- 3.15 = ules * # of PSUs 1.97 = Total This total is in kilograms. Multiply the total by 2.2 to get the total weight in pounds. Mellanox Technologies Confidential...

  • Page 100: Appendix D: Calculating The Power Of A Chassis

    4. Get the total switch system power consumption system power by summing up all multiplica- tion products. Table 21 - Calculating Total Switch System Power Consumption Quantity x Part Quantity Power Power IB FDR or FDR10 leaf with passive cables Mellanox Technologies Confidential...
  • Page 101 Power IB FDR or FDR10 leaf with active cables IB FDR or FDR10 leaf with optical cables Spine FDR or FDR10 Chassis fans Management module Total power N+N redundancy ONLY works with a supply voltage of 220V. Mellanox Technologies Confidential...

  • Page 102: Appendix E: Qsfp Interface

    Receiver Inverted Data Output Rx1n Ground Ground Receiver Inverted Data Output 3 Rx2n Receiver Non-Inverted Data Output 3 Rx2p Ground Rx4n Receiver Inverted Data Output 3 Rx4p Receiver Non-Inverted Data Output 3 Ground Mod- Module Present PrsL IntL Interrupt Mellanox Technologies Confidential...

  • Page 103: Figure 73: Infiniband Qsfp Connector Symbol

    Transmitter Inverted Data Input Tx1n Ground Figure 73: InfiniBand QSFP Connector Symbol J3 and J6 Rx1n Rx2n Rx1p Rx2p Rx4n Rx3n Rx4p Rx3p ModPrsL IntL VccTx Vcc Rx Vcc1 ResetL LPMode ModSelL Tx3p Tx4p Tx3n Tx4n Tx1p Tx2p Tx1n Tx2n Mellanox Technologies Confidential...

  • Page 104: Figure 74: Qsfp Connector Male And Female Views

    Figure 74: QSFP Connector Male and Female Views 18.35 View into Rear of Connector 8.50 18.35 View into Front of Cage 8.50 Mellanox Technologies Confidential...

  • Page 105: Appendix F: Replacement Parts Ordering Numbers

    Power cord Type E/F for Sweden, France, Germany, ACC000207 Netherlands, Russia Power cord Type G for UK ACC000208 Power cord Type D for India ACC000209 Power cord Type I for China ACC000210 Power cord Type J for Switzerland ACC000211 Mellanox Technologies Confidential...
  • Page 106 Table 23 - Replacement Parts Ordering Numbers (Continued) Part Description Power cord Type B for Japan ACC000212 Power cord Type I for Australia ACC000213 Mellanox Technologies Confidential...

  • Page 107: Appendix G: Safety Warnings (Multiple Languages)

    8cm (3 inches) of clearance around the ventilation openings. 6. Stacking the Chassis The chassis should not be stacked on any other equipment. If the chassis falls, it can cause bodily injury and equipment damage. Mellanox Technologies Confidential...
  • Page 108 13. Equipment Disposal Disposal of this equipment should be in accordance to all national laws and regula- tions. 14. Local and National Electrical Codes This equipment should be installed in compliance with local and national electrical codes. Mellanox Technologies Confidential...
  • Page 109 According to the WEEE Directive 2002/96/EC, all waste electrical and electronic equipment (EEE) should be collected separately and not disposed of with regular household waste. Dispose of this product and all of its parts in a responsible and environmentally friendly way. Mellanox Technologies Confidential...
  • Page 110 This unit is intended for connection to a TN power system and an IT power system of Norway only. (‫אזהרות בטיחות בהתקנה )עברית‬ 1. ‫הוראות התקנה‬ 2. ‫תקן ישראלי‬ ‫חבלת גוף כתוצאה מנשיאת משקל יתר‬ ‫ציוד כבד‬ ‫!סכנת התחשמלות‬ Mellanox Technologies Confidential...
  • Page 111 6. ‫התחממות יתר‬ ‫ערימת המערכת‬ ‫חיבור ספק כוח נוסף -סכנת חשמל‬ ‫מספר שקעים חשמליים‬ 10. ‫בעת סופות ברקים - סכנת חשמל‬ 11. ‫חיבור או ניתוק של כבלי אינפיניבאנד מנחושת‬ Mellanox Technologies Confidential...
  • Page 112 12. ‫הרכבה על גבי מדף בארון‬ 13. ‫התקנת המוצר‬ 14. ‫השלכת פסולת‬ 15. ‫תקנות חשמל מקומיות ולאומיות‬ 16. ‫כבל אספקת חשמל‬ 17. ‫תקנות התקנה‬ Mellanox Technologies Confidential...

  • Page 113: 安裝安全性警告 (Chinese)

    ‫חיבור בין מערכות‬ 19. ‫הגנה מפני מתח גבוה‬ 20. ‫אין להשתמש במערכת כמדף או שטח עבודה‬ 21. WEEE ‫תקנת‬ 22. ‫מגבלות חשמליות בנורבגיה‬ 安裝安全性警告 (Chinese) 1. 安裝指示 本設備附有備援電源供應器或在適當位置配有空白蓋板。 Mellanox Technologies Confidential...
  • Page 114 18 - 32 kgs 3. 重設備 本設備極重，應使用機械式起重機來搬移，以避免人員受傷。 4. 有觸電的危險 有觸電的危險！ 拆除風扇模組後，即可接觸到模組空腔內的電源針腳。 請勿將工具或機身零件插入到風扇模組空腔內。 5. 溫度過高 本設備不應在超過所建議的最高環境溫度的區域中運作：45°C (113°F)。此外， 為了保證氣流的流通正常，請在通風口旁保留至少 8 公分 (3 英吋 ) 的間距。 6. 堆疊機箱 機箱不應堆疊在任何其他設備上。如果機箱掉落，可能造成人員受傷與設備損 壞。 7. 複式電源連接時的電擊危險 本設備附有備援電源供應器或在適當位置配有空白蓋板。如果是電源供應器空 白蓋板，在空白蓋板已取下或未牢牢固訂的情況下，請勿操作本產品。 8. 多電源輸入座 電擊與能源危害的危險。 所有 PSU 均各自獨立。 將所有電源供應器斷電，確保交換器平台內部在電源關閉狀態。 Mellanox Technologies Confidential...
  • Page 115 15. UL 列名和 CSA 認證電源線 北美地區在接上電源時，請選用獲得 UL 列名和 CSA 認證、三個導體、[16 AWG] 附成型插頭，額定值為 125 V、[13 A]，長度至少 1.5 公尺 [ 六英尺 ]，但 不超過 4.5 公尺的電源線。 歐洲地區在接上電源時，請選用國際協調式且標示有 <HAR> 字樣、三個導體、 標稱截面至少 1.0 平方公厘，額定值為 300 V，採用 PVC 絕緣的電源線。電源 線需有成型插頭，額定值為 250 V, 10 A。 16. 高漏電流 Mellanox Technologies Confidential...
  • Page 116 連接至 RS232 設備和乙太網路介面的纜線必須是 UL 認證類型 DP-1 或 DP-2。 ( 請注意位於非 LPS 電路時 ) 過電流保護：準備好使用的列名分支電路過電流保護裝置最大額定值 20 A 必須 整合在配線中。 19. 切換開關不可用作機架或工作空間 小心：滑軌 / 導軌安裝設備不可用作機架或工作空間。導軌不適用於將設備滑 出機架使用。僅限永久安裝在最後安置區域時使用，不可用於維修和保養。 20. WEEE 指令 根據 WEEE 指令 2002/96/EC，所有廢棄的電氣與電子設備 (EEE)，應分開集 中，而且不應與一般家庭廢棄物一起棄置。 請以負責和環保的方式棄置本產品及其所有零件。 21. 挪威國家電源限制 本設備僅限連接至挪威的 TN 電源系統和 IT 電源系統。 Mellanox Technologies Confidential...

  • Page 117: Avertissements De Sécurité Pour L'installation (French)

    8 cm (3") autour des orifices de ventilation. 6. Châssis empilé sur d'autres équipements Le châssis ne doit pas être empilé sur d'autres équipements. S'il tombe, il peut endom- mager l'équipement ou entraîner des blessures. Mellanox Technologies Confidential...
  • Page 118 La mise au rebut de cet équipement doit se faire conformément à toutes les lois et réglementations nationales. 14. Codes électriques locaux et nationaux Cet équipement doit être installé conformément aux codes électriques locaux et nation- aux. Mellanox Technologies Confidential...
  • Page 119 Selon la Directive 2002/96/CE (DEEE), tous les déchets d'équipements électriques et électroniques (EEE) doivent être collectés séparément et ne pas être mis au rebut avec les déchets ménagers habituels. Ce produit et toutes ses pièces doivent être mis au rebut d'une manière responsable, respectant l'environnement. Mellanox Technologies Confidential...

  • Page 120: Installation Sicherheitshinweise(German)

    Temperatur von 45°C (113°F) betrieben werden. Es ist ein Luft- strom von 200 LFM bei maximaler Umgebungstemperatur erforderlich. Außerdem sollten mindestens 8 cm (3 in.) Freiraum um die Belüftungsöffnungen sein, um einen einwandfreien Luftstrom zu gewährleisten. Mellanox Technologies Confidential...
  • Page 121 12. Geräteentsorgung Die Entsorgung dieses Geräts sollte unter Beachtung aller nationalen Gesetze Bestim- mungen erfolgen. 13. Regionale und nationale elektrische Bestimmungen Dieses Gerät sollte unter Beachtung der regionalen und nationalen elektrischen Bes- timmungen installiert werden. Mellanox Technologies Confidential...
  • Page 122 Achtung: Auf Schieber/Schienen montiertes Gerät ist nicht als Regal oder Arbeitsbere- ich zu nutzen. Die Schienen sind nicht dafür bestimmt, die Einheit aus dem Gestell weg zu ziehen. Sie sind nur für die permanente Installation an einem endgültigen Stan- dort gedacht, nicht für Instandhaltung und Wartung. Mellanox Technologies Confidential...

  • Page 123: Weee-Direktive

    ¡Riesgo de descarga eléctrica! Con el módulo del ventilador quitado, se obtiene acceso a las clavijas de alimentación desde dentro de la cavidad del módulo. NO introducir herramientas ni partes del cuerpo en la cavidad del módulo del ventila- dor. Mellanox Technologies Confidential...
  • Page 124 Al instalar o realizar el mantenimiento de este aparato en un bastidor, es preciso adoptar precauciones especiales para garantizar que el sistema se mantenga estable. En general, en un bastidor, los equipos se deben instalar comenzando desde abajo hacia arriba. Mellanox Technologies Confidential...
  • Page 125 Este dispositivo se debe instalar conforme a la versión más reciente de los códigos eléctricos nacionales del país en cuestión. En América del Norte, el equipo se debe instalar de acuerdo con las disposiciones vigentes del Código Eléctrico Nacional de los EE.UU. y del Código Eléctrico de Canadá. Mellanox Technologies Confidential...

  • Page 126: Предупреждения По Технике Безопасности При Установке (Russian)

    2. Травмы при переносе тяжелых предметов Для поднятия этого изделия следует задействовать достаточное количество людей. <40 lbs 70 - 121 lbs >121 lbs 40 - 70 lbs <18 kgs 32 - 55 kgs >55 kgs 18 - 32 kgs Mellanox Technologies Confidential...
  • Page 127 Блоки питания независимы друг от друга. Чтобы обесточить все компоненты внутри платформы коммутации, следует отсоединить все блоки питания. 9. Опасность поражения электрическим током во время грозы Во время грозы запрещается использовать оборудование и подключать или отключать кабели. Mellanox Technologies Confidential...
  • Page 128 Подключение к электропитанию в Европе выполняется с помощью гармонизированного шнура питания с маркировкой <HAR>, 3-жильного, с сечением жилы не менее 1,0 мм², рассчитанного на номинальное напряжение 300 В, с ПВХ оболочкой. Шнур должен иметь литую вилку, рассчитанную на 250 В, 10 А. Mellanox Technologies Confidential...

  • Page 129: Avertismente Privind Siguranţa La Instalare (Romanian)

    электронного оборудования должны собираться и утилизироваться отдельно от обычных бытовых отходов. Следует утилизировать это изделие и все его части ответственным и экологически безопасным способом. Avertismente privind siguranţa la instalare (Romanian) 1. Instrucţiuni de instalare Citiţi toate instrucţiunile de instalare înainte de a conecta Mellanox Technologies Confidential...
  • Page 130 8. Multiple mufe electrice Risc de şoc electric şi pericol electric. Toate aparatele cu alimentare de la reţea sunt independente. Deconectaţi toate sursele de alimentare cu energie pentru a asigura decuplarea în inter- iorul platformei de comutare. Mellanox Technologies Confidential...
  • Page 131 Acest dispozitiv trebuie să fie instalat în conformitate cu ultima versiune a codurilor electrice naţionale ale ţării în cauză. Pentru America de Nord, echipamentul trebuie să fie instalat conform cerințelor aplicabile din Codul electric naţional al SUA şi Codul electric canadian. Mellanox Technologies Confidential...
  • Page 132 În conformitate cu Directiva DEEE 2002/96/CE, toate deşeurile de echipamente elec- trice şi electronice (EEE) trebuie colectate separat şi nu trebuie eliminate împreună cu deşeurile menajere obişnuite. Eliminaţi acest produs şi toate componentele sale în mod responsabil şi ecologic. Mellanox Technologies Confidential...

  • Page 133: Sigurnosna Upozorenja Za Instaliranje (Croatian)

    8 cm (3 inča) razmaka oko otvora ventilatora. 6. Slaganje kućišta Kućište se ne bi trebalo slagati na drugu opremu. Ako kućište padne, može izazvati tje- lesne ozljede i oštećenje opreme. Mellanox Technologies Confidential...
  • Page 134 Ovu bi opremu trebalo instalirati, zamjenjivati i/ili servisirati samo obučeno i kvalifici- rano osoblje. 13. Odlaganje opreme Odlaganje opreme trebalo bi se vršiti sukladno nacionalnim zakonima i propisima. 14. Lokalni i nacionalni električni kodovi Ova oprema trebala bi se instalirati u skladu s lokalnim i nacionalnim električnim kodovima. Mellanox Technologies Confidential...
  • Page 135 Pozor: Oprema montirana na klizače/vodilice ne bi se trebala koristiti kao polica ili radna površina. Vodilice nisu namijenjene za povlačenje uređaja iz ormarića. Služe samo za trajnu instalaciju na konačnom položaju, a ne za servisiranje i održavanje. Mellanox Technologies Confidential...

  • Page 136: Avvertenze Di Sicurezza Per L'installazione (Italiano)

    70 - 121 lbs >121 lbs 40 - 70 lbs <18 kgs 32 - 55 kgs >55 kgs 18 - 32 kgs Apparecchiatura pesante Questa apparecchiatura è molto pesante e va spostata mediante un sollevatore meccanico, per evitare lesioni. Mellanox Technologies Confidential...
  • Page 137 10. Collegamento/scollegamento del cavo di rame InfiniBand I cavi di rame InfiniBand sono pesanti e non flessibili. Di conseguenza, vanno collegati o scollegati con cura dai connettori. Per avvertenze/istruzioni speciali, rivolgersi al produttore di cavi. Mellanox Technologies Confidential...
  • Page 138 PVC. Il cavo deve disporre di una spina stampata di potenza nominale pari a 250 V, 10 A. 17. Corrente di dispersione elevata Avvertenza: corrente di dispersione elevata; il collegamento a terra è essenziale prima di collegare l’alimentazione. Mellanox Technologies Confidential...
  • Page 139 Smaltire questo prodotto e tutte le sue parti in modo responsabile e rispettoso dell’ambiente 21. Limitazioni relative all’alimentazione per la Norvegia Questa apparecchiatura è progettata esclusivamente per il collegamento a un sistema di alimentazione TN e a un sistema di alimentazione IT. Mellanox Technologies Confidential...

  • Page 140: Montaj Güvenlik Uyarıları (Türkçe)

    45 °C (113 °F). Ayrıca, düzgün hava akışı sağlamak için havalandırma deliklerinin etrafında en az 8 cm (3 inç) açıklık bırakılmalıdır. 6. Şasi İstif Şasinin diğer herhangi bir ekipmanın üzerine istiflenmemesi gerekir. Şasi düşerse, fiziksel yaralanmalara ve ekipmanda hasara neden olabilir. Mellanox Technologies Confidential...
  • Page 141 Ekipmanın yalnızca eğitimli ve nitelikli personel tarafından monte edilmesi, değiştirilmesi ve/veya bakımının yapılması gerekir. 13. Ekipmanın Atılması Bu ekipmanın imhasında tüm ulusal yasalara ve düzenlemelere uyulması gerekir. 14. Yerel ve Ulusal Elektrik Kodları Bu ekipmanın montajında yerel ve ulusal elektrik kodlarına uyulması gerekir. Mellanox Technologies Confidential...
  • Page 142 20. WEEE Yönergesi WEEE Yönergesi 2002/96/EC uyarınca, tüm elektrikli ve elektronik ekipman atıkları (EEE) ayrı olarak toplanmalı ve evsel atıklarla birlikte çöpe atılmamalıdır. Bu ürün ve tüm parçaları çevreye dost ve sorumlu bir şekilde imha edilmelidir. Mellanox Technologies Confidential...

  • Page 143: Japan Vcci Class A Statement

    21. Norveç Güç Kısıtlamaları Bu ünite, bir TN güç sistemine ve sadece Norveç'in IT güç sistemine bağlanmak içindir. G.13 Japan VCCI Class A Statement Mellanox Technologies Confidential...

This manual is also suitable for:

Infiniband fdr switchx sx6518-6r

Table of Contents