Cisco InfiniBand  4x User Manual
Cisco InfiniBand  4x User Manual

Cisco InfiniBand 4x User Manual

Cisco systems switch module for ibm bladecenter user guide
Table of Contents

Advertisement

Quick Links

Cisco 4x InfiniBand Switch Module for IBM
BladeCenter User Guide
Corporate Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 526-4100
Text Part Number: 78-17838-01

Advertisement

Table of Contents
loading

Summary of Contents for Cisco InfiniBand Cisco 4x

  • Page 1 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Corporate Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 526-4100 Text Part Number: 78-17838-01...
  • Page 2 OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. CCSP, the Cisco Square Bridge logo, Follow Me Browsing, and StackWise are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.;...
  • Page 3: Table Of Contents

    IBM Support Contact Information Before You Call Determining the Software Version Downloading the Latest Software Release Cisco Systems Support Contact Information Cisco Technical Support & Documentation Website Submitting a Service Request Definitions of Service Request Severity IBM Statement of Limited Warranty Z125-4753-08 04/2004...
  • Page 4 FRANCE GERMANY HUNGARY IRELAND SLOVAKIA SOUTH AFRICA, NAMIBIA, BOTSWANA, LESOTHO AND SWAZILAND UNITED KINGDOM Part 3 - Warranty Information Warranty Period Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xvii xvii xviii xviii xviii xviii xxii xxii xxiii...
  • Page 5 Installing the Server Switch Module Downloading Operating System Software from Cisco.com Configuring the Server Switch Module 78-17838-01 xxvii xxviii xxxi xxxi xxxi xxxii xxxiii xxxiv xxxv Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxii xxxiii xxxiii xxxiii xxxiii xxxiv Contents...
  • Page 6 Changing the Default Password Configuring the System Clock Connecting Beyond BladeCenter Connecting InfiniBand Cables Specifications C H A P T E R Electrical Requirements Environmental Requirements N D E X Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
  • Page 7: Preface

    Preface Audience The intended audience is the administrator responsible for installing, configuring, and managing Server Switch equipment. This administrator should have experience administering similar networking or storage equipment. Organization This publication is organized as follows: Chapter Chapter 1 Chapter 2...
  • Page 8: Related Documentation

    Cisco Boot over IB User Guide for Linux • Cisco Boot over IB User Guide for Windows • Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Description Series indicate a pop-up menu sequence to open a form or execute a desired function.
  • Page 9: Contacting Technical Support

    IBM and Cisco are both available for more information. For support, contact IBM before Cisco. Note 78-17838-01 Table 1 EU Country Telephone List Italy—9-02-482-9202 Luxembourg—352-360385-1 Netherlands—020-514-5770 Portugal—351-21-7915-147 Spain—34-91-662-4916 Sweden—46-8-477-4420 United Kingdom—01475-555-055 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Contacting Technical Support Table...
  • Page 10: Before You Call

    Cisco support website at http://www.cisco.com/cgi-bin/tablebuild.pl/sfs-linux. After registering your product, you should have received a username and password to give you access to this site. Cisco Systems Support Contact Information Cisco Technical Support provides 24-hour-a-day award-winning technical assistance. The Cisco Technical Support &...
  • Page 11: Cisco Technical Support & Documentation Website

    Severity 1 (S1)—Your network is “down,” or there is a critical impact to your business operations. You and Cisco will commit all necessary resources around the clock to resolve the situation. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Contacting Technical Support...
  • Page 12 You and Cisco will commit resources during normal business hours to restore service to satisfactory levels. Severity 4 (S4)—You require information or assistance with Cisco product capabilities, installation, or configuration. There is little or no effect on your business operations. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
  • Page 13: Ibm Statement Of Limited Warranty Z125-4753-08 04/2004

    THE WARRANTY PERIOD. NO WARRANTIES APPLY AFTER THAT PERIOD. SOME STATES OR JURISDICTIONS DO NOT ALLOW LIMITATIONS ON HOW LONG AN IMPLIED WARRANTY LASTS, SO THE ABOVE LIMITATION MAY NOT APPLY TO YOU. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide...
  • Page 14: What This Warranty Does Not Cover

    IBM or your reseller will also manage and install selected engineering changes that apply to the Machine. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IBM Statement of Limited Warranty Z125-4753-08 04/2004 78-17838-01...
  • Page 15: Exchange Of A Machine Or Part

    IBM is liable for no more than damages for bodily injury (including death) and damage to real property and tangible personal property; and 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 1 - General Terms xiii...
  • Page 16: Governing Law

    Any litigation arising from this Statement of Limited Warranty will be settled exclusively by the Ordinary Commercial Court of the city of Buenos Aires. BOLIVIA Jurisdiction: The following is added after the first sentence: Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IBM Statement of Limited Warranty Z125-4753-08 04/2004 78-17838-01...
  • Page 17: Brazil

    In accordance with Article 1328 of the Peruvian Civil Code the limitations and exclusions specified in this section will not apply to damages caused by IBM's willful misconduct (“dolo”) or gross negligence (“culpa inexcusable”). 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms...
  • Page 18: Uruguay

    The warranties specified in this Section are in addition to any rights you may have under the Trade Practices Act 1974 or other similar legislation and are only limited to the extent permitted by the applicable legislation. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IBM Statement of Limited Warranty Z125-4753-08 04/2004 78-17838-01...
  • Page 19: Cambodia And Laos

    Governing Law: The following replaces “laws of the country in which you acquired the Machine” in the first sentence: laws of Hong Kong Special Administrative Region of China. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms xvii...
  • Page 20: India

    Where Machines are not acquired for the purposes of a business as defined in the Consumer Guarantees Act 1993, the limitations in this section are subject to the limitations in that Act. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xviii...
  • Page 21: People's Republic Of China (Prc)

    The warranty for Machines acquired in Western Europe shall be valid and applicable in all Western Europe countries provided the Machines have been announced and made available in such countries. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms...
  • Page 22 Commercial Court of Paris; 5) in Russia, all disputes arising out of or in relation to the interpretation, the violation, the termination, the nullity of the Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
  • Page 23: European Union (Eu)

    Information. You may contact IBM at the following address: IBM Warranty & Service Quality Dept. PO Box 30 Spango Valley Greenock Scotland PA16 0AH 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms...
  • Page 24: Consumers

    AS AN IMMEDIATE CONSEQUENCE OF THE EVENT THAT GENERATED THE DAMAGES; OR 4) LOSS OF BUSINESS, REVENUE, GOODWILL, OR ANTICIPATED SAVINGS. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxii IBM Statement of Limited Warranty Z125-4753-08 04/2004 78-17838-01...
  • Page 25: The Following Terms Apply To The Country Specified

    In such instances, regardless of the basis on which you are entitled to claim damages from IBM, IBM is liable for no more than: (items 1 and 2 unchanged). 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms xxiii...
  • Page 26: Germany

    Default. Circumstances may arise where, because of a Default, you are entitled to recover damages from IBM. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxiv IBM Statement of Limited Warranty Z125-4753-08 04/2004...
  • Page 27: Slovakia

    IBM will accept unlimited liability, subject always to the Items for Which IBM is Not Liable below, for physical damage to your tangible property resulting from the negligence of IBM. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 2 - Country-unique Terms...
  • Page 28: Part 3 - Warranty Information

    Country or Region of Purchase Worldwide * See “Types of Warranty Service” for the legend and explanations of warranty-service types. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxvi IBM Statement of Limited Warranty Z125-4753-08 04/2004 Table 1...
  • Page 29: Types Of Warranty Service

    CRU and Customer Carry-In or Mail-In Service This type of Warranty Service is a combination of Type 1 and Type 4 (see above). 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Part 3 - Warranty Information xxvii...
  • Page 30: Eu Country Telephone List

    Finland —+358-8001-4260 France —+33-238-557-450 Germany —+49-1805-253553 Greece —+30-210-680-1700 Hungary —+36-1-382-5720 Ireland —+353-1-815-4000 Italy —+39-800-820-094 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxviii IBM Statement of Limited Warranty Z125-4753-08 04/2004 Latvia —+386-61-1796-699 Lithuania —+386-61-1796-699 Luxembourg —+352-298-977-5063 Malta —+356-23-4175 Netherlands —+31-20-514-5770...
  • Page 31: Notices

    IBM product, and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxix...
  • Page 32: Edition Notice

    Red Hat, the Red Hat “Shadow Man” logo, and all Red Hat-based trademarks and logos are trademarks or registered trademarks of Red Hat, Inc., in the United States and other countries. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Predictive Failure Analysis...
  • Page 33: Important Notes

    Recycling facilities may not be available in your area. For information on disposal of batteries outside the United States, go to http://www.ibm.com/ibm/environment/products/batteryrecycle.shtml or contact your local waste disposal facility. 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Important Notes xxxi...
  • Page 34: Electronic Emission Notices

    This device complies with Part 15 of the FCC Rules. Operation is subject to the following two conditions: (1) this device may not cause harmful interference, and (2) this device must accept any interference received, including interference that may cause undesired operation. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxii Notices...
  • Page 35: Industry Canada Class A Emission Compliance Statement

    This is a Class A product. In a domestic environment this product may cause radio interference in which case the user may be required to take adequate measures. Taiwanese Class A warning statement 78-17838-01 Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Electronic Emission Notices xxxiii...
  • Page 36: Chinese Class A Warning Statement

    Notices Electronic Emission Notices Chinese Class A warning statement Japanese Voluntary Control Council for Interference (VCCI) statement Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxiv 78-17838-01...
  • Page 37: Weee Notice

    European Union. This label is applied to various products to indicate that the product is not to be thrown away, but rather reclaimed upon end of life per this Directive. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxv 78-17838-01...
  • Page 38 WEEE Notice Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxvi 78-17838-01...
  • Page 39: Safety

    Safety Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxvii 78-17838-01...
  • Page 40 Safety Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxviii 78-17838-01...
  • Page 41 Safety Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide xxxix 78-17838-01...
  • Page 42 Safety Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
  • Page 43: Chapter 1 About The Server Switch Module

    Server Switch Modules to your BladeCenter chassis and add HCA expansion cards to your BladeCenter hosts, your hosts can communicate to one another over IB within the chassis. When you connect the Server Switch Module to an outside IB fabric, BladeCenter hosts can communicate with all nodes that connect to the IB network.
  • Page 44: Server Switch Module And Hca Expansion Cards

    Cisco 4x InfiniBand Switch Module for IBM BladeCenter Each Server Switch Module includes 16 4x ports to the backplane and 8 4x ports (in the form of 2 4x connectors and 2 12x connectors) on the front panel. The Server Switch Module provides full non-blocking switching for all 24 ports.
  • Page 45: Topologies

    With the Server Switch Module and HCA expansion cards, you can create a non-redundant, single-switch topology or a redundant, dual-switch topology. There are four I/O module bays in the switch, labelled 7-10. Bay 7 is for the first InfiniBand interface, Note and bay 9 is for the second InfiniBand interface.
  • Page 46: Leds

    - OK Server Switch Module Status LEDs Module Status LEDs provide an at-a-glance indication of the health of the Server Switch Module. The tables list and describe the states of the Module Status LEDs. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide...
  • Page 47: Port Status Leds

    TX/RX External InfiniBand Ports The Server Switch Module has both 4x IB and 12x IB external connectors. The 12x IB connectors support three 4x connections using an "octopus cable." This cable begins as one 12x connector (which connects to your Server Switch Module), then branches into three 4x connectors that you can attach to 4x IB interfaces.
  • Page 48 Chapter 1 About the Server Switch Module External InfiniBand Ports Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
  • Page 49: Chapter 2 Installing And Configuring The Server Switch

    Ensure that you are properly grounded by attaching the ESD wrist strap to bare skin and attaching the clip on the other end to unpainted metal. Choose the I/O slot into which you want to install the Server Switch Module. You must install the module Step 2 in I/O slot 7 or slot 9.
  • Page 50: Configuring The Server Switch Module

    Step 5 A heading appears, where X is the number of the slot. Step 6 Enter an IP address for the Server Switch Module in the IP address field of the New Static IP Configuration section. Note The IP address of the Server Switch Module must fall in the same subnet as the modules in the other slots.
  • Page 51: Changing The Default Password

    Chapter 2 Installing and Configuring the Server Switch Open a Telnet session to the Server Switch Module with the IP address that you configured in Step 9 Log in to your Server Switch Module, and then proceed to your Server Switch Module. Log in with the default user name of USERID and password of PASSW0RD, where the 0 in PASSW0RD is a zero.
  • Page 52 The example that follows sets the date to 1:45 am on June 3rd, 2004: Cisco BladeCenterH# clock set 01:45:00 03 06 04 To configure primary, secondary, and tertiary NTP servers to synchronize the Server Switch with the network, follow these steps: Step 1 Switch to Privileged Exec mode by entering the enable command.
  • Page 53: Connecting Beyond Bladecenter

    To extend your InfiniBand fabric beyond your BladeCenter cluster, you must configure the priority of the subnet manager on your Server Switch Module, then connect one or more of the external interfaces on your Server Switch Module to an outside InfiniBand fabric.
  • Page 54 Chapter 2 Installing and Configuring the Server Switch Connecting Beyond BladeCenter Figure 2-1 Removing a Pull Connector Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide 78-17838-01...
  • Page 55: Chapter 3 Specifications

    0 to 11,000 meters 5% to 85% non-condensing 5% to 95% non-condensing 3.5G for 3 mSec. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide C H A P T E R Requirement 12.2 Volts nominal 6.0 Amps maximum...
  • Page 56 Storage Vibration Note In the case of non-uniform air flow, the velocity at any point must meet or exceed 150 LFM. Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide Requirement 0.27G at 5Hz to 500Hz for 30 minutes 50G with a velocity change of 3454 mm/sec squared 0.5G at 2Hz to 200Hz for 15 minutes...
  • Page 57: I N D E X

    LEDs server switch module initial configuration management ports status LEDs status LEDs subnet mask switch module about switch module status LEDs system clock, setting Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide I N D E X IN-1...
  • Page 58 Index topology dual-switch single-switch Cisco 4x InfiniBand Switch Module for IBM BladeCenter User Guide IN-2 78-17838-01...

Table of Contents