Related Manuals for Cisco Topspin SFS 3012
Summary of Contents for Cisco Topspin SFS 3012
- Page 1 Cisco SFS 3012 Multifabric Server Switch Hardware Guide (Topspin 360 Hardware Guide) Web Release v2.1.0 Part Number: OL-7722-01 Product ID: 140262...
- Page 2 Copyright © 2004 - 2005 Topspin Communications, Inc. All rights reserved. The Topspin Server Switch, Topspin Host Channel Adapter, Topspin Element Manager, and collateral software programs and documentation are subject to and made available only pursuant to the license agreement signed by you and Topspin, Communications, Inc.
-
Page 3: Table Of Contents
Contents Regulatory Notices........vii FCC Statement ............................vii Safety Information ............................ vii Electrical Cautions ........................vii General Cautions........................... ix Laser Cautions ..........................ix Contact Information .............................x 1: Product Registration........xi THANK YOU FOR PURCHASING YOUR TOPSPIN® PRODUCT! ........... xi 2: Standard Product Agreement....xiii 3: Topspin 360 Overview ....... - Page 4 Introduction..............................5 Controller Cards............................6 Install a Controller Card ......................... 6 Ethernet and Fibre Channel Gateways......................9 Replacing and Updating Your Gateways..................9 Install an Ethernet or Fibre Channel Gateway................10 InfiniBand Switch Cards..........................10 Install InfiniBand Switch Cards....................11 Remove InfiniBand Switch Cards ....................13 Power Supply Modules ..........................13 Install Power Supply Modules ......................
- Page 5 Installing a Power Supply ......................44 Removing a Power Supply ......................45 Replacing Blower Trays ..........................46 Installing a Blower Tray ....................... 46 Removing a Blower Tray......................47 7: Topspin 360 LEDs........49 Introduction..............................49 Topspin 360 System LED..........................50 Power Supply LEDs...........................51 Blower Tray LEDs.............................52 Controller Card LEDs ..........................53 IB Switch Card LEDs ..........................54...
-
Page 7: Regulatory Notices
Regulatory Notices FCC Statement This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to Part 15 of the FCC Rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. - Page 8 viii CAUTION: Grounding is supplied by the ground-prong on the 3-prong power cable. Do not attach a separate ground cable. Do not use adapter plugs. Do not remove the ground prong from the cable. Ensure the ground connection on the power supply is correct and functioning before applying power to the Topspin 360.
-
Page 9: General Cautions
CAUTION: Use only Topspin-approved gateway expansion modules, controller cards, fans, and power supply modules. CAUTION: Never place your hand inside an empty card or module bay. You should never have cause to place a hand anywhere inside the Topspin chassis. Unused card and module bays should always have a Topspin cover over the bay to ensure proper safety, ventilation, and cooling. -
Page 10: Contact Information
CAUTION: Only open, modify, or service laser devices as specified in the manufacturer documentation. CAUTION: Use of controls or adjustments or performance of procedures other than those specified herein may result in hazardous laser radiation exposure. Laser Specs Table iii-1: Topspin P/N Application Optical Parameters... -
Page 11: 1: Product Registration
Product Registration THANK YOU FOR PURCHASING YOUR TOPSPIN® PRODUCT! Topspin Communications is pleased that you have selected Topspin for your computing infrastructure. Topspin provides comprehensive customer support resources via telephone and Web access. Topspin’s Customer Support Web site has the latest TopspinOS (IB Switch and I/O Chassis Operating System), Linux and Windows Host Side Drivers, Boot over InfiniBand Firmware and technical documentation. - Page 12 • Phone: (800) 499-1473 • Online Help Desk: http://support.topspin.com...
-
Page 13: 2: Standard Product Agreement
xiii Standard Product Agreement PLEASE READ THESE TERMS AND CONDITIONS CAREFULLY BEFORE USING THE PRODUCTS THAT ARE PROVIDED WITH THIS AGREEMENT. BY USING ANY PRODUCTS, YOU ACKNOWLEDGE THAT YOU HAVE READ AND UNDERSTOOD ALL THESE TERMS AND CONDITIONS AND YOU WILL BE CONSENTING TO BE BOUND BY THEM. - Page 14 Payment/Financing. Topspin will determine Customer's credit terms on a per-order basis and such terms are subject to a credit review and approval by Topspin in its sole discretion. The amount of credit may be changed or credit withdrawn by Topspin at anytime in its sole discretion. All payments shall be made in U.S.
- Page 15 Intellectual Property Rights. Title to and ownership of the Software and any associated documentation, and any improved, updated, modified or additional parts thereof, and all copyright, patent, trade secret, trademark and other intellectual property rights embodied in the Products, shall at all times remain the property of Topspin or Topspin's licensors.
- Page 16 HAVE OTHER RIGHTS, WHICH VARY FROM STATE/JURISDICTION TO STATE/JURISDICTION. 10. Confidentiality. “Confidential Information” means any oral, written, graphic or machine-readable information disclosed by Topspin that is designated in writing to be confidential or proprietary or should be reasonably understood to be confidential. Customer agrees to: (a) refrain from using Confidential Information except as necessary to exercise its rights herein and (b) use best efforts preserve and protect the confidentiality of the Confidential Information.
- Page 17 xvii acquisition or sale of all or substantially all of assets relating to this Agreement. Topspin or any successor may assign all of part of the right to payments under this Agreement. Any assignment or transfer of this Agreement made in contravention of the terms hereof shall be null and void. Subject to the foregoing, the terms and conditions of this Agreement shall be binding upon and inure to the benefit of the parties to it and their respective heirs, successors, assigns and legal representatives.
- Page 18 xviii...
-
Page 19: 3: Topspin 360 Overview
Topspin 360 Overview The following sections appear in this chapter: • “Introduction” on page 1 • “Features” on page 2 Introduction The Topspin 360 Server Switch provides data center managers with a high-performance, low-latency interconnect that runs Fibre Channel and Ethernet traffic transparently over an InfiniBand (IB) backplane. -
Page 20: Features
Figure 1-2 displays the back of the Topspin 360 Server Switch and numbers the slots on the chassis. Figure 1-2: Topspin 360 Chassis - Back View (Service End) with Slot Numbers The vertical slots at the top of the service end of the Topspin 360 are numbered 1 to 14 from left to right. The horizontal slots at the bottom of the service end of the chassis are numbered 15 to 16 from top to bottom. -
Page 21: Redundancy And High-Availability
Redundancy and High-Availability The Topspin 360 is architected for full redundancy within a single box. Every component in the data path can be dual-connected from end to end. For example, a server with a 2-port HCA can be dual-attached to the two switch blades on the Topspin 360. Each gateway is also dual attached to the dual switch blades. -
Page 23: 4: Topspin 360 Components
Topspin 360 Components The following sections appear in this chapter: • “Introduction” on page 5 • “Controller Cards” on page 6 • “Ethernet and Fibre Channel Gateways” on page 9 • “InfiniBand Switch Cards” on page 10 • “Power Supply Modules” on page 13 •... -
Page 24: Controller Cards
CAUTION: Never place your hand inside an empty card or module bay. You should never have cause to place a hand anywhere inside the Topspin 360 chassis. Unused card and module bays should always have a Topspin 360 blanking panel over the bay to ensure proper safety, ventilation, and cooling. Controller Cards Controller cards manage the Topspin 360 and provide Ethernet and Serial Console port access to the Server Switch. - Page 25 Remove the controller card from any packaging. Extend the ejector lever on the card completely. Insert the controller card module into slot 1 or slot 14 such that the ejector lever sits on the bottom-left-hand side of the module. Push the controller card firmly into the slot. The ejector lever begins to close. Figure 2-2: Inserting the Controller Card...
- Page 26 Push the ejector lever up until it clicks into place. Figure 2-3: Closing the Ejector Lever...
-
Page 27: Ethernet And Fibre Channel Gateways
Ethernet and Fibre Channel Gateways Topspin Ethernet and Fibre Channel gateways connect the Topspin 360, and IB-connected hosts, to IP and FC networks. All gateways are dual-connected to the redundant InfiniBand backplane of the Topspin 360. Figure 2-4 displays an Ethernet gateway. Figure 2-5 displays a Fibre Channel gateway. -
Page 28: Install An Ethernet Or Fibre Channel Gateway
Install an Ethernet or Fibre Channel Gateway To install a gateway, perform the following steps: Ground yourself appropriately. Remove the blanking panel if one resides in the slot. Remove the gateway module from any packaging. Extend the ejector lever on the gateway completely. Insert the gateway into an expansion module slot such that the ejector lever sits on the bottom-left-hand side of the module. -
Page 29: Install Infiniband Switch Cards
Install InfiniBand Switch Cards To install a switch card, perform the following steps: Ground yourself appropriately. Remove the blanking panel if one resides in the slot with a #1 Phillips-head screwdriver. Remove the switch card from any packaging. Extend the ejector levers on the left- and right-hand sides of the switch card completely. Insert the switch card into slot 15 or slot 16 such that the ejector levers sit on the bottom of the switch card. - Page 30 Close both fastener-levers simultaneously and completely. Figure 2-8: Closing Switch Ejector Levers...
-
Page 31: Remove Infiniband Switch Cards
Tighten the screws on either side of the switch card to secure it to the chassis. Figure 2-9: Securing Switch Card Remove InfiniBand Switch Cards To remove an InfiniBand switch card from the Topspin 360 chassis, perform the following steps: Make sure the Topspin 360 chassis is secure. -
Page 32: Install Power Supply Modules
power supply, you can add a second power supply while the chassis runs. If you have two power supplies installed, you can remove one of them without removing power from the chassis. Figure 2-10 displays the location of the power supplies in the Topspin 360. Power Supplies Figure 2-10: Power Supply Modules Install Power Supply Modules... -
Page 33: Removing A Power Supply
NOTE: The power socket on the back (service end) of the Topspin 360 corresponds to the power supply module on the same side of the Server Switch. Removing a Power Supply To remove the power supply, perform the following steps: Remove the power cord from the power supply. -
Page 34: Removing Blower Trays
Insert the blower tray into an open slot until it is fully seated, such that the captive screw sits at the top of the tray. Tighten the captive screw on the front panel to secure the tray to the chassis. Removing Blower Trays To remove a blower tray, perform the following steps: Ground yourself appropriately. - Page 35 Insert a replacement blower tray into the empty chassis bay to ensure appropriate cooling. NOTE: Each blower module contains two fans for a total of four fans. The system can tolerate one out of four blowers failing. When you remove a blower module, you have three minutes to replace it with a fully-functioning blower module.
-
Page 37: 5: Install And Boot The Topspin 360
Install and Boot the Topspin 360 The following sections appear in this chapter: • “Introduction” on page 19 • “Prepare the Site for the Topspin 360” on page 19 • “Install Topspin 360 Components” on page 20 • “Rack the Topspin 360” on page 20 •... -
Page 38: Read The Cautionary Statements
Read the Cautionary Statements Refer to the“FCC Statement” on page vii and the “Safety Information” on page vii before you install your Topspin 360. Prepare the Physical Environment for the Topspin 360 • Ground yourself using an approved ground wrist-strap. •... - Page 39 Attach the side flanges to the sides of the chassis with the screws provided, as shown Figure 3-1 Figure 3-2. Figure 3-1: Positioning the Side Flanges Figure 3-2: Fastening the Side Flanges If necessary for your rack, add fasteners with appropriate threads to your rack to prepare your rack for the rails.
- Page 40 Position the chassis brackets to the rack rails, as shown in Figure 3-3. Figure 3-3: Rail Positioning...
- Page 41 10. Screw the top and bottom of the front of the rails to the rack but do not fully tighten the screws. Figure 3-4 displays the rail loosely connected to the rack. NOTE: When you connect the rails, do not fully tighten the screws. When you slide the chassis into the rails, you may need to reposition the rails to align the flange screws with the rails.
- Page 42 11. Screw the top and bottom of the back of the rails to the rack, as shown in Figure 3-5. Figure 3-5: Attaching Rails from the Back...
- Page 43 12. Slide the Topspin 360 onto the rails as shown in Figure 3-6 Figure 3-7. Figure 3-6: Positioning the Chassis on the Rails Figure 3-7: Sliding the Chassis onto the Rails...
-
Page 44: Attach A Serial Console Cable To A Pc Or Terminal
13. Adjust the front of the rails (that you left loose in step 10) so that the threaded hole in the center of the front of the rail aligns with the screw in the flange. 14. Screw the flange screws tightly into the rails, as shown in Figure 3-8. -
Page 45: Boot Your Topspin 360 And Configure Basic Connectivity
• Stop Bits: 1 • Flow control: None For the VFrame Evaluation Test Plan, continue to Step b “Hardware Configuration Steps” on page 14 in the “VFrame Evaluation Test Plan”. Boot your Topspin 360 and Configure Basic Connectivity To configure basic connectivity, you must power-on the Server Switch and configure it with the Serial Console port and a terminal or workstation. -
Page 46: Obtain An Ip Address With Dhcp
Enter the interface mgmt-ethernet command. Example Topspin-360(config)# interface mgmt-ethernet Enter the IP address of the management port followed by the net mask. Example Topspin-360(config-if-mgmt-ethernet)# ip address 10.10.0.22 255.255.255.0 Set the default gateway of the management port. Example Topspin-360(config-if-mgmt-ethernet)# gateway 10.10.0.1 Enable the management port with the no shutdown command. -
Page 47: Connect Infiniband Hosts
Enable the management port with the no shutdown command. You can now telnet to your Topspin 360, or enter the IP address of your Topspin 360 in a Web browser to launch Chassis Manager. Example Topspin-360(config-if-mgmt-ethernet)# no shutdown Save the configuration to preserve it between reboots. Example Topspin-360(config-if-mgmt-ethernet)# exit Topspin-360(config)# exit... - Page 48 Use InfiniBand cables to connect the HCA in your host to the InfiniBand switch card of your Topspin 360. To plug in an InfiniBand cable, push the connector into the interface until you hear/feel a click. Figure 3-9: Fully Installed IB Cable with Pinch Connector Figure 3-10: Fully Installed IB Cable with Pull Connector NOTE: If your host does not provide an ample amount of free space around a given IB port, double-check that your IB cable connector engages fully.
- Page 49 To remove a cable with a pinch connector, pinch both sides of the back of the connector and pull the connector away from the port. Figure 3-11: Removing a Pinch Connector...
-
Page 50: Manage The Topspin 360
To remove a cable with a pull connector, grasp the connector with one hand and push it toward the port, then pull the latch away from the port with your other hand and gently wiggle the connector away from the port. 1. -
Page 51: Viewing And Updating The Software Image On The Chassis
Viewing and Updating the Software Image on the Chassis View the Software Image Version with the CLI To view the primary software image version for all cards in the chassis, use the show card-inventory command. Look for the primary-image-source field for each slot. (The slots are described in Figure 1-2.) -
Page 52: View The Software Image Version With The Gui
An installed image file will have a slash (/) in the name. Example Topspin-360> enable Topspin-360# dir image ============================================================================= Existing Boot-Images on System ============================================================================= slot date-created size file-name -------------------------------------------------------------------------------- 1 Tue Nov 25 18:46:32 2003 30695264 Topspin360-TopspinOS-2.0.0-build212.img 1 Tue Nov 25 19:34:08 2003 19228160 TopspinOS-2.0.0/build211 1 Tue Nov 25 19:32:16 2003 15539200 TopspinOS-2.0.0/build212 Topspin-360# View the Software Image Version with the GUI... -
Page 53: Upgrade The Software Image With The Cli
View the software version on individual cards in the chassis: Double-click on any card in the main GUI. Click the Inventory tab. The properties window appears for that card. Figure 3-14: View Current Image of an Individual Card View the Current Image Source field. If you find a card that has a different image than the others, you can update a single card by re-installing the image on the chassis, but rebooting only the card that needs to be updated. -
Page 54: Upgrade The Software Image With The Gui
NOTE: If you do not have enough available memory to install an additional image, delete an old image. “Delete an Image with the CLI” on page Specify the new boot image to be used. Find the name of your new system image by entering the dir image command. Copy the name of the new image. - Page 55 Click the Import button. The Import File window appears. Figure 3-15: Copy an Image File to the Switch Select Image from the File Type drop-down menu. Click the Remote FTP radio button from the Copy From section. Enter the name or IP address of the FTP server from which to copy the file in the Server Name or IP Address field.
-
Page 56: Delete An Image With The Cli
• All cards must have an oper-status of “Up” Click on the newly copied.img file in the File Management window. Click the Install button. Click Yes to the warning asking if you wish to proceed. NOTE: If you do not have enough available memory to install an additional image, delete an old image. -
Page 57: Delete An Image With The Gui
The following example deletes an image file from slot 1 (slots 1 and 14 can contain controller cards). Example Topspin-360# delete 1:image:Topspin360-TopspinOS-2.0.0-build488.img Delete file 14:Topspin360-TopspinOS-2.0.0-build488.img? [yes(default) | no] yes ****** Delete an Image with the GUI You can delete an .img file through the GUI or the CLI. A .img file is an image file that is not installed. Do not delete an installed image file. -
Page 59: 6: Topspin 360 Field Replaceable Units
Topspin 360 Field Replaceable Units The following sections appear in this chapter: • “Controller Cards” on page 41 • “Expansion Modules” on page 41 • “InfiniBand Switch Cards” on page 42 • “Power Modules” on page 44 • “Replacing Blower Trays” on page 46 Controller Cards The Topspin 360 has two removable controllers, one on each side of the Server Switch. -
Page 60: Infiniband Switch Cards
InfiniBand Switch Cards Each InfiniBand switch card is held in place by two screws, one on the left side and one on the right side of the card. Installing an InfiniBand Switch Card NOTE: Slot 16 must always have an Infiniband switch card inserted in order to maintain connectivity. To install the InfiniBand switch card in the Topspin 360 chassis: Remove the blanking panel from the switch card bay, if it is still in place. - Page 61 • The keyword all (The CLI defaults to “all” if you enter just “show card”). Example Topspin-360# show card 5,9,14,16 ============================================================================ Card Information ============================================================================ admin oper admin oper oper slot type type status status code ---------------------------------------------------------------------------- en4port1G en4port1G normal fc2port2G fc2port2G normal controller...
-
Page 62: Removing An Infiniband Switch Card
Reboot the switch card. Example Topspin-360# reload System configuration has been modified. Save? [yes(default)/no/*.cfg] yes Proceed with reload? [confirm] Topspin-360# Removing an InfiniBand Switch Card To remove an InfiniBand switch card from the Topspin 360 chassis: Make sure the Topspin 360 chassis is secure. Properly ground yourself to avoid potential electrostatic discharge and damage to the card. -
Page 63: Removing A Power Supply
NOTE: The power supplies should be inserted in opposite directions of each other. See Figure 4-2. Power Supply Figure 4-2: Power Supply Module Positioning Press the power supply firmly into the chassis. Removing a Power Supply To remove the power supply, perform the following steps: Remove the power cord from the power supply. -
Page 64: Replacing Blower Trays
Replacing Blower Trays The Topspin 360 fan tray is hot-swappable. You do not need to turn off power to the chassis to replace the fan tray. Access the fan tray from the front of the Topspin 360 chassis. Installing a Blower Tray To insert a blower tray, perform the following steps: If a blanking panel is already attached to the module bay, remove it. -
Page 65: Removing A Blower Tray
Tighten the two captive panel screws on the front panel to secure the card to the chassis. Removing a Blower Tray To remove a blower tray, perform the following steps: Unfasten the fan tray screws. Pull the fan toward you until it completely disengages from the chassis. Fasten a blanking panel to the empty chassis bay to ensure appropriate cooling. -
Page 67: 7: Topspin 360 Leds
Topspin 360 LEDs The following sections appear in this chapter: • “Introduction” on page 49 • “Topspin 360 System LED” on page 50 • “Power Supply LEDs” on page 51 • “Blower Tray LEDs” on page 52 • “Controller Card LEDs” on page 53 •... -
Page 68: Topspin 360 System Led
• Ethernet cards • Topspin 360 bezel side The LEDs have special significance depending on the chassis component. This section describes the LED indicators of each component Topspin 360 System LED The Topspin 360 system LED appears on the top-left-hand corner of the front (bezel end) of the Server Switch. -
Page 69: Power Supply Leds
Power Supply LEDs Power supply status LEDs are located on either side of the power supply module pull handle. Controller cards mirror Power Supply LEDs on the service end of the chassis. Figure 5-1 displays power supply LEDs. Figure 5-1: Power Supply LEDs Table 5-2 displays AC LED indications. -
Page 70: Blower Tray Leds
Blower Tray LEDs Blower tray LEDs appear immediately to the left of the System LED on the bezel end of the Topspin 360. Each blower LED is a dual-color green/yellow LED. Controller Cards mirror the blower LEDs. Figure 5-2 displays the blower LEDs. Figure 5-2: Blower Tray LEDs Table 5-4 explains blower tray LED indications. -
Page 71: Controller Card Leds
Controller Card LEDs Controller cards include card status LEDs as well as Management Ethernet port LEDs. The top-left-most LED represents the controller card in slot 1, and the top-right-most LED represents the controller card in port 14. Figure 5-3 displays the LEDs on the controller card. Controller Card 1 Status LED Controller Card 2 Status LED Management Ethernet Port Link LED... -
Page 72: Ib Switch Card Leds
IB Switch Card LEDs Each InfiniBand switch card includes two LEDs (one green and one yellow) at one side of the interface-end of the card. Figure 5-4 displays the IB switch card LEDs. Figure 5-4: InfiniBand Switch Card LEDs Table 5-7 explains IB switch card LED indications. -
Page 73: Fibre Channel Gateway Leds
Fibre Channel Gateway LEDs The FC gateway provides one yellow (amber) LED and one green LED. Table 5-9 lists and describes the LED indications. Table 5-9: Gateway LED Indications Display Indication yellow on, green off Auxiliary power has reached the gateway. green on, yellow off Gateway runs successfully. -
Page 74: Ethernet Gateway Leds
Table 5-11 explains the FC port activity LED. Table 5-11: Interpreting the Fibre Channel Port Activity LEDs Color Description No system power or LED failure. solid yellow Fan error detected. Operator intervention required. solid green Fans running with no detected errors. Ethernet Gateway LEDs The Ethernet gateway provides one yellow (amber) LED and one green LED. -
Page 75: 8: Specifications And Compliance
Specifications and Compliance This chapter details the Topspin 360 specifications and compliance certifications. General Specifications Table 6-1: General Specifications Specification Detail Operating Temperature Range 0 to 35C Non-Operating Temperature Range -40 to 65 degrees Celsius Operating Temperature Gradient 20C maximum per 60 min Operating Altitude 0 - 10,000 ft Non-Operating Alt. -
Page 76: Electrical Specifications
Table 6-1: General Specifications Specification Detail Non-Operating Vibration, Sinusoidal 3 axis with the following parameters • “X and Y axis: 0.0004 G 2 / Hz flat from 5-500Hz (0.472 GRMS) • Z-axis: 0.0008 G 2 / Hz flat from 5-500Hz (0.629GRMS) Weight <... - Page 77 Table 6-4: Laser Specifications Topspin P/N Application Optical Parameters *Standards used: CDRH - 21CFR Chapter I§ 1040.10; IEC - IEC60825-1:1994...
- Page 79 Index architecture ..............2 transparent topology ..........2 basic connectivity ..........27 configure basic connectivity .........27 connectivity, basic ..........27 controller, installing & removing ......41 high availability ............3 InfiniBand switch cards, installing & removing ...42 installing the chassis in a rack .......20 performance ............2 product registration ..........