Sun Microsystems PCI Installation Manual
  1. Manuals
  2. Brands
  3. Sun Microsystems Manuals
  4. PCI Card
  5. PCI
  6. Installation manual
Sun Microsystems PCI Installation Manual

Sun Microsystems PCI Installation Manual

8 gb fc pci-express hba, emulex
Hide thumbs
Table of Contents

Advertisement

Quick Links

Download this manual
Sun™ Dual Port 4x IB Host Channel
Adapter PCI-X Installation and
User's Guide
Sun Microsystems, Inc.
www.sun.com
Part No. 819-1280-10
September 2005, Revision A
Submit comments about this document at:
http://www.sun.com/hwdocs/feedback

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the PCI and is the answer not in the manual?

Questions and answers

Related Manuals for Sun Microsystems PCI

Summary of Contents for Sun Microsystems PCI

  • Page 1 Sun™ Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide Sun Microsystems, Inc. www.sun.com Part No. 819-1280-10 September 2005, Revision A Submit comments about this document at: http://www.sun.com/hwdocs/feedback...
  • Page 2 Copyright 2005 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, Californie 95054, Etats-Unis. Tous droits réservés. Sun Microsystems, Inc. a les droits de propriété intellectuels relatants à la technologie qui est décrit dans ce document. En particulier, et sans la limitation, ces droits de propriété...

  • Page 3: Table Of Contents

    Related Documentation xix Documentation, Support, and Training xx Third-Party Web Sites xx Contacting Sun Technical Support xx Sun Welcomes Your Comments xxi Product Overview 1 Product Features 1 Hardware Description 1 I/O Interfaces 3 InfiniBand Interface 3 PCI-X Interface 3...
  • Page 4 InfiniBand Cable Pin Assignments 19 Diagnostic Software and Troubleshooting 23 SunVTS Diagnostic Testing 23 Using the SunVTS ibhcatest 24 ▼ Using the ibhcatest 25 Troubleshooting Tasks 25 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...
  • Page 5 Other Useful Utilities 26 cfgadm 26 snoop 27 netstat 27 kstat 27 Index 29 Contents...
  • Page 6 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...
  • Page 7 Figures Sun Dual Port 4x IB Host Channel Adapter PCI-X Card 2 FIGURE 1-1 LED Layout 4 FIGURE 1-2 Typical Adapter-to-Switch Configuration 9 FIGURE 2-1 InfiniBand Software Stack 16 FIGURE 4-1 Sun 4x-to-12x InfiniBand Cable Pin Assignments 21 FIGURE A-1...
  • Page 8 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...
  • Page 9 Tables IB-HCA LEDs 3 TABLE 1-1 Hardware, Software, and Cable Requirements 5 TABLE 1-2 SunVTS Documentation 24 TABLE B-1...
  • Page 10 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 11: Fcc Class A Notice

    Operation of this equipment in a residential area is likely to cause harmful interference, in which case the user will be required to correct the interference at his own expense. Modifications: Any modifications made to this device that are not approved by Sun Microsystems, Inc. may void the authority granted to the user by the FCC to operate this equipment.

  • Page 12: Bsmi Class A Notice

    BSMI Class A Notice The following statement is applicable to products shipped to Taiwan and marked as Class A on the product compliance label. Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...
  • Page 13 CCC Class A Notice The following statement is applicable to products shipped to China and marked with “Class A” on the product’s compliance label. GOST-R Certification Mark Regulatory Compliance Statements xiii...
  • Page 14 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 15: Supplementary Information

    4150 Network Circle, MPK15-102 Santa Clara, CA 95054 U.S.A. Tel: 650-786-3255 Fax: 650-786-3723 Declaration of Conformity IB-HCA Sun Dual Port 4x IB Host Channel Adapter PCI-X TÜV Rheinland Certificate No. CB Scheme Certificate No. File: DATE Donald Cameron Program Manager...
  • Page 16 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 17: Preface

    These instructions are designed for system administrators with experience installing network hardware and software. Note – The Sun Dual Port 4x IB Host Channel Adapter PCI-X card is an InfiniBand (IB) host channel adapter (HCA) card. The adapter is referred to as IB-HCA in this manual.

  • Page 18: Using Unix Commands

    Shell Prompts Shell C shell C shell superuser Bourne shell and Korn shell Bourne shell and Korn shell superuser xviii Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 Prompt machine_name% machine_name#...

  • Page 19: Typographic Conventions

    Related Documentation The following documents provide additional information regarding the IB-HCA and related products. Sun Dual Port 4x IB Host Channel Adapter PCI-X Release Notes (819-1281) System Administration Guide: Devices and Files System (817-5093) System Administration Guide: Network Services (816-4555)

  • Page 20: Documentation, Support, And Training

    If you have technical questions about this product that are not answered in this document, go to: http://www.sun.com/service/contacting xx Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 Description Download PDF and HTML documents,...

  • Page 21: Sun Welcomes Your Comments

    You can submit your comments by going to: http://www.sun.com/hwdocs/feedback Please include the title and part number of your document with your feedback: Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide, part number 819-1280-10 Preface...
  • Page 22 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 23: Hardware Description

    ■ Hardware Description The Sun Dual Port 4x IB Host Channel Adapter PCI-X card is an InfiniBand (IB) host channel adapter (HCA) card and is referred to as the IB-HCA in this manual. This adapter is a PCI-X Revision 2.2, 64-bit, 133MHz compatible card with dual 4x MicroGigaCN InfiniBand-compliant connectors for copper cables.

  • Page 24: Figure 1-1 Sun Dual Port 4X Ib Host Channel Adapter Pci-X Card

    Low profile PCI form-factor card (approx. 2.5 in. by approx. 6.7 in.) ■ Requires 4x InfiniBand cables ■ 3.3V PCI-X or PCI slot (Note that the IB-HCA can not be plugged into a 5V ■ connector.) Link status indication LED ■...

  • Page 25: I/O Interfaces

    The PCI-X bus is a PCI-X version 2.2-compliant 64-bit 133 MHz interface. The adapter can be either a master initiating the PCI bus operations or a slave responding to PCI bus operations. The PCI/PCI-X bus can connect to a host CPU in an HCA application.

  • Page 26: Figure 1-2 Led Layout

    LED Layout FIGURE 1-2 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 Physical link (Green LED) Logical link (Amber LED)

  • Page 27: Hardware, Software, And Cable Requirements

    Hardware, Software, and Cable Requirements The hardware, software, and cable requirements for the Sun Dual Port 4x IB Host Channel Adapter PCI-X card are listed in Hardware, Software, and Cable Requirements TABLE 1-2 Operating System Servers Solaris 10 SPARC Sun Fire™ V240, V440, V490, V890, V1280, E2900, E4900, E6900,...
  • Page 28 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 29: Adapter Installation

    For optimal performance, the IB-HCA card should be installed in the fastest available bus, preferably a 133 MHz PCI-X bus. But the IB-HCA can operate on the slower PCI-X and PCI buses. The IB-HCA should also be installed on its own PCI leaf.
  • Page 30 2. Remove the cover from the unit to access the card slots and connectors. 3. Select an available 3.3 Volt PCI-X or PCI slot and remove the blank front panel. If you are replacing an existing card, remove the card.

  • Page 31: Verifying The Installation

    This completes the hardware installation. Proceed to the verification instructions in “Verifying the Installation” on page Sun 9-Port InfiniBand switch Sun 9-Port InfiniBand switch Typical Adapter-to-Switch Configuration FIGURE 2-1 ▼ Verifying the Installation Before you can verify the installation, you must install the adapter in the chassis, power the server, and cable it to an operational InfiniBand switch.
  • Page 32 2. Check that the amber LED is illuminated for each port that is connected to the switch. If the amber LED is not on, refer to Appendix “Diagnostic Software and Troubleshooting” on page 23 for more information. Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 33: Configuring Ip Over Infiniband

    C H A P T E R Configuring IP Over InfiniBand This chapter contains the following sections: “IPoIB Configuration” on page 11 ■ “Managing IB Devices for IPoIB” on page 13 ■ IPoIB Configuration IP Over InfiniBand (IPoIB) is automatically configured when the system is booted. The following procedure can be used when making changes to your configuration.
  • Page 34 IPv4 address for the device. If you are using IPv6, use the following command instead: # ifconfig ibdn inet6 plumb host-ipv6-address up Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 host-ipv4-address is the designated...

  • Page 35: Managing Ib Devices For Ipoib

    5. Verify that the devices are plumbed. To do this, type the command: # ifconfig -a The output includes information on the IB devices. 6. Check that data can be send over the InfiniBand link. Use the ping command to send ICMP (ICMP6) ECHO_REQUEST packets and receive ICMP (ICMP6) ECHO_RESPONSE packets between InfiniBand hosts.
  • Page 36 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 37: Infiniband Software Overview

    C H A P T E R InfiniBand Software Overview This chapter includes the following topics: “InfiniBand Software” on page 15 ■ “Related Software Documentation” on page 18 ■ InfiniBand Software The InfiniBand software is included in the Solaris 10 Operating System. shows a block diagram of the InfiniBand software stack which includes the following modules: Solaris 10 SPARC®...

  • Page 38: User Direct Access Programming Library

    IB multicast support, extends ARP/ND address resolution, and changes the network stack to support link addresses greater than 6 bytes. Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 Solaris 10 OS...

  • Page 39: Infiniband Transport Framework

    InfiniBand Transport Framework IBTF is the Solaris implementation of the InfiniBand Transport Layer (IBTL). IBTF consists of InfiniBand Device Manager (IBDM), InfiniBand Communications Manager (IBCM) and an InfiniBand Nexus driver. The Nexus driver along with IBDM creates and manages InfiniBand devices. The following devices are supported: IB I/O controller (IOC) devices ■...

  • Page 40: Related Software Documentation

    – InfiniBand Communication Manager ■ ibd(7D) – Infiniband IPoIB device driver ■ ibdm(7D) – Solaris InfiniBand device manager ■ tavor(7D) – InfiniBand (IB) Tavor driver ■ Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 41: Infiniband Cable Pin Assignments

    A P P E N D I X InfiniBand Cable Pin Assignments shows the InfiniBand cable connectors and pin assignments. FIGURE A-1...

  • Page 42: Figure A-1 Sun 4X-To-12X Infiniband Cable Pin Assignments

    P A I R 8 Sun 4x-to-12x InfiniBand Cable Pin Assignments FIGURE A-1 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 P 1 ( 1 2 X ) S 4 1 S 4 2...

  • Page 43: Diagnostic Software And Troubleshooting

    A P P E N D I X Diagnostic Software and Troubleshooting This appendix provides an overview of the SunVTS™ diagnostic application and troubleshooting tips for checking the adapter. There is also a section outlining some common troubleshooting issues. This appendix contains the following sections: “SunVTS Diagnostic Testing”...

  • Page 44: Using The Sunvts Ibhcatest

    Use the following procedure when running the ibhcatest. Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 Descriptions Describes the new SunVTS features and tests, including the ibhcatest.

  • Page 45: B. Diagnostic Software And Troubleshooting

    Using the ibhcatest ▼ 1. Ensure that the SunVTS software and the Tavor driver are installed on your system. To do this, type the following command: # pkginfo SUNWvts SUNWvtsx SUNWtavor If a SunVTS software package is not installed, refer to the SunVTS User’s Guide for installation instruction.

  • Page 46: Other Useful Utilities

    Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005 is the instance of the Tavor device number and m is the port Type Receptacle Occupant IB-HCA connected...

  • Page 47: Snoop

    snoop The snoop program captures and inspects network packets. See the snoop(1M) man page for details. For example: # snoop -d ibd1 Using device /dev/ibd1 (promiscuous mode) ib-1-167 -> * ib-1-168 -> ib-1-167 0:2:4:7:0:0:0:0:a:4:7c:4f:0:2:c9:2:0:0:55:91 ib-1-167 -> ib-1-168 ib-1-168 -> ib-1-167 netstat netstat shows network status.
  • Page 48 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

  • Page 49: Index

    18 SunVTS, 23 I/O interfaces, 3 IB-HCA installation, 8 Sun Dual Port 4x IB Host Channel Adapter PCI-X card, 1 ibhcatest diagnostics, 24 IBMF, 15, 17 IBTF, 15, 17 InfiniBand cable, 19 InfiniBand Management Framework, 17 InfiniBand software overview, 15 InfiniBand Transport Layer, 17...
  • Page 50 5 Tavor driver, 17 troubleshooting IB-HCA, 25 tasks, 25 utilities, 26 uDAPL, 15, 16 utilities, 26 cfgadm, 26 kstat, 27 netstat, 27 snoop, 27 Sun Dual Port 4x IB Host Channel Adapter PCI-X Installation and User’s Guide • September 2005...

This manual is also suitable for:

Storagetek sg-xpcie1fc-em8-zStoragetek sg-xpcie2fc-em8-z

Table of Contents