1. Manuals
  2. Brands
  3. Oracle Manuals
  4. Switch
  5. Sun Datacenter InfiniBand Switch 648
  6. Service manual

Oracle Sun Datacenter InfiniBand Switch 648 Service Manual

Hide thumbs Also See for Sun Datacenter InfiniBand Switch 648:
Table of Contents

Advertisement

Quick Links

Download this manual
Sun Datacenter InfiniBand
Switch 648
Service Manual
Part No. 820-7740-12
May 2010, Revision A

Advertisement

Table of Contents
loading
Need help?

Need help?

Do you have a question about the Sun Datacenter InfiniBand Switch 648 and is the answer not in the manual?

Questions and answers

Subscribe to Our Youtube Channel

Related Manuals for Oracle Sun Datacenter InfiniBand Switch 648

Summary of Contents for Oracle Sun Datacenter InfiniBand Switch 648

  • Page 1 Sun Datacenter InfiniBand Switch 648 Service Manual Part No. 820-7740-12 May 2010, Revision A...
  • Page 2 Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.

  • Page 3: Table Of Contents

    Contents Using This Documentation vii Servicing the Switch 1 Understanding Service Procedures 1 Antistatic Precautions 2 Suggested Tools 2 Component Addressing 3 Connector Addressing 4 Replaceable Components 5 Servicing the Power Supplies 6 Inspecting the Power Supplies 7 ▼ Identify the Power Supply 7 ▼...
  • Page 4 Servicing the InfiniBand Cables 52 Inspecting the InfiniBand Cables 53 ▼ Identify the InfiniBand Cable 54 ▼ Inspect the InfiniBand Cable Hardware 55 ▼ Inspect the InfiniBand Cable Connectors 55 Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 5 ▼ Remove an InfiniBand Cable 56 ▼ Install an InfiniBand Cable 59 Servicing the Line Cards 64 Inspecting the Line Cards 65 ▼ Identify the Line Card 65 ▼ Inspect the Line Card Chassis 67 ▼ Inspect the Line Card Retainer Bolts 67 ▼...
  • Page 6 Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 7: Using This Documentation

    Using This Documentation This service manual provides detailed procedures that describe the removal and installation of components in the Sun Datacenter InfiniBand Switch 648 from ® Oracle . This document is written for technicians, system administrators, authorized service providers, and users who have advanced experience servicing hardware.
  • Page 8 Submit comments about this document by clicking the Feedback[+] link at (http://docs.sun.com). Include the title and part number of your document with your feedback: Sun Datacenter InfiniBand Switch 648 Service Manual, part number 820-7740-12. viii Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 9: Servicing The Switch

    Servicing the Switch Once a failed part is identified, it can be replaced. The topics listed here help you service Oracle’s Sun Datacenter InfiniBand Switch 648 chassis components. “Understanding Service Procedures” on page 1 ■ “Servicing the Power Supplies” on page 6 ■...

  • Page 10: Antistatic Precautions

    Proper servicing of the switch requires the following tools: No. 2 Phillips screwdriver ■ 10 mm socket or wrench ■ 20 mm socket or wrench ■ Ratchet drive for sockets ■ Flashlight ■ Magnifying glass ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 11: Component Addressing

    Other tools ■ Related Information “Antistatic Precautions” on page 2 ■ Component Addressing When you monitor or control switch components (power supplies, CMCs, line cards, and fabric cards) with administrative commands, you are often required to provide either a slot number or an Integrated Peripherals Management Bus (IPMB) address to the command.

  • Page 12: Connector Addressing

    IPMB address FRU numbers for the line cards. Connector IPMB Address Connector IPMB Address Connector IPMB Address Number FRU Number Number FRU Number Number FRU Number Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 13: Replaceable Components

    Related Information “Component Addressing” on page 3 ■ Replaceable Components The following illustration identifies the replaceable components of the switch. Replaceable Components FIGURE: Figure Legend InfiniBand cable Fabric card or Fabric card filler Line card Midplane stiffener or Line card filler Switch chassis Power Supply Related Information...

  • Page 14: Servicing The Power Supplies

    “Servicing the Fabric Cards and Fabric Card Fillers” on page 35 ■ “Servicing the InfiniBand Cables” on page 52 ■ “Servicing the Line Cards” on page 64 ■ “Doing Supportive Software Tasks” on page 82 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 15: Inspecting The Power Supplies

    Inspecting the Power Supplies Before installing a power supply, inspect its hardware and connectors to verify its suitability for installation. Step Description Links Identify the power supply. “Identify the Power Supply” on page 7 Inspect the power supply hardware. “Inspect the Power Supply Hardware” on page 9 Inspect the power supply connectors.
  • Page 16 ▼ Inspect the Power Supply Hardware 1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Inspecting the Power Supplies” on page Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 17: Inspect The Power Supply Hardware

    2. Unwrap the replacement power supply from its antistatic packaging. 3. Verify that there is no visible damage to the power supply chassis. 4. Verify that the release button unsecures the release handle and permits the handle to snap closed. 5.

  • Page 18: Inspect The Power Supply Connectors

    Note – Use the table in “Component Addressing” on page 3 to determine the slot number or IPMB address of your component. 4. At the rear of the chassis, remove the respective power cord. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 19: Remove A Power Supply

    Note – The power cord for power supply 0 is on the right side. The power cord for power supply 3 is of the left side. The power supply is completely powered off. 5. Remove the power supply. “Remove a Power Supply” on page Related Information Switch Reference, disablepsu command ■...
  • Page 20 5. Set the power supply aside. 6. Inspect the slot from where the power supply was removed for debris and loose gasketing. 7. Remove any debris or gasketing material from the slot. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 21: Install A Power Supply

    8. Install a replacement power supply. “Install a Power Supply” on page Related Information “Remove a CMC” on page 22 ■ “Remove a Fan” on page 31 ■ “Remove a Fabric Card or Fabric Card Filler” on page 44 ■ “Remove an InfiniBand Cable”...
  • Page 22 8. Power on the power supply. “Power On a Power Supply” on page Related Information “Remove a Power Supply” on page 11 ■ “Install a CMC” on page 24 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 23: Power On A Power Supply

    “Install a Fan” on page 33 ■ “Install a Fabric Card or Fabric Card Filler” on page 47 ■ “Install an InfiniBand Cable” on page 59 ■ “Install a Line Card” on page 77 ■ ▼ Power On a Power Supply Note –...
  • Page 24 Product Part / Model# = 300-2133-04 Product Version = 04 Product Serial Number = A234EP000B02P Asset Tag FRU Programmer File ID Multi Record Area: Record Type = Power Supply Information Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 25: Servicing The Cmcs

    Version = 2 Record Type = DC Output Version = 2 Record Type = DC Output Version = 2 Related Information Switch Reference, enablepsu command ■ Switch Reference, showpresent command ■ Switch Reference, psustatus command ■ “Power Off a Power Supply” on page 10 ■...

  • Page 26: Inspecting The Cmcs

    1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Inspecting the CMCs” on page 2. Use the illustration to identify the various features of a CMC. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 27 CMC Features FIGURE: Figure Legend Release levers Ready to Remove LED Attention LED OK LED NET MGT port Debug port SER MGT port 3. Inspect the CMC hardare. “Inspect the CMC Hardware” on page Related Information “Identify the Power Supply” on page 7 ■...

  • Page 28: Inspect The Cmc Hardware

    “Inspect the Fan Hardware” on page 30 ■ “Inspect the Fabric Card Chassis” on page 39 ■ “Inspect the InfiniBand Cable Hardware” on page 55 ■ “Inspect the Line Card Chassis” on page 67 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 29: Inspect The Cmc Connectors

    ▼ Inspect the CMC Connectors 1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Inspecting the CMCs” on page 2. Verify that the connectors are clean and without damage. 3.

  • Page 30: Remove A Cmc

    3. Loosen the two thumbscrews above the release levers. 4. Squeeze the latches on the release levers and wait for the blue Ready-to-Remove LED to light. 5. Swing the release levers outward. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 31 6. Slowly pull the CMC from the chassis. 7. Set the CMC aside. 8. Install a replacement CMC. “Install a CMC” on page Related Information “Remove a Power Supply” on page 11 ■ Servicing the Switch...

  • Page 32: Install A Cmc

    5. Set the back of the CMC onto the rails inside the chassis. 6. Carefully slide the CMC into the chassis until the release levers move. 7. Swing the release levers inward to secure the CMC into the chassis. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 33 8. Tighten the thumbscrews above the release levers. Note – Make the thumbscrews only finger tight. Do not use a screwdriver. 9. Reconnect the cables you removed previously to the front panel. 10. Power on the CMC. “Power On a CMC” on page Related Information “Remove a CMC”...

  • Page 34: Power On A Cmc

    = Sun Datacenter Switch DCS 648 - CMC board Board Serial Number = 0110SJC-0911CM0001 Board Part Number = 541-3807-02 FRU Programmer File ID = M9_CMC_FRU_541-3807-02_0110SJC-0911CM0001.txt Product Info Area: Version Language Code = 25 Manufacturer Name = Sun Microsystems Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 35: Servicing The Fans

    Product Name = Sun Datacenter Switch DCS 648 - CMC board Product Part / Model# = 541-3807-02 Product Version = Rev 50 Product Serial Number = 0110SJC-0905CM0003 Asset Tag = none FRU Programmer File ID = M9_CMC_FRU_541-3807-02_0110SJC-0911CM0001.txt Multi Record Area: PICMG Board Point-to-Point Connectivity Record (ID=0x14) Version = 0 Record Type...

  • Page 36: Inspecting The Fans

    1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Inspecting the Fans” on page 2. Use the illustration to identify the various features of a fan. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 37 Fan Features FIGURE: Figure Legend Thumbscrew Status LED Connector 3. Inspect the fan hardare. “Inspect the Fan Hardware” on page Related Information “Identify the Power Supply” on page 7 ■ “Identify the CMC” on page 18 ■ “Identify the Fabric Card” on page 37 ■...

  • Page 38: Inspect The Fan Hardware

    “Inspect the CMC Hardware” on page 20 ■ “Inspect the Fabric Card Chassis” on page 39 ■ “Inspect the InfiniBand Cable Hardware” on page 55 ■ “Inspect the Line Card Chassis” on page 67 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 39: Inspect The Fan Connector

    ▼ Inspect the Fan Connector 1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Inspecting the Fans” on page 2. Verify that the connector is clean and without damage. 3.
  • Page 40 4. Grasp the handle and pull the fan straight out. 5. Set the fan aside. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 41: Install A Fan

    6. Install a replacement fan. “Install a Fan” on page Related Information “Remove a Power Supply” on page 11 ■ “Remove a CMC” on page 22 ■ “Remove a Fabric Card or Fabric Card Filler” on page 44 ■ “Remove an InfiniBand Cable” on page 56 ■...
  • Page 42 The fan might immediately power on. 6. Tighten the captive thumbscrew to secure the fan in the fabric card. 7. Verify that the fan LED goes out. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 43: Servicing The Fabric Cards And Fabric Card Fillers

    8. Use the checkfans command to verify the fan’s operation. For example, to check the fans in fabric card 0: # checkfans |grep ‘FC 0’ Checking M9 fans... FC 0 Fan 0 RPM = 24356.000000 FC 0 Fan 1 RPM = 24356.000000 FC 0 Fan 2 RPM = 24070.000000 FC 0 Fan 3 RPM = 24356.000000 Note –...

  • Page 44: Inspecting The Fabric Cards

    Before installing a fabric card, inspect its chassis, hardware, and connectors to verify its suitability for installation. Step Description Links Identify the fabric card. “Identify the Fabric Card” on page 37 Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 45: Identify The Fabric Card

    Step Description Links Inspect the fabric card chassis. “Inspect the Fabric Card Chassis” on page 39 Inspect the fabric card retainer bolts. “Inspect the Fabric Card Retainer Bolts” on page 39 Inspect the fabric card fans. “Inspect the Fabric Card Fans” on page 40 Inspect the fabric card XBOW “Inspect the Fabric Card XBOW connectors.
  • Page 46 “Identify the CMC” on page 18 ■ “Identify the Fan” on page 28 ■ “Identify the InfiniBand Cable” on page 54 ■ “Identify the Line Card” on page 65 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 47: Inspect The Fabric Card Chassis

    ▼ Inspect the Fabric Card Chassis 1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Inspecting the Fabric Cards” on page 2. Unwrap the replacement fabric card from its antistatic packaging. 3.

  • Page 48: Inspect The Fabric Card Fans

    2. Place the fabric card on a work surface with the XBOW connectors facing you. 3. Using the flashlight positioned to the side and above you, look at the holes of the left-end connector. Look at the holes straight on. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 49 Note – Do not look at the individual holes, rather look at all of the connector’s holes as a group. A damaged or contaminated hole will be apparent. 4. Look for any closed over or contaminated holes. A closed-over or contaminated hole appears brighter than the surrounding holes. 5.

  • Page 50: Power Off A Fabric Card

    3. If you are permanently removing a fabric card from an InfiniBand fabric using the fat-tree routing algorithm, you must remove the respective GUIDs from the /etc/opensm/guid.txt file. Otherwise, go to Step Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 51 a. Type the following command on the CMC to retrieve the fabric card base GUID: # getbaseguid fc slot where slot is the number of the fabric card to be removed. For example: # getbaseguid fc 0 Base GUID = 0x00bad0fc0009f000 b.

  • Page 52: Remove A Fabric Card Or Fabric Card Filler

    See the table in “Servicing the Fabric Cards and Fabric Card Fillers” on page 2. Using a 10 mm socket and ratchet, turn the lower retainer bolt 1/4 turn counter-clockwise. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 53 3. Turn the upper retainer bolt 1/4 turn counter-clockwise. Caution – Do not turn the retainer bolts more than 1/4 revolution each time. Turning more than 1/4 turn damages the fabric card and midplane XBOW connectors. 4. Repeat Step 2 Step 3 at least 12 times until you feel very little resistance to turning and the fabric card is extended approximately 10 mm (0.5 in.) from the...
  • Page 54 “Remove a CMC” on page 22 ■ “Remove a Fan” on page 31 ■ “Remove an InfiniBand Cable” on page 56 ■ “Remove a Line Card” on page 71 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 55: Install A Fabric Card Or Fabric Card Filler

    ▼ Install a Fabric Card or Fabric Card Filler Note – Installing a fabric card or a fabric card filler is the same task. Caution – Installing the fabric card is a methodical task and cannot be rushed. To prevent damage to the fabric card or the midplane XBOW connectors, pay close attention to the procedure and work slowly.
  • Page 56 When you feel resistance in the retainer bolts, stop. 8. Using a 10 mm socket and ratchet, turn the lower retainer bolt 1/4 turn clockwise. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 57 9. Turn the upper retainer bolt 1/4 turn clockwise. Caution – Do not turn the retainer bolts more than 1/4 revolution each time. Turning more than 1/4 turn damages the fabric card and midplane XBOW connectors. 10. Repeat Step 8 Step 9 about 12 times until you feel resistance to turning and the fabric card is flush to the rear of the chassis.

  • Page 58: Power On A Fabric Card

    Type the following command on the CMC to retrieve the fabric card base GUID: # getbaseguid fc slot where slot is the number of the fabric card you installed. For example: # getbaseguid fc 0 Base GUID = 0x00bad0fc0009f000 Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 59 b. For the GUID, add the following values to make two new sums: 0xa2 ■ 0xb2 ■ Note – For a GUID that ends with two zeros (00), you are effectively replacing the zeros with a2 and b2. For example, 0x00bad0fc0009f000 becomes: 0x00bad0fc0009f0a2 ■...

  • Page 60: Servicing The Infiniband Cables

    The following table provides a description of a goal you want to achieve, and links to the sequential tasks required to reach that goal. Description Links Add an InfiniBand cable. “Inspecting the InfiniBand Cables” on page 53 “Install an InfiniBand Cable” on page 59 Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 61: Inspecting The Infiniband Cables

    Description Links Replace an InfiniBand cable. “Inspecting the InfiniBand Cables” on page 53 “Remove an InfiniBand Cable” on page 56 “Install an InfiniBand Cable” on page 59 Subtract an InfiniBand cable. “Remove an InfiniBand Cable” on page 56 Related Information “Servicing the Power Supplies”...

  • Page 62: Identify The Infiniband Cable

    “Identify the CMC” on page 18 ■ “Identify the Fan” on page 28 ■ “Identify the Fabric Card” on page 37 ■ “Identify the Line Card” on page 65 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 63: Inspect The Infiniband Cable Hardware

    ▼ Inspect the InfiniBand Cable Hardware 1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Inspecting the InfiniBand Cables” on page 2. Verify that the cable is not cut or damaged. 3.

  • Page 64: Remove An Infiniband Cable

    3. Grasp the cable connector to support its weight and apply the removal force. 4. Pull on the loop of the retractor strap while simultaneously pulling on the cable connector. The cable connector comes free. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 65 5. Carefully move the cable out of the main trunk of the cable tree. Servicing the Switch...
  • Page 66 Step c. Go to 7. If you are disconnecting all cables for a line card replacement, gently lower the cable so that it is hanging from the side branch. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 67: Install An Infiniband Cable

    8. Consider your next steps: If you are removing a cable from connectors 0A through 5B, repeat Step 3 ■ through Step 7, moving to the left for cables 5B, 4A, 4B, and so on to 0B or the cable that needs replacement. If you are removing a cable from connectors 6A through 11B, repeat Step 3 ■...
  • Page 68 (127 mm) radius. Do not allow any optical InfiniBand cable to bend tighter than a 3.4 inch (85 mm) radius. Tight bends can damage the cable internally. 10. Plug the cable into the appropriate connector. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 69 a. Visually inspect the cable connector to see whether the shell is not bent and is parallel to the inner boards. If the shell is bent or damaged, get a different cable. b. Ensure that the retraction strap is forward. c.
  • Page 70 10, moving to the right, working your way to connector 5B. Then repeat Step 8 through Step 10 with the cable to connector 11A, moving to the left, working your way to connector 6B. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 71 For example, 0B, 0A, 1B and so on to 5B and 5A. Then 11B, 11A, 10B, and so on to 6B and 6A. 12. (Optional) Display and record the replacement cable’s FRU ID information: # clia fruinfo IPMB_address FRU_number where: IPMB_address is from “Component Addressing”...

  • Page 72: Servicing The Line Cards

    “Remove an InfiniBand Cable” on page 56 “Remove a Line Card” on page 71 “Install a Line Card Filler” on page 75 Related Information “Servicing the Power Supplies” on page 6 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 73: Inspecting The Line Cards

    “Servicing the CMCs” on page 17 ■ “Servicing the Fans” on page 27 ■ “Servicing the Fabric Cards and Fabric Card Fillers” on page 35 ■ “Servicing the InfiniBand Cables” on page 52 ■ “Doing Supportive Software Tasks” on page 82 ■...
  • Page 74 “Identify the CMC” on page 18 ■ “Identify the Fan” on page 28 ■ “Identify the Fabric Card” on page 37 ■ “Identify the InfiniBand Cable” on page 54 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 75: Inspect The Line Card Chassis

    ▼ Inspect the Line Card Chassis 1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Inspecting the Line Cards” on page 2. Unwrap the replacement line card from its antistatic packaging. 3.

  • Page 76: Inspect The Line Card Xbow Connectors

    5. Look for any enlarged or cracked holes. An enlarged or cracked hole appears darker than the surrounding holes. 6. Move your head slightly right to the next connector, and inspect its pins, repeating Step 4 Step Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 77: Inspect The Line Card Cxp Connectors

    7. Continue in this manner for each connector until you reach the right side of the line card. 8. Inspect the CXP connectors. “Inspect the Line Card CXP Connectors” on page Related Information “Inspect the Power Supply Connectors” on page 10 ■...

  • Page 78: Power Off A Line Card

    Note – Once deactivated and disabled, the links provided by the line card are now lost. The Subnet Manager will attempt to establish new routes for those links while you continue the procedure. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 79: Remove A Line Card

    3. Remove the line card. “Remove a Line Card” on page Related Information Switch Reference, deactivate command ■ Switch Reference, disableboard command ■ “Power On a Line Card” on page 81 ■ “Power Off a Power Supply” on page 10 ■...
  • Page 80 4. Slowly move the cable shelf out and away from the cable trees. 5. Set the cable shelf aside. 6. Using a 20 mm socket and ratchet, turn the left retainer bolt 1/4 turn counter-clockwise. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 81 7. Turn the right retainer bolt 1/4 turn counter-clockwise. Caution – Do not turn the retainer bolts more than 1/4 revolution each time. Turning more than 1/4 turn damages the line card and midplane XBOW connectors. 8. Repeat Step 6 Step 7 at least 12 times until you feel very little resistance to turning and the line card is extended approximately 10mm (1/2 in.) from the...
  • Page 82 “Remove a Power Supply” on page 11 ■ “Remove a CMC” on page 22 ■ “Remove a Fan” on page 31 ■ “Remove a Fabric Card or Fabric Card Filler” on page 44 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 83: Install A Line Card Filler

    “Remove an InfiniBand Cable” on page 56 ■ “Remove a Line Card Filler” on page 76 ■ ▼ Install a Line Card Filler 1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Servicing the Line Cards”...

  • Page 84: Remove A Line Card Filler

    2. Determine which line card filler is to be removed. 3. Loosen the retainer bolts until they turn freely. 4. Lift the filler away from the chassis by the handles. 5. Set the filler aside. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 85: Install A Line Card

    6. Install a line card into the slot. “Install a Line Card” on page Related Information “Install a Line Card Filler” on page 75 ■ “Remove a Power Supply” on page 11 ■ “Remove a CMC” on page 22 ■ “Remove a Fan”...
  • Page 86 When you feel resistance in the retainer bolts, stop. 9. Using a 20 mm socket and ratchet, turn the left retainer bolt 1/4 turn clockwise. Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...
  • Page 87 10. Turn the right retainer bolt 1/4 turn clockwise. Caution – Do not turn the retainer bolts more than 1/4 revolution each time. Turning more than 1/4 turn damages the line card and midplane XBOW connectors. 11. Repeat Step 9 Step 10 about 12 times until you feel resistance to turning and the line card is flush to the front of the chassis.
  • Page 88 ■ “Install a Fabric Card or Fabric Card Filler” on page 47 ■ “Install an InfiniBand Cable” on page 59 ■ “Install a Line Card Filler” on page 75 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 89: Power On A Line Card

    ▼ Power On a Line Card Note – Use the table in “Component Addressing” on page 3 to determine the slot number or IPMB address of your component. 1. Identify the prerequisite and subsequent tasks you must perform in conjunction with this procedure, to achieve a goal. See the table in “Servicing the Line Cards”...

  • Page 90: Doing Supportive Software Tasks

    “Configure CMCs for Identical Addresses” on page 85 ■ “Upgrading the Firmware” on page 87 ■ “Set Up a TFTP Server” on page 87 ■ Related Information Switch Remote Management, upgrading the firmware ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 91: Reperform Initial Setup

    ▼ Reperform Initial Setup Follow this procedure when you install a new CMC and want to configure the network management parameters. 1. Access the new CMC from the serial management port. 2. Log in, then invoke a reboot of the CMC: # reboot The CMC reboots.
  • Page 92 11. Open a Telnet session and connect to the CMC by specifying the CMC’s network address. For example: % telnet 123.45.67.89 Trying 123.45.67.89... Connected to 123.45.67.89. Escape character is ’^]’. shmm1500 login: Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 93: Configure Cmcs For Identical Addresses

    12. Login as root using the root password. shmm1500 login: root Password: password The # prompt is displayed. Note – As shipped, the root password is not set and therefore not required. Press the Enter key when prompted for the password. Refer to the Sun Datacenter InfiniBand Switch 648 Administration Guide for instructions on how to set the root password.
  • Page 94 CMC is reachable at the same active state IP address. Related Information Switch Installation, powering on the switch ■ “Reperform Initial Setup” on page 83 ■ Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 95: Upgrading The Firmware

    With ILOM support, upgrades of CMC, fabric card, fabric card filler, and line card firmware become a much simpler task. For instructions to upgrade firmware through ILOM, see the Sun Datacenter InfiniBand Switch 648 Integrated Lights Out Manager (ILOM) 3.0 Supplement, part number 821-0896.
  • Page 96 Note – TFTP is not the same as FTP. It does not display the same error messages as FTP, and you cannot use the cd,ls, or most other commands recognized by FTP Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

  • Page 97: Index

    Index setenv, 83 setlanconfig, 26, 83 activate command, 50, 81 showpresent, 15 addressing switchover, 21, 26 CXP connector, 4 telnet, 83 antistatic precautions, 2 user add, 26 version, 26 component checkfans command, 33 addressing, 3 CXP connector configuring identical addresses, 85 addressing, 4 features, 18 firmware...
  • Page 98 inspecting, 36 servicing, 52 chassis, 39 initial fans, 40 setup, 83 retainer bolts, 39 inspecting XBOW connectors, 40 CMC, 18 installing, 47 connectors, 21 powering off, 42 hardware, 20 powering on, 50 fabric card, 36 removing, 44 chassis, 39 servicing, 35 fans, 40 retainer bolts, 39 features, 28...
  • Page 99 removing, 76 fabric card, 44 FRU ID, 81 fabric card filler, 44 inspecting, 65 fan, 31 chassis, 67 InfiniBand cable, 56 CXP connectors, 69 line card, 71 retainer bolts, 67 line card filler, 76 XBOW connectors, 68 power supply, 11 installing, 77 replaceable components, 5 powering off, 70...
  • Page 100 Sun Datacenter InfiniBand Switch 648 Service Manual • May 2010...

Table of Contents