Summary of Contents for Oracle InfiniBand Switch IS2-46
Page 1
Oracle InfiniBand Switch IS2-46 Service ® Manual Part No: E65869-01 September 2016...
Page 3
Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.
Page 4
Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers, sauf mention contraire stipulée dans un contrat entre vous et Oracle. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l’accès à...
Page 6
▼ Enable Oracle ILOM Legacy Targets ........... 52 ▼ Display Status for Specific Components .......... 53 ▼ Clear a Fault Manually .............. 54 Oracle ILOM Legacy Targets for Fault State and FRUID ...... 55 Oracle ILOM Legacy Targets for Alarm State .......... 56 Oracle ILOM Legacy Targets for Power Consumption ......... 58 Oracle ILOM Legacy Targets for Voltage ..........
Page 7
Contents ▼ Remove the Air Duct Assembly .............. 68 ▼ Remove a Snap Bracket ................ 72 ▼ Prepare to Power Off the Switch .............. 75 ▼ Power Off a Power Supply ................ 76 Servicing Data Cables .................. 81 Identifying Data Cable Types ................ 82 PrizmMT Cables ................... 83 SFP+ Cable Transceivers ................
Page 8
▼ Power On a Power Supply ............... 157 ▼ Connect the SER MGT Cable .............. 158 ▼ Configure the Replacement Switch ............ 161 ▼ Install a Snap Bracket ................ 164 ▼ Install the Air Duct Assembly .............. 166 Glossary ...................... 169 Index ........................ 177 Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Using This Documentation Overview – Describes how to troubleshoot and service the Oracle InfiniBand Switch IS2- ■ Audience – Technicians, system administrators, and authorized service providers. ■ Required knowledge – Advanced experience servicing network hardware. ■ Product Documentation Library Documentation and resources for this product and related products are available at http://www.
Page 10
Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Identifying Components These topics identify switch components and provide links to service tasks. “Switch Components” on page 12 ■ “Front Panel Components” on page 13 ■ “Rear Panel Components” on page 14 ■ Related Information “Detecting and Managing Faults” on page 15 ■...
SER MGT module (SMM) (battery inside) “Servicing the Battery” on page 107 Power supply “Servicing Power Supplies” on page 129 Related Information “Front Panel Components” on page 13 ■ “Rear Panel Components” on page 14 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Fan modules “Servicing Fan Modules” on page 119 RFID tag “Read the Switch Serial Number (Oracle ILOM)” on page 66 “Remove the Air Duct Assembly” on page 68 SER MGT port “Access the SP (SER MGT Port)” on page 38 SMM, with battery inside “Servicing the Battery”...
RJ-45 connectors “RJ-45 Cables” on page 84 Chassis status LEDs “Front Panel LEDs” on page 33 Related Information “Switch Components” on page 12 ■ “Front Panel Components” on page 13 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
■ Software – The solution is beyond the scope of this document. ■ Refer to the Oracle EDR InfiniBand Switch and Virtualized I/O Systems Administration Guide and the software troubleshooting documentation. If necessary, contact your authorized Oracle Service Provider. Are you sure you know the solution to the hardware problem? Yes –...
No – Go to Step ■ Check the power cords and supplied power, and correct as necessary. Refer to the Oracle InfiniBand Switch IS2-46 Installation Guide, then return to Step Is a power supply amber Attention LED on? Yes – Go to Step ■...
Page 18
Inspect and replace cabling as necessary. Then return to Step Are you sure you know that the communication hardware (terminals, routers, and switches) are fully functional? Yes – Go to Step ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
“Perform Additional Diagnostics at the Switch” on page 19 ■ “Perform Diagnostics Remotely” on page 21 ■ “Perform Oracle ILOM Target Diagnostics” on page 25 ■ Perform Additional Diagnostics at the Switch This procedure assumes that the air duct assembly has been removed and the SER MGT Note - cable remains attached.
Page 20
No – Go to Step ■ Are you sure you know that the cabling for the suspect ports is wired correctly with good cables and firm connections? Yes – Go to Step ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
“Perform Diagnostics at the Switch” on page 17 ■ “Perform Diagnostics Remotely” on page 21 ■ “Perform Oracle ILOM Target Diagnostics” on page 25 ■ Perform Diagnostics Remotely Are you starting a new troubleshooting session with this task? Yes – Go to “Begin the Diagnostic Process”...
Page 22
-> set /SP/network commitpending=true Then return to Step Can you access the SP from the terminal server or serial interface? “Access the SP (SER MGT Port)” on page 38, then return to this step. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 23
■ Stop bits – 1 ■ Flow control – none ■ From the network, configure the correct serial parameters through Oracle ILOM. “Access the SP (RJ-45 Port)” on page 37, then return to this step. Type: -> set /SP/serial/external pendingflowcontrol=none ->...
Page 24
“Begin the Diagnostic Process” on page 16 ■ “Perform Diagnostics at the Switch” on page 17 ■ “Perform Additional Diagnostics at the Switch” on page 19 ■ “Perform Oracle ILOM Target Diagnostics” on page 25 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Are any individual components unhealthy? Yes – Go to Step ■ No – Go to Step ■ Are you sure that Oracle ILOM legacy targets are enabled? Yes – Go to Step ■ No – Go to Step ■ Enable Oracle ILOM legacy targets.
Legacy Target Type Links Alarm state and FRUID “Oracle ILOM Legacy Targets for Fault State and FRUID” on page 55 Power consumption “Oracle ILOM Legacy Targets for Power Consumption” on page 58 Voltage “Oracle ILOM Legacy Targets for Voltage” on page 58 Temperature “Oracle ILOM Legacy Targets for Temperature”...
/SP and all subtargets Battery “Servicing the Battery” on page 107 /SP/clock Related Information “Oracle ILOM /System Target Replaceable Components” on page 27 ■ “Oracle ILOM Legacy Target Replaceable Components” on page 28 ■ “Faulted Item Replaceable Components” on page 30 ■...
Power supply 1 “Servicing Power /System/Power/Power_Supplies/Power_Supply_1 Supplies” on page 129 Related Information “Oracle ILOM /HOST and /SP Target Replaceable Components” on page 27 ■ “Oracle ILOM Legacy Target Replaceable Components” on page 28 ■ “Faulted Item Replaceable Components” on page 30 ■...
Page 29
“Replacing the Switch” on page 139 /SYS/SP/V_+1V8 Switch “Replacing the Switch” on page 139 /SYS/SP/V_+3V3 Related Information “Oracle ILOM /HOST and /SP Target Replaceable Components” on page 27 ■ “Oracle ILOM /System Target Replaceable Components” on page 27 ■ Detecting and Managing Faults...
Cable “Servicing Data Cables” on page 81 Switch “Replacing the Switch” on page 139 SP LED Switch “Replacing the Switch” on page 139 Time Battery “Servicing the Battery” on page 107 Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
“Troubleshooting (Oracle ILOM /SP configuration parameters for Oracle ILOM. Targets)” on page 42 Oracle ILOM logs – Provide an accounting of events that happened with the “Display Oracle ILOM Log Entries” on page 43 switch hardware. Oracle ILOM/System targets – Provide information about the hardware of the “Troubleshooting (Oracle ILOM /System...
“Faulted Item Replaceable Components” on page 30 for links to service tasks. Related Information “Front Panel LEDs” on page 33 ■ “SFP+ Link LEDs” on page 33 ■ “RJ-45 Link LEDs” on page 34 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Inspect the LEDs “Chassis Status LEDs” on page 35 ■ “Component LEDs” on page 36 ■ Front Panel LEDs Other than the power supply LEDs and the fan module LEDs, there are no other LEDs on the front panel of the switch. See “Component LEDs”...
■ Flashing – There is activity on the link. ■ Off – No link is established or the link is down. RJ-45 Link LED (right of connector, green) Indicates these conditions for the lower row of RJ-45 connectors: Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Description Locate LED (top, white) Indicates these conditions: ■ Fast blink – Switch is identifying itself. To turn on a Locate LED, at the Oracle ILOM CLI type: set /SYS/LOCATE value=Fast_Blink ■ Off – Switch is not identifying itself. To turn off a Locate LED, at the Oracle ILOM CLI...
Inspect the LEDs Description ■ On – Service is required. Oracle ILOM can detect a fault or failure resulting in this indication. The Oracle ILOM /System/Open_Problems target provides details about any faults that cause this indicator to light. Under some fault conditions, individual component fault LEDs are turned on in addition to the Service Required LED.
IP_address is the IP address or host name of the SP. ■ At the prompt, type the password for the user. The default password is changeme. The Oracle ILOM CLI prompt is displayed, and you can now manage the SP. Detecting and Managing Faults...
SP. Log in to the Oracle ILOM interface with the user name that manages the SP, and its password. The default user name is root, and the password is changeme.
Network Management (SER MGT Port)” on page The Oracle Fabric Manager login page is displayed. Type the username and password of the Oracle Fabric Manager administrator. The default username is root, and the password is changeme. If you performed this task as part of configuring a replacement switch, return to that procedure.
Page 40
Configure and Enable Network Management (SER MGT Port) Access the Oracle ILOM CLI from the SER MGT port. “Access the SP (SER MGT Port)” on page Configure the SP network management parameters. -> set /SP/network property=value property=value ... where: property is the parameter of the network to configure.
Page 41
SCP_IP_address is the IP address to be assigned to the SCP. Alternatively, you can configure IP address assignment by DHCP or Oracle ILOM. If you provide a static IP address, you will also need to provide the netmask and the IP address of the management network gateway.
“Display the SP Firmware Version” on page 46 ■ Oracle ILOM Log Entry Filters When you display the Oracle ILOM log, you can filter the output to specific classes and types of entries using combinations of the Class and Type parameters. See “Display Oracle ILOM Log Entries”...
Troubleshooting (Oracle ILOM /System Targets) “Oracle ILOM Log Entry Filters” on page 44 ■ “Display Network Management Configuration Properties” on page 45 ■ “Display User Session Information” on page 45 ■ Troubleshooting (Oracle ILOM /System Targets) These topics display the health state of various systems and components.
In the output of the /System/Open_Problems target, the string of text that begins with fault. chassis. identifies the faulty component. Additionally, the text that follows Resourceis the Oracle ILOM target of the component that faulted. This table provides example output for various component faults. Once you identify the component that has faulted, see “Oracle ILOM...
“Display Problematic Components” on page 47 ■ “Display the General Health of Components and Systems” on page 49 ■ “Oracle ILOM Target Health States” on page 50 ■ “Display Active Power Consumption” on page 51 ■ Display the General Health of Components and Systems Access the Oracle ILOM CLI.
“Display Active Power Consumption” on page 51 ■ Oracle ILOM Target Health States Use this table to clarify health states as seen in the health property of Oracle ILOM /System targets. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Oracle ILOM might require HMP to be installed. See the Oracle HMP documentation library at http:// docs.oracle.com/cd/E20451_01/. Oracle ILOM has detected a problem with the component that requires a service action to resolve the issue. Service Required If this health status appears at the system level, view the /System/Open_Problems target. See “Display...
“Troubleshooting (Oracle ILOM /System Targets)” on page 47 ■ Enable Oracle ILOM Legacy Targets Oracle ILOM legacy targets are not available or functional until you perform this task. Access the Oracle ILOM CLI. “Accessing the SP” on page Enable the Oracle ILOM legacy targets.
“Display Status for Specific Components” on page 53 ■ “Clear a Fault Manually” on page 54 ■ “Oracle ILOM Legacy Targets for Fault State and FRUID” on page 55 ■ “Oracle ILOM Legacy Targets for Alarm State” on page 56 ■...
■ Clear a Fault Manually Faults can be detected by Oracle ILOM or the native OS of the SP. The native OS communicates the faults it has discovered to Oracle ILOM. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Clear a Fault Manually If Oracle ILOM detects a fault and subsequent component replacement, Oracle ILOM automatically clears the fault. However, if the native OS detects a fault, you might need to manually clear the fault from Oracle ILOM after resolution.
“Display Status for Specific Components” on page 53 ■ “Clear a Fault Manually” on page 54 ■ “Oracle ILOM Legacy Targets for Alarm State” on page 56 ■ “Oracle ILOM Legacy Targets for Power Consumption” on page 58 ■ “Oracle ILOM Legacy Targets for Voltage” on page 58 ■...
Page 57
“Display Status for Specific Components” on page 53 ■ “Clear a Fault Manually” on page 54 ■ “Oracle ILOM Legacy Targets for Fault State and FRUID” on page 55 ■ “Oracle ILOM Legacy Targets for Power Consumption” on page 58 ■...
“Display Status for Specific Components” on page 53 ■ “Clear a Fault Manually” on page 54 ■ “Oracle ILOM Legacy Targets for Fault State and FRUID” on page 55 ■ “Oracle ILOM Legacy Targets for Alarm State” on page 56 ■...
“Display Status for Specific Components” on page 53 ■ “Clear a Fault Manually” on page 54 ■ “Oracle ILOM Legacy Targets for Fault State and FRUID” on page 55 ■ “Oracle ILOM Legacy Targets for Alarm State” on page 56 ■...
“Display Status for Specific Components” on page 53 ■ “Clear a Fault Manually” on page 54 ■ “Oracle ILOM Legacy Targets for Fault State and FRUID” on page 55 ■ “Oracle ILOM Legacy Targets for Alarm State” on page 56 ■...
“Display Status for Specific Components” on page 53 ■ “Clear a Fault Manually” on page 54 ■ “Oracle ILOM Legacy Targets for Fault State and FRUID” on page 55 ■ “Oracle ILOM Legacy Targets for Alarm State” on page 56 ■...
Clear a Fault Manually Resettable Targets This table lists Oracle ILOM targets that are resettable through their clear_fault_action property and the related components with manual reset capability. Use this table for the task “Clear a Fault Manually” on page Target...
Review safety and handling information. “Safety Information” on page 64 Gather the tools for service. “Service Tools” on page 65 Read the switch serial number through Oracle ILOM. “Read the Switch Serial Number (Oracle ILOM)” on page 66 Identify the switch to be serviced.
Caution - electricity. Ordinary amounts of static electricity from clothing or the work environment can destroy the components located on these boards. Do not touch the components along their connector edges. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Disposable ESD mat (shipped with some replacement parts) ■ Related Information “Service Tools” on page 65 ■ “Read the Switch Serial Number (Oracle ILOM)” on page 66 ■ “Locate the Switch” on page 67 ■ “Prevent ESD Damage” on page 67 ■...
You can use the Locate LED to pinpoint the location of a switch. This task is helpful when you need to identify one particular switch from many other rack components. Unlike other Oracle products, the Locate LED is only at the rear of the switch and does Note - not have button functionality.
Related Information “Safety Information” on page 64 ■ “Service Tools” on page 65 ■ “Read the Switch Serial Number (Oracle ILOM)” on page 66 ■ “Locate the Switch” on page 67 ■ “Remove the Air Duct Assembly” on page 68 ■...
Page 69
Remove the Air Duct Assembly Grasp the air duct assembly at the left and right edges. Pull the air duct assembly straight out of the rack. Consider your next step. If there is no SER MGT connection, set the air duct assembly aside. ■...
Page 70
Pass the SER MGT cable back through the air duct assembly until the cable is free. Set the air duct assembly aside. (Optional) Read the chassis serial number from the plastic strip. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 71
Remove the Air Duct Assembly Pull the plastic strip that is to the left of the left power supply. Read the serial number. Slide the strip back into the chassis. If you are not performing further service on the switch, install the air duct assembly.
“Remove a Snap Bracket” on page Related Information “Safety Information” on page 64 ■ “Service Tools” on page 65 ■ “Read the Switch Serial Number (Oracle ILOM)” on page 66 ■ “Locate the Switch” on page 67 ■ “Prevent ESD Damage” on page 67 ■...
Page 73
Remove a Snap Bracket Do not remove the screws, just loosen them for one rotation. Note - Preparing for Service...
Page 74
Related Information “Safety Information” on page 64 ■ “Service Tools” on page 65 ■ “Read the Switch Serial Number (Oracle ILOM)” on page 66 ■ “Locate the Switch” on page 67 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
The steps of this task are a generalization of what is to be done. For more thorough Note - instructions on how to accomplish these steps, refer to the Oracle EDR InfiniBand Switch and Virtualized I/O Systems Administration Guide, the Oracle Fabric Manager 5.0 Administration Guide, and the Oracle Fabric OS 1.0 Administration Guide.
Power Off a Power Supply This backs up the SEFOS configuration. Refer to the Oracle EDR InfiniBand Switch and Virtualized I/O Systems Administration Guide. Transfer the backup files to a safe place. Power off the switch. “Power Off a Power Supply” on page Related Information “Safety Information”...
Page 77
Power Off a Power Supply Remove the power cord from the power supply. Remove the power cord in a slow wiggling motion. Pulling or jerking the cord out from the power supply might cause your hand to catch Caution - on the snap brackets, if they are installed.
Page 78
Power Off a Power Supply Feed the power cord back through the opening in the long front mounting bracket. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 79
Power Off a Power Supply If the SER MGT cable also passes through the opening with the power cord, feed the SER MGT cable back through the opening. Consider your next step. If you powered off the power supply to remove it, do that now. ■...
Page 80
Power Off a Power Supply Related Information “Safety Information” on page 64 ■ “Service Tools” on page 65 ■ “Read the Switch Serial Number (Oracle ILOM)” on page 66 ■ “Locate the Switch” on page 67 ■ “Prevent ESD Damage” on page 67 ■...
Servicing Data Cables Data cables provide a conduit for high-speed Ethernet network and IB fabric communication. The data cables are located across the rear of the switch. PrizmMT cables are on the left, SFP+ cables are in the center, and RJ-45 cables are on the right. See “Identifying Components”...
“(Optional) Assemble the Optical Fiber SFP+ Cables” on page 96 ■ “Install a Data Cable” on page 98 ■ “Verify a Data Cable” on page 104 ■ “(Optional) Clean the PrizmMT Receptacle or Connector Ferrule” on page 104 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
SFP+ 10GBASE-T adapter (transceiver) Related Information “PrizmMT Cables” on page 83 ■ “RJ-45 Cables” on page 84 ■ RJ-45 Cables RJ-45 cables should be of Category 6, 6e, or 7 specifications. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
■ “Remove a Data Cable” on page If you cannot determine if a cable is faulty, go to Step ■ Access the Oracle ILOM CLI. “Accessing the SP” on page Verify Ethernet connectivity. Start the EMS console. Servicing Data Cables...
Page 86
The Capability indicates if auto-crossover capability is on. If the status of a known-connected port is anything other than connected, replace the cable attached to that port. “Remove a Data Cable” on page Exit the EMS console. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 87
[OFOS]. If this is the first time OFOS is accessed, the OFOS configuration wizard might start. If this happens, refer to the Oracle InfiniBand Switch IS2-46 Installation Guide, running the OFOS configuration wizard. Display the state of the gateway ports.
Page 88
“Remove a Data Cable” on page Exit the host console. Type and press the Enter key, then press the Esc key and type exit [OFOS] exit [root@nsn156-168 ~]# Serial console stopped. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Remove a Data Cable -> If you are unable to determine if a data cable is faulty, seek further information. “Detecting and Managing Faults” on page Related Information “Identifying Data Cable Types” on page 82 ■ “Remove a Data Cable” on page 89 ■...
Page 90
If you are removing an assembled fiber optic SFP+ cable, go to Step ■ If you are removing an RJ-45 cable, go to Step ■ Remove a PrizmMT cable. Pinch the PrizmMT cable connector housing between your thumb and forefinger. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 91
Remove a Data Cable The housing is either magenta or black colored. With a steady force, pull the PrizmMT connector straight out of the receptacle. Go to Step Remove a one-piece copper core SFP+ cable. Grasp the cable connector to support its weight, and apply the removal force.
Page 92
Remove a Data Cable The cable connector comes free. Go to Step Remove an assembled fiber optic SFP+ cable. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 93
Remove a Data Cable Press the release tab on the LC connector and pull back. The LC connector and fiber optic cable come free of the transceiver. Servicing Data Cables...
Page 94
The transceiver comes free. Set the transceiver aside. Go to Step Remove an RJ-45 cable. Pinch the top and bottom of the RJ-45 connector between your thumb and forefinger. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 95
Remove a Data Cable Use either your thumb or forefinger to depress the release tab. Pull the RJ-45 connector straight out of the receptacle. Carefully move the cable out of the CMB. Cap the end of the cable if possible. Repeat Step 4 Step 12...
The ends of the optical fiber cable and the receptacles of the transceivers must be clean Note - and optically clear before assembly. Do not remove the protective caps from the optical fiber cable or the plugs from the transceivers until instructed to do so. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 97
(Optional) Assemble the Optical Fiber SFP+ Cables Identify the prerequisite and subsequent installation tasks that you must perform in conjunction with this task. “Servicing Data Cables” on page Remove the SFP+ optical transceivers and fiber optic cables from their packaging. Remove the plug from an SFP+ optical transceiver.
If you are adding an additional data cable, see “Preparing for ■ Service” on page 63, and then go to Step If you are installing a data cable as part of a switch installation task, go to ■ Step Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 99
“(Optional) Assemble the Optical Fiber SFP+ Cables” on page Connect the replacement cable to the remote location, and route it to the switch. See the Oracle InfiniBand Switch IS2-46 Installation Guide, routing data cables. Bring the cable to the switch and remove its protective cap, if installed.
Page 100
Firmly press the RJ-45 connector into the receptacle until you hear a click. Go to Step Install an SFP+ cable. Align the SFP+ connector to where it will install into the receptacle. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 101
Install a Data Cable The product label on the connector is up for the upper row of receptacles and down for the lower row of receptacles. Slowly move the connector in and continue to push until you feel a detent. Go to Step Install a PrizmMT cable.
Page 102
If you installed a data cable as part of a switch configuration, return to that task. “Configure the Replacement Switch” on page 161. Repeat Step 3 through Step 10 for any additional cables to be installed. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 103
Install a Data Cable Replace the cover for the CMB and tighten the thumbscrews. Consider your next steps. If you installed a data cable as part of a replacement operation, go to Step ■ ■ If you installed all data cables as part of the switch installation task, return to switch configuration.
If you are cleaning a connector, go to Step ■ Remove the adapter from the end of the PrizmMT cleaner. Insert the cleaner tip into the receptacle. Pump the cleaner into the receptacle, advancing the cleaning cloth inside. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 105
(Optional) Clean the PrizmMT Receptacle or Connector Ferrule Remove the cleaner from the receptacle and verify cleanliness. Repeat from Step 3 if the fibers are still dirty. Replace the adapter onto the end of the PrizmMT cleaner. Return to “Install a Data Cable” on page Open the lid of the adapter on the end of the PrizmMT cleaner.
Page 106
Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Servicing the Battery The battery maintains a constant voltage to the real time clock, which provides the time and date of the switch. The battery is located inside the SMM, on the left side of the front panel, adjacent to PS 0. See “Identifying Components”...
Page 108
MB, Part Number:7092577, Serial Number:AK00329231, Reference Document:http://support.oracle.com/msg/---) -> ■ If you see the word battery and Oracle ILOM target /SYS/MB in the output, then replace the battery. Start at “Remove the Air Duct Assembly” on page If you see a different component identified in the output, then service that ■...
Remove the Battery Remove the Battery Removing the battery is a hot-swap operation. You do not need to power off the switch before you remove the battery. The battery is located inside the SMM. Description Battery Consider your first step. If you have not prepared for service, you must do so now.
Page 110
Remove the Battery Fully loosen the captive thumbscrew at the right side of the SMM. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 111
Remove the Battery Grasp the thumbscrew and pull the SMM straight out. Set the SMM down on an antistatic mat. Servicing the Battery...
Page 112
“Install the Battery” on page 113. Related Information “Determine if the Battery Is Faulty” on page 107 ■ “Install the Battery” on page 113 ■ “Verify the Battery” on page 117 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Install the Battery Install the Battery Installing the battery is a hot-swap operation. You do not need to power off the switch before installing the battery. The slot located adjacent to the left power supply, PS 0, is dedicated to the SMM. Caution - Inserting a fan module in this slot, or inserting the SMM into a fan module slot, will damage both the module and the switch.
Page 114
Install the Battery With the SER MGT port facing away from you, the plus side of the battery is angled down toward the left. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 115
Install the Battery Insert the edge of the battery into the holder, and then pivot it upright to the right. The battery clicks into place. Servicing the Battery...
Page 116
Install the Battery Align the SMM to the opening in the switch chassis with the thumbscrew on the right. Firmly slide the SMM into the chassis until the SMM stops. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Verify the Battery Tighten the captive thumbscrew to secure the SMM in the switch chassis. Verify the battery. “Verify the Battery” on page 117. Related Information “Determine if the Battery Is Faulty” on page 107 ■ “Remove the Battery” on page 109 ■...
Page 118
“Install the Air Duct Assembly” on page 166. Related Information “Determine if the Battery Is Faulty” on page 107 ■ “Remove the Battery” on page 109 ■ “Install the Battery” on page 113 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Servicing Fan Modules Fan modules contain two redundant fan units that are variable speed. The four fan modules are located across the front of the switch chassis. See “Identifying Components” on page Description Links Replace a faulty fan module. “Locate a Faulty Fan Module” on page 120 “Fan Module LED”...
Locate a Faulty Fan Module You must determine which fan module is faulty before you replace it. Consider your first step. If you are at the switch, go to Step ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 121
Number:N/A, Serial Number:N/A, Reference Document:http://support.oracle.com/msg/---) -> where x is 0 to 3. ■ If you see the word fan and Oracle ILOM target /SYS/FMx in the output, then replace that fan module. Start at “Remove the Air Duct Assembly” on page If you see a different component identified in the output, then service that ■...
Page 122
/System Target Replaceable Components” on page 27 to determine which fan module to replace. Then, start at “Remove the Air Duct Assembly” on page Within the Oracle ILOM interface, display the fault state of the fan modules. -> show /SYS -t fault_state Target | Property...
Remove a Fan Module Related Information “Fan Module LED” on page 120 ■ “Remove a Fan Module” on page 123 ■ “Install a Fan Module” on page 125 ■ “Verify a Fan Module” on page 127 ■ Remove a Fan Module Removing a fan module is a hot-swap operation.
Page 124
Loosen the captive thumbscrew at the right side of the fan module. Grasp the handle and pull the fan module straight out. Set the fan module aside. Install a new fan module. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Install a Fan Module “Install a Fan Module” on page 125. Related Information “Fan Module LED” on page 120 ■ “Locate a Faulty Fan Module” on page 120 ■ “Install a Fan Module” on page 125 ■ “Verify a Fan Module” on page 127 ■...
Page 126
“Fan Module LED” on page 120 ■ “Locate a Faulty Fan Module” on page 120 ■ “Remove a Fan Module” on page 123 ■ “Verify a Fan Module” on page 127 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Verify a Fan Module After you install a fan module, you can verify its functionality. From the Oracle ILOM CLI, reset the fan module. -> set /SYS/FMx clear_fault_action=true where x is 0 to 3. For example, to reset fan module FM1, type: ->...
Page 128
Verify a Fan Module “Remove a Fan Module” on page 123 ■ “Install a Fan Module” on page 125 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Servicing Power Supplies Power supplies provide the 12 VDC necessary to power the switch. The power supplies are located at the left and right sides of the front panel. See “Identifying Components” on page Description Links Replace a faulty power supply. “Locate a Faulty Power Supply”...
■ On – Input power present and good. ■ Off – Input power not present. Related Information “Locate a Faulty Power Supply” on page 131 ■ “Remove a Power Supply” on page 133 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
“Remove the Air Duct Assembly” on page If you cannot determine if a power supply is faulty, go to Step ■ Access the Oracle ILOM CLI. “Accessing the SP” on page Determine if a power supply has become problematic. -> show /System/Open_Problems...
Page 132
Locate a Faulty Power Supply ■ If you see psu and Oracle ILOM target /SYS/PSx in the output, then replace that power supply. Start at “Remove the Air Duct Assembly” on page If you see a different component identified in the output, then service that ■...
Remove a Power Supply Related Information “Power Supply LEDs” on page 130 ■ “Remove a Power Supply” on page 133 ■ “Install a Power Supply” on page 135 ■ “Verify a Power Supply” on page 137 ■ Remove a Power Supply Removing a power supply is a hot-swap operation.
Page 134
Remove a Power Supply Press and hold the release tab to the left, and pull on the handle of the power supply. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Install a Power Supply Continue to pull the handle of the power supply to remove it from the chassis. Set the power supply aside. Install a new power supply. “Install a Power Supply” on page 135. Related Information “Power Supply LEDs” on page 130 ■...
Page 136
Align the power supply to the opening in the switch chassis with the status LEDs on the left and the release tab on the right. Slide the power supply into the open slot, pushing at the handle. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
“Remove a Power Supply” on page 133 ■ “Verify a Power Supply” on page 137 ■ Verify a Power Supply After you install a power supply, you can verify its functionality. From the Oracle ILOM CLI, reset the power supply. Servicing Power Supplies...
Page 138
“Power Supply LEDs” on page 130 ■ “Locate a Faulty Power Supply” on page 131 ■ “Remove a Power Supply” on page 133 ■ “Install a Power Supply” on page 135 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Replacing the Switch These topics describe how to replace the entire switch. Description Links Replace the faulty switch. “Determine if the Switch Is Faulty” on page 141 “Switch LEDs” on page 140 “Remove the Air Duct Assembly” on page 68 “Remove a Snap Bracket”...
Green Indicates these conditions: ■ On – Switch is functional without fault. ■ Off – Switch is off or initializing. Bottom Green Indicates these conditions: ■ On – SP is ready. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
“Remove the Air Duct Assembly” on page If you cannot determine if the switch is faulty, go to Step ■ Access the Oracle ILOM CLI. “Accessing the SP” on page Determine if the switch has become problematic. -> show /System/Open_Problems...
Page 142
Number:7096151, Serial Number:489089M+15306P00M0, Reference Document:http://support.oracle.com/msg/---) -> ■ If you see the words temp and fail, or voltage and fail, and either Oracle ILOM targets /SYS/MB or /SYS/SP in the output, then replace the switch. Start at “Remove the Air Duct Assembly” on page If you see a different component identified in the output, then service that ■...
Remove the Switch Related Information “Switch LEDs” on page 140 ■ “Remove the Switch” on page 143 ■ “Install the Switch” on page 145 ■ “Verify the Switch” on page 148 ■ Remove the Switch Removing the switch is a cold-service operation. You must power off the switch before you remove it.
Page 144
While supporting the weight of the switch, use a No. 2 Phillips screwdriver to remove the four screws from the long brackets. Gently slide the switch out of the rack. Ensure that the switch does not bind on the long rails as it slides. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Install the Switch Set the switch down onto a stable work surface. Use a No. 1 Phillips screwdriver to remove the eight screws that secure the C- shaped brackets at the rear sides of the switch chassis. Remove the eight screws that secure the long front mounting brackets at the front sides of the switch chassis.
Page 146
Use the eight screws to attach the two C-shaped brackets to the rear sides of the chassis. Carefully lift the switch and slide it into the rack, from the front rearward. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 147
Install the Switch Ensure that the ends of the long rails slide into the C-shaped brackets at the rear of the switch chassis. Use a No. 2 Phillips screwdriver to mount the front of the switch chassis to the front rack posts with two screws at each side. Replacing the Switch...
Log in to SEFOS. SEFOS login: root Password: password SEFOS# The default password is admin123. Display the state of the Ethernet ports. SEFOS# show interfaces status Port Status Duplex Speed Negotiation Capability Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 149
Verify the Switch ---- ------ ------ ----- ----------- ---------- Gi0/1 not connected Half Auto Auto-MDIX on Gi0/2 connected Full 1 Gbps Auto Auto-MDIX on Gi0/3 connected Full 1 Gbps Auto Auto-MDIX on Gi0/4 not connected Half Auto Auto-MDIX on Gi0/5 not connected Half Auto...
Page 150
[OFOS]. If this is the first time OFOS is accessed, the OFOS configuration wizard might start. If this happens, refer to Oracle InfiniBand Switch IS2-46 Installation Guide, running the OFOS configuration wizard. Display the state of the gateway ports.
Page 151
Verify the Switch up/down switching up/down switching up/down switching up/down switching 0/27 up/down switching 0/28 up/down switching 0/29 up/down switching 0/30 up/down switching 30 records displayed [OFOS] In the output, the name is the port number as seen on the rear panel. The state is of the local port/peer port.
Page 152
Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Returning the Switch to Operation These topics explain how to return the switch to operation after you perform service tasks. Step Description Link Retrieve the power cord and if necessary, the SER MGT “Retrieve the Power Cords and SER MGT cable.
Open the larger loop approximately 45 degrees to form a hook. Holding the smaller loop, insert the larger loop into the opening of the long front mounting bracket, and hook the power cord hanging inside. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 155
Retrieve the Power Cords and SER MGT Cable Pull the power cord through the opening, until you can grasp it with your fingers. Pull the cord through, so that the end of the cord is free. Repeat Step 4 through Step 6 for the other power cord if necessary.
Page 156
“Connect the SER MGT Cable” on page 158 ■ “Configure the Replacement Switch” on page 161 ■ “Install a Snap Bracket” on page 164 ■ “Install the Air Duct Assembly” on page 166 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Power On a Power Supply Power On a Power Supply Powering on either power supply powers on the switch. Note - Reconnect the power cord to the power supply. The AC LED lights green to indicate that the power supply is connected to facility power. A moment later, the OK LED lights green to indicate the power supply is at full power.
If the SER MGT cable is on the left, pass the cable from the outside, through ■ the opening on the left side (no Oracle logo) in to the center. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 159
If the SER MGT cable is on the right, pass the cable from the outside, ■ through the opening on the right side (Oracle logo) in to the center. Continue to pull the cable through the air duct assembly to take up the slack.
Page 160
“Power On a Power Supply” on page 157 ■ “Configure the Replacement Switch” on page 161 ■ “Install a Snap Bracket” on page 164 ■ “Install the Air Duct Assembly” on page 166 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
The steps of this task are a generalization of what is to be done. For more thorough Note - instructions on how to accomplish these steps, refer to the Oracle EDR InfiniBand Switch and Virtualized I/O Systems Administration Guide, the Oracle Fabric Manager 5.0 Administration Guide, and the Oracle Fabric OS 1.0 Administration Guide.
Page 162
Access the SP. “Access the SP (RJ-45 Port)” on page 37 Restore the Oracle ILOM configuration. The SEFOS configuration will be restored as well. Refer to the Oracle EDR InfiniBand Switch and Virtualized I/O Systems Administration Guide. Access the SCP.
Page 163
Configure the Replacement Switch Wait for the security policies to propagate to the replacement switch. Configure the Subnet Manager nodes list and fabric configuration file for the replacement switch, and ensure that the list and file are consistent with other switch lists and files. (Optional) Set the priority of the replacement switch to that originally of the removed switch.
Install a Snap Bracket Install a Snap Bracket Slide the snap bracket under the loosened screws with the tabs pointing into the rack. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 165
Install a Snap Bracket While pressing on the word PUSH embossed in the snap bracket, fully tighten the screws with the No. 2 Phillips screwdriver. Consider your next steps. If you replaced a power supply, and have a full-time SER MGT connection, ■...
Tuck the power cord slack into the opening of the long front mounting bracket. Tuck the SER MGT cable slack into the cavity of the air duct assembly. Lift the air duct assembly to where it installs on the rack. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 167
Install the Air Duct Assembly Though the air duct assembly is symmetrical, if there is any logo present, it goes to the right. Push the air duct assembly in to the front of the switch, until it snaps into place. If installed, close the rack front door.
Page 168
“Power On a Power Supply” on page 157 ■ “Connect the SER MGT Cable” on page 158 ■ “Configure the Replacement Switch” on page 161 ■ “Install a Snap Bracket” on page 164 ■ Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Ethernet technology that supports cable distances of over 300 meters. 10GBASE-T – Standard 10GbE Ethernet technology. 10GbE/40GbE I/O module for the Oracle Fabric Interconnect F2-12 that supports 10GbE and 40GbE module interfaces. 16G Fibre I/O module for the Oracle Fabric Interconnect F2-12 that supports Fibre Channel interfaces.
Page 170
A data transference technology used primarily for storage systems. Abbreviation of fan module. See also fan module. Abbreviation of GigaByte. 1 GB is approximately 1000 MB. Abbreviation of Gigabit Ethernet. A unit of throughput for Ethernet technology. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 171
Is used in conjunction with the IB SLM modules. See also fabric card and IB SLM module. IB SLM I/O module for the Oracle InfiniBand Switch IS2-254 that provides a leaf topology construct. Is module used in conjunction with the IB SFM modules. See also line card and IB SFM module.
Page 172
A component of an IB switch chassis, that provides a leaf topology construct. See also IB SLM module. long-range IB I/O module for the Oracle Fabric Interconnect F2-12 that supports QDR IB throughput over module distances up to 40 kilometers.
Page 173
A web-based GUI to administer the switch and fabric. Manager Oracle Fabric Oracle Fabric Operating System. The operating system within the SCP, that manages the fabric. See also SCP. Oracle ILOM Oracle Integrated Lights-Out Manager. An interface using the IPMI protocol and the I C bus to monitor and control the hardware components of a chassis.
Page 174
See also MIB and OID. Service processor. A device that monitors and controls a chassis, regardless of the chassis host state, through an Oracle ILOM interface. See also Oracle ILOM and SCP. Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Page 175
WWPN spine switch A switch utilizing a spine topology. A transceiver used for short range Fibre Channel communication. transceiver Secure Shell. A secure network communication protocol, utilizing DSA and RSA algorithms. See also DSA and RSA. standard MT An optical fiber connector and cable used for IB 4x technology. Appears similar to, but is not cable compatible with PrizmMT cables and connectors.
Page 176
Oracle InfiniBand Switch IS2-46 Service Manual • September 2016...
Need help?
Do you have a question about the InfiniBand Switch IS2-46 and is the answer not in the manual?
Questions and answers