Summary of Contents for Oracle Sun Network QDR InfiniBand Gateway Switch
Page 1
Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 Part No.: E36262-01 March 2013, Revision A...
Page 2
Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.
Clear a Fault Manually 10 Clearable Fault Targets 11 ▼ Identify Faults in the Oracle ILOM Event Log 12 Determining the Alarm State of a Component or System 13 ▼ Display the General Alarm State of Systems and Components 14...
Page 4
Indicator State Conditions 33 Accessing CLI Prompts 34 ▼ Access the Oracle ILOM CLI (NET MGT Port) 35 ▼ Enter the Restricted Linux Shell 35 Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 5
▼ Exit the Restricted Linux Shell 36 Understanding Service Procedures 37 Replaceable Components 37 Suggested Tools for Service 39 Antistatic Precautions for Service 39 Servicing Power Supplies 41 ▼ Determine If a Power Supply Is Faulty 41 Inspecting a Power Supply 43 ▼...
Page 6
Servicing the Battery 75 ▼ Determine If the Battery Is Faulty 75 ▼ Remove the Gateway From the Rack 77 ▼ Replace the Battery 78 Index 85 Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Using This Documentation This service manual provides detailed procedures that describe the service of the Sun Network QDR InfiniBand Gateway Switch from Oracle. This document is written for technicians, system administrators, and users who have advanced experience servicing InfiniBand fabric hardware.
Page 8
Access to Oracle Support Oracle customers have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id= visit info http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs if you are hearing impaired. viii Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Investigate whether there is a fault condition. “Interpreting Status LEDs” on page 1 “Managing Faulty Components” on page 7 “Identify Faults in the Oracle ILOM Event Log” on page 12 Investigate whether there is an alarm condition. “Determining the Alarm State of a Component or System”...
“Check Power Supply Status LEDs” on page 6 Power supply OK LED “Check Power Supply Status LEDs” on page 6 Fan Attention LED “Check Fan Status LEDs” on page 7 Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Related Information “Rear Panel LEDs” on page 3 ■ “Check Chassis Status LEDs” on page 4 ■ “Check NET MGT Port Status LEDs” on page 4 ■ “Check Link Status LEDs” on page 5 ■ “Check Power Supply Status LEDs” on page 6 ■...
The NET MGT port status LEDs are located on the NET MGT connector of the rear panel. See “Rear Panel LEDs” on page 1. Visually inspect the NET status LEDs. 2. Compare what you see to this table. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Name Location Color State and Meaning Link speed Left Amber or green Amber on – 100BASE-T link. Green on – 1000BASE-T link. Off – No link or link down. Flashing – No function. Activity Right Green On – No function. Off –...
3. If the Attention LED is lit, there is a fault with that power supply. “Servicing Power Supplies” on page Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“Check Power Supply Status LEDs” on page 6 ■ Managing Faulty Components If Oracle ILOM has detected a fault with a component, you can display and clear that fault with these topics: “Display Faulty Components (fault_state)” on page 8 ■...
| OK -> 3. Look in the Value column for Faulted. 4. Look in the same row under the Target column, to find the Oracle ILOM target of the faulty component. For example, /SYS/FAN2. 5. Identify the component that has faulted and might need to be replaced.
▼ Display Faulty Components (/SP/faultmgmt) 1. Access the Oracle ILOM CLI. “Access the Oracle ILOM CLI (NET MGT Port)” on page 2. Display any faulty components. -> show -d targets /SP/faultmgmt /SP/faultmgmt Targets: x (faulted_target) -> where: x is the target sequence number (starting at 0).
ILOM automatically clears the fault. However, you can manually clear the fault after replacing the component, if necessary. 1. Access the Oracle ILOM CLI. “Access the Oracle ILOM CLI (NET MGT Port)” on page Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
■ “Display Faulty Components (/SP/faultmgmt)” on page 9 ■ “Clearable Fault Targets” on page 11 ■ Clearable Fault Targets This table lists the components, their Oracle ILOM targets that are clearable, and links to servicing procedures. Component Target Links Battery “Servicing the Battery”...
Tue Sep 18 15:51:48 2012 Fault Fault critical Fault detected at time = Tue Sep 18 15:51:48 2012. The suspect component: /SYS/PSU0 has fault.chassis.device.psu.fail with probability=100. Refer Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“Clearable Fault Targets” on page 11. Following the Oracle ILOM target is the reason for the fault. A URL is provided for more information about the fault. Moving up the output, Event ID 18569 on September 18, at 16:43, indicated that a repair action was taken on the component with Oracle ILOM target /SYS/PSU0.
4. If the alarm state is major or critical, you might need to replace the component. “Clearable Fault Targets” on page 11 for servicing links. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“Oracle ILOM Target Alarm States” on page 16 ■ System Alarm Targets This table lists systems that have the ability to report an alarm and their Oracle ILOM targets.Use these targets for the procedure, “Display the General Alarm State of Systems and Components”...
Use this table to clarify alarm states as seen in the alarm_status = alarm_state parameter of Oracle ILOM targets and in the output of the procedure “Display the General Alarm State of Systems and Components” on page Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
The operation of the gateway is compromised or at risk. Oracle ILOM is unable to provide an alarm state for this component. indeterminate The component or its alarm is not available to Oracle ILOM. (The component might (none) have been removed.) Related Information “Display the General Alarm State of Systems and Components”...
| alarm_status | cleared /SYS/MB/V_3.3VMain | alarm_status | cleared /SYS/MB/ | alarm_status | cleared V_3.3VMainOK /SYS/MB/V_3.3VStby | alarm_status | minor /SYS/FAN3/PRSNT | alarm_status | cleared Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 27
For example, minor. For more information about alarm states, see “Oracle ILOM Target Alarm States” on page 4. Look in the same row under the Target column, to find the Oracle ILOM sensor target. For example, /SYS/MB/V_3.3VStby. 5. Display the value of the sensor target.
Evaluating a Voltage Sensor Alarm These topics help you resolve voltage sensor alarms. “Evaluate a Voltage Sensor” on page 21 ■ “Voltage Sensor Values” on page 22 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“Voltage Out of Range” on page 22 ■ Related Information “Display Oracle ILOM Sensor Status” on page 18 ■ “Determine Oracle ILOM Sensor Target Types” on page 20 ■ “Evaluating a Temperature Sensor Alarm” on page 23 ■ “Evaluating a Speed Sensor Alarm” on page 26 ■...
The load for which the voltage is provided, is missing – A component has failed or ■ has been removed from the electrical connection. The regulator for that voltage has failed. ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Because of this configuration, you must recheck the 3.3VMain, 3.3VStby, and 12V with only one power supply operational at a time. Re-perform “Display Oracle ILOM Sensor Status” on page 18 with only the power cord for PSU0 disconnected, and then again with only the power cord for PSU1 disconnected.
“Temperature Out of Range” on page Temperature Sensor Target Typical Value Acceptable Range 30˚C 25 to 70˚C /SYS/MB/T_BACK 29˚C 25 to 70˚C /SYS/MB/T_FRONT Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Temperature Sensor Target Typical Value Acceptable Range 45˚C 25 to 60˚C /SYS/MB/T_SP 39˚C 25 to 70˚C /SYS/MB/T_I4A 48˚C 25 to 70˚C /SYS/MB/T_B0 49˚C 25 to 70˚C /SYS/MB/T_B1 Related Information “Evaluate a Temperature Sensor” on page 24 ■ “Temperature Out of Range” on page 25 ■...
■ “Evaluating an Indicator State” on page 32 ■ ▼ Evaluate a Speed Sensor 1. Display the sensor status and determine the target type. See: Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“Display Oracle ILOM Sensor Status” on page 18 ■ “Determine Oracle ILOM Sensor Target Types” on page 20 ■ 2. Compare the displayed value with a known good range. “Speed Sensor Values” on page 3. Learn why a speed sensor might alarm and take action.
Page 36
55. If new fans do not resolve the problem, then replace the gateway. Related Information “Evaluate a Speed Sensor” on page 26 ■ “Speed Sensor Values” on page 27 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“State Sensor Alarm Conditions” on page 30 ■ Related Information “Display Oracle ILOM Sensor Status” on page 18 ■ “Determine Oracle ILOM Sensor Target Types” on page 20 ■ “Evaluating a Voltage Sensor Alarm” on page 20 ■ “Evaluating a Temperature Sensor Alarm” on page 23 ■...
Evaluating a Presence Sensor Alarm These topics help you resolve presence sensor alarms. “Evaluate a Presence Sensor” on page 31 ■ “Presence Sensor Alarm Conditions” on page 31 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
The sensors do not provide status or health of a component. During the boot process, the management controller looks for presence sensors to build a list of Oracle ILOM targets. If the presence sensor cannot be read, yet the component is physically installed, the management controller does not propagate the component to the list of targets.
2. Compare the displayed value with a known good range. “Indicator State Values” on page 3. Learn why an indicator might change state and take action. “Indicator State Conditions” on page 33 Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
LEDs. You use this table in conjunction with the value you recorded in “Display Oracle ILOM Sensor Status” on page 18. If your indicator target’s value is outside of the acceptable range, refer to “Indicator State Conditions”...
“Identify Faults in the Oracle ILOM Event Log” on page 12 ■ “Determining the Alarm State of a Component or System” on page 13 ■ “Evaluating Sensor Alarms” on page 17 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“Enter the Restricted Linux Shell” on page 35 ■ “Exit the Restricted Linux Shell” on page 36 ■ ▼ Enter the Restricted Linux Shell 1. Access the Oracle ILOM CLI. “Access the Oracle ILOM CLI (NET MGT Port)” on page Detecting and Managing Faults...
-> Related Information “Access the Oracle ILOM CLI (NET MGT Port)” on page 35 ■ “Enter the Restricted Linux Shell” on page 35 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Understanding Service Procedures Servicing the gateway means a component addition, replacement, or subtraction. A component addition means installing a component to increase the functionality of the gateway. Component replacement means removing a failed component and installing a functional one. Component subtraction means removing a component. Once a failed part is identified, it can be replaced.
Page 46
“Servicing Data Cables” on page 65 ■ “Servicing the Battery” on page 75 ■ “Suggested Tools for Service” on page 39 ■ “Antistatic Precautions for Service” on page 39 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Suggested Tools for Service These tools are necessary or beneficial for servicing the gateway: Antistatic wrist strap ■ Antistatic mat ■ No. 2 Phillips screwdriver ■ No. 1 Phillips screwdriver ■ Flashlight ■ Gloves ■ Magnifying glass ■ Related Information “Replaceable Components”...
Page 48
Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Servicing Power Supplies These topics provide procedures for servicing the power supplies. Description Links Add a power supply. “Inspecting a Power Supply” on page 43 “Install a Power Supply” on page 49 “Power On a Power Supply” on page 51 Replace a power supply.
Page 50
“Detecting and Managing Faults” on page Related Information “Determine If a Fan Is Faulty” on page 55 ■ “Determine If the Battery Is Faulty” on page 75 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Inspecting a Power Supply Before installing a power supply, perform these tasks to verify its suitability for installation. Step Description Links Identify the Power Supply. “Identify the Power Supply” on page 43 Inspect the hardware. “Inspect the Power Supply Hardware” on page 45 Inspect the connectors.
Page 52
3. Inspect the power supply hardware. “Inspect the Power Supply Hardware” on page Related Information “Identify the Fan” on page 57 ■ “Identify the Data Cable” on page 66 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
▼ Inspect the Power Supply Hardware 1. Identify the prerequisite and subsequent service tasks you must perform in conjunction with this procedure. “Inspecting a Power Supply” on page 2. Unwrap the replacement power supply from its antistatic packaging. 3. Verify that there is no visible damage to the power supply chassis. 4.
2. Determine which power supply is to be removed. 3. At the front of the gateway chassis, remove the power cord from the respective power supply. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
The power supply is completely powered off. 4. Remove the power supply. “Remove a Power Supply” on page Related Information “Power On a Power Supply” on page 51 ■ ▼ Remove a Power Supply 1. Identify the prerequisite and subsequent service tasks you must perform in conjunction with this procedure.
Page 56
3. Press and hold the release tab to the left and pull on the handle of the power supply. 4. Continue to pull the handle of the power supply to remove it from the chassis. 5. Set the power supply aside. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
6. Install a replacement power supply. “Install a Power Supply” on page Related Information “Remove a Fan” on page 60 ■ “Remove a Data Cable” on page 68 ■ “Remove the Gateway From the Rack” on page 77 ■ “Replace the Battery” on page 78 ■...
Page 58
8. When the power supply seats, push firmly so that the release tab clicks to secure the power supply into the chassis. 9. Power on the power supply. “Power On a Power Supply” on page Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Related Information “Install a Fan” on page 61 ■ “Install a Data Cable” on page 72 ■ “Replace the Battery” on page 78 ■ ▼ Power On a Power Supply 1. For residual power discharge, the power cord must remain unattached to the power supply for at least one minute before powering on a power supply.
Page 60
For example, to check the power supplies: FabMan@gateway_name->checkpower PSU 0 present status: OK PSU 1 present status: OK All PSUs OK FabMan@gateway_name-> Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
FabMan@gateway_name->checkvoltages Voltage ECB OK Measured 3.3V Main = 3.30 V Measured 3.3V Standby = 3.42 V Measured 12V = 12.06 V Measured 5V = 5.03 V Measured VBAT = 3.17 V Measured 1.0V = 1.01 V Measured I4 1.2V = 1.22 V Measured 2.5V = 2.51 V Measured V1P2 DIG = 1.18 V Measured V1P2 ANG = 1.18 V...
Page 62
Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Servicing Fans These topics provide procedures for servicing the fans. Description Links Add a fan. “Inspecting a Fan” on page 57 “Install a Fan” on page 61 Replace a fan. “Determine If a Fan Is Faulty” on page 55 “Remove a Fan” on page 60 “Inspecting a Fan”...
Page 64
6. Compare the value seen with the typical value and range provided in Sensor Values” on page If the fan is faulty, replace it. See “Remove a Fan” on page Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
7. If you are unable to determine if a fan is faulty, seek further information. “Detecting and Managing Faults” on page Related Information “Determine If a Power Supply Is Faulty” on page 41 ■ “Determine If the Battery Is Faulty” on page 75 ■...
4. Verify that the thumbscrew spins freely and smoothly. 5. Inspect the fan connector. “Inspect the Fan Connector” on page Related Information “Inspect the Power Supply Hardware” on page 45 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“Inspect the Data Cable Hardware” on page 67 ■ ▼ Inspect the Fan Connector 1. Identify the prerequisite and subsequent service tasks you must perform in conjunction with this procedure. “Inspecting a Fan” on page 2. Verify that the connector is clean and without damage. 3.
If a fan has failed, its Attention LED lights. 3. Loosen the captive thumbscrew at the right side of the fan. 4. Grasp the handle and pull the fan straight out. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
5. Set the fan aside. 6. Consider your next steps: If you are removing the fan for replacement, install a new fan. ■ “Install a Fan” on page If you are removing the fan as a subtractive action, you are finished. ■...
Page 70
6. Firmly slide the fan into the chassis until the fan stops. The fan might immediately power on. 7. Tighten the captive thumbscrew to secure the fan in the gateway chassis. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 71
8. Verify that the fan Attention LED goes out. 9. Access the Oracle ILOM CLI. “Access the Oracle ILOM CLI (NET MGT Port)” on page 10. Enter the restricted Linux shell. “Enter the Restricted Linux Shell” on page 11. Use the getfanspeed command on the management controller to verify the fan’s operation.
Page 72
Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Servicing Data Cables These topics provide procedures for servicing the data cables. Description Links Add a data cable. “Inspecting the Data Cables” on page 65 “Install a Data Cable” on page 72 Replace a data cable. “Remove a Data Cable” on page 68 “Inspecting the Data Cables”...
“Inspecting the Data Cables” on page 2. Use this illustration to identify the various features of the data cable. Retraction strap L groove Paddle board Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
3. Inspect the data cable hardware. “Inspect the Data Cable Hardware” on page Related Information “Identify the Power Supply” on page 43 ■ “Identify the Fan” on page 57 ■ ▼ Inspect the Data Cable Hardware 1. Identify the prerequisite and subsequent service tasks you must perform in conjunction with this procedure.
1. Identify the prerequisite and subsequent service tasks you must perform in conjunction with this procedure. “Servicing Data Cables” on page 2. Loosen the thumbscrews and remove the cover for the cable management bracket. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 77
3. Locate the cable to be removed. 4. Consider your next steps: If the cable is a one-piece data cable, follow these steps: ■ a. Grasp the cable connector to support its weight and apply the removal force. b. Pull on the retractor strap while simultaneously pulling on the cable connector.
Page 78
Step If the cable is an assembled data cable, follow these steps: ■ a. Grasp the release collar on the MTP connector and pull back. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 79
The MTP connector and fiber optic cable come free of the transceiver. b. Carefully move the fiber optic cable out of the cable management hardware. c. Release the latch on the QSFP transceiver and pull on the latch to remove the transceiver.
Ensure that the L groove is up for the top row of receptacles, or that the L groove is down for the bottom row of receptacles. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 81
Note – On some QSFP cable connectors, there is a retraction strap. Both the retraction strap and L groove indicate the reference surface for the connector. When installing QSFP cables in the top row receptacles (0A, 1A, 2A, and so on), ensure that the L groove and retraction strap are up.
Page 82
Related Information “Install a Power Supply” on page 49 ■ “Install a Fan” on page 61 ■ “Replace the Battery” on page 78 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Servicing the Battery The gateway has a battery on the main board that supports the management controller. You can only replace the battery because the management controller is dependent upon the battery. You cannot add or subtract the battery. Perform these tasks in order to replace the battery: Step Description...
Page 84
= fault.chassis.device.battery.low sunw-msg-id = DCSIB-8000-45 uuid = 82e90599-8650-47dc-b613-1e602607441b timestamp = 2002-01-01/00:07:27 fru_part_number = 3002234 fru_serial_number = 006541 product_serial_number = AK00022680 chassis_serial_number = AK00022680 -> Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
“Clearable Fault Targets” on page 11 identify which component is faulty. If no Oracle ILOM targets are listed in Step a, go to Step 4. Within the Oracle ILOM interface, verify the battery voltage. -> show /SYS/MB/V_BAT value /SYS/MB/V_BAT Properties: value = 3.136 Volts ->...
2. Use a No. 1 Phillips screwdriver to remove the eight screws that secure the C-shaped brackets at the rear sides of the gateway chassis. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 87
3. Remove the eight screws that secure the long front brackets at the front sides of the gateway chassis. 4. Remove the 16 screws that secure the top cover to the chassis. There are five screws on each side and six screws across the top front of the cover. Servicing the Battery...
Page 88
5. Slide the cover forward and lift it off. 6. Depress the clip that retains the battery and release the battery from the main board. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 89
7. Properly dispose of the old battery. 8. Unwrap the replacement battery from its antistatic packaging. 9. Install the replacement battery into the main board with the + side up. Servicing the Battery...
Page 90
11. Slide the cover rearward so that it engages at the rear panel. Ensure that the screw holes in the cover align with the holes in the chassis. Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Page 91
12. Use a No. 1 Phillips screwdriver to install the 16 screws that secure the cover to the chassis. 13. Use eight screws to attach the two front brackets to the front sides of the chassis. Servicing the Battery...
Page 92
Related Information “Install a Power Supply” on page 49 ■ “Install a Fan” on page 61 ■ “Install a Data Cable” on page 72 ■ Sun Network QDR InfiniBand Gateway Switch Service Manual for Firmware Version 2.1 • March 2013...
Need help?
Do you have a question about the Sun Network QDR InfiniBand Gateway Switch and is the answer not in the manual?
Questions and answers