Summary of Contents for Sun Microsystems SPARC Enterprise T5120
Page 1
External I/O Expansion Unit for SPARC Enterprise T5120/T5140/T5220/T5240/T5440 Servers Installation and Service Manual Part No. E20767-01 April 2011...
Page 2
INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. Unless otherwise expressly set forth in such agreement, to the extent allowed by applicable law, in no event shall Oracle or Fujitsu Limited, and/or any of their affiliates have any liability to any third party under any legal theory for any loss of revenues or profits, loss of use or data, or business interruptions, or for...
Page 3
Ce document, bien qu’il vous ait été fourni, ne vous confère aucun droit et aucune licence, expresses ou tacites, concernant le produit ou la technologie auxquels il se rapporte. Par ailleurs, il ne contient ni ne représente aucun engagement, de quelque type que ce soit, de la part d’Oracle ou de Fujitsu Limited, ou des sociétés affiliées de l’une ou l’autre entité.
Contents Preface xi Overview 1–1 General Description 1–2 1.1.1 Chassis 1–4 1.1.2 Power Supply Units 1–5 1.1.2.1 AC Power 1–6 1.1.2.2 Fans 1–6 1.1.3 I/O Boats 1–6 Card Slots 1–8 1.2.1 Carriers 1–9 1.2.1.1 Carrier Slots 1–9 1.2.1.2 Dummy Cards 1–11 1.2.2 Link Kit 1–12 1.2.3...
Page 6
1.3.3 Tightening Sequence for Card Locks 1–20 1.3.3.1 Tightening Sequence for Wide Cards 1–21 1.3.3.2 Tightening Sequence for Narrow Cards 1–21 1.3.4 Examples of PCI Card Installation 1–22 1.3.4.1 Using Card Locks With Tall PCI Cards 1–24 1.3.4.2 Using Card Locks With Low and Very Low Height PCI Cards 1–25 1.3.4.3 Using Card Locks With Unusual PCI Card Shapes 1–26...
Page 7
Installing the AC Cords 2–13 Installing the Link Kit 2–16 Working With PCI Cards 3–1 Installing a PCI Card 3–1 Replacing a PCI Card 3–9 Installing Cables for PCI Cards 3–16 Servicing and Replacing Components 4–1 Service Procedures Task Map 4–2 Replacing a Power Supply Unit 4–3 Preparing the Cable Plate for Service 4–5 Replacing a Carrier 4–7...
Page 8
Power Source Requirements A–4 Acoustic Noise Emissions A–4 Agency Compliance Specifications A–5 B. External I/O Expansion Unit LED Status Indicators B–1 LED Locations B–2 LED States B–4 C. Using the ILOM CLI C–1 Using the CLI C–1 C.1.1 Starting the CLI C–2 C.1.2 Listing Valid Targets In a Namespace C–2 C.1.3...
Page 9
D.6.1.1 Causes D–10 D.6.1.2 Power Supply Fault D–10 D.6.1.3 Clearing a Chassis or Boat Fault D–11 D.6.2 Main I2C Bus D–11 D.6.2.1 Causes D–11 D.6.2.2 Clearing a Power Supply Fault D–11 D.6.2.3 Clearing Other Faults D–12 D.6.3 Platform I2C Bus D–12 D.6.3.1 Causes D–12 D.6.3.2...
Page 10
D.6.10 Boat Presence Signal Failure D–17 D.6.10.1 Cause D–17 D.6.10.2 Resolution D–17 D.6.11 Cannot Determine Boat Location D–17 D.6.11.1 Cause D–17 D.6.11.2 Resolution D–17 D.6.12 Link Card Operating in Wrong Mode D–18 D.6.12.1 Cause D–18 D.6.12.2 Resolution D–18 D.6.13 Can’t Read ManR FRUID Data D–18 D.6.13.1 Cause D–18 D.6.13.2...
Preface This manual provides procedures for installing, using, and maintaining the External I/O Expansion Unit on SPARC Enterprise T5120, T5140, T5220, T5240, and T5440 servers. This document is written for technicians, system administrators, authorized service providers, and users who have advanced experience troubleshooting and replacing hardware.
Sun Oracle software-related manuals (Oracle Solaris OS, and so on): ■ http://www.oracle.com/technetwork/documentation/index.html Book Title External I/O Expansion Unit Installation and Service Manual External I/O Expansion Unit Product Notes External I/O Expansion Unit Safety and Compliance Guide Text Conventions This manual uses the following fonts and symbols to express specific types of information.
Notes on Safety Read the following documents thoroughly before using or handling any External I/O Expansion Unit. External I/O Expansion Unit Safety and Compliance Guide. ■ SPARC Enterprise M3000/M4000/M5000/M8000/M9000 Servers Important Legal and ■ Safety Information Documentation Feedback If you have any comments or requests regarding this document, go to the following web site.
Page 14
External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
C H A P T E R Overview The External I/O Expansion Unit provides a host server with additional slots for PCI cards. The single I/O boat configuration provides six slots for I/O cards. ■ The optional two I/O boat configuration ( ) provides twelve slots.
External I/O Expansion Unit, Front and Rear Views FIGURE 1-1 Figure Legend Front view Rear view General Description shows the major units for the External I/O Expansion Unit, which are FIGURE 1-2 described separately in this chapter. Note – All slot numbers run from left to right, regardless of whether you are viewing the front or the back of the External I/O Expansion Unit.
Page 17
Major Units for the External I/O Expansion Unit, Top View FIGURE 1-2 Chassis I/O boat 1 Power Supply Unit 1 Internal AC cable Power Supply Unit 0 Cable management unit (one of two types is included) Centerplane Link kit I/O boat 0 Chapter 1 Overview...
1.1.1 Chassis The External I/O Expansion Unit chassis includes the centerplane (item 4 in ) and two non-removable internal AC cables (item 7 in FIGURE 1-2 FIGURE 1-2 There are no serviceable components inside the chassis. If the centerplane or the internal AC cables are damaged, the chassis must be replaced.
1.1.2 Power Supply Units The External I/O Expansion Unit has two power supply units (PSUs) for redundancy. See FIGURE 1-3 Each PSU includes an integral fan. Power Supply Unit FIGURE 1-3 Figure Legend Handle locking screw Caution labels Handle PSU slot 0 AC switch PSU slot 1 Chapter 1...
Number of I/O Boats Per Server TABLE 1-1 Server Model Maximum Number of I/O Boats Supported SPARC Enterprise T5120 Server SPARC Enterprise T5140 Server SPARC Enterprise T5220 Server 2 (Maximum of 1 I/O boat per processor) SPARC Enterprise T5240 Server...
Logical Layout of I/O Boats FIGURE 1-5 Note – A switch is a device that connects multiple busses to a single bus. In the I/O boat, all PCI card data passes through the link card. When you run system diagnostics, the switches are displayed in the output of OpenBoot PROM probing.
Slots 1-6 are for PCI cards. (PCI cards are sometimes known as host adapters or ■ host bus adapters.) PCI card slots are not hot-pluggable. ■ The PCI Express I/O boat supports up to x8 card sockets. PCI Express x16 cards ■...
Page 24
PCI Carrier FIGURE 1-6 Figure Legend Carrier handle Carrier locking screw 1-10 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
1.2.1.2 Dummy Cards New carriers are shipped with dummy cards ( FIGURE 1-7 Dummy Card FIGURE 1-7 PCI-E The service labels (not shown in ) on the dummy card include simplified FIGURE 1-7 instructions for removing and installing PCI cards. Note –...
1.2.2 Link Kit One link kit ( ) is required for each I/O boat. FIGURE 1-8 A link kit includes two link cards. One link card goes into the host server. The other link card goes into the I/O boat. The link cards are physically identical. A low-profile bracket is included so a link card can be used in a low-profile I/O card slot.
1.2.3 Cable Management A cable management unit attaches to the rear of the system rack. There are two types of cable management units. Some racks allow the routing of cables along both sides of the rack. The type A ■ cable plate ( ) supports cable routing along both the left and right sides FIGURE 1-9...
Page 28
Cable Management Unit (Type B) for Routing Cables Only to the Right Side of FIGURE 1-10 a Rack Figure Legend Type B cable plate Support brackets Cable plate locking screws Note – If the PSU1 power cable does not reach the rack power distribution unit, route the cable on the left side of the rack.
1.2.3.1 Minimum Bend Radius for Link Cables The link cables might be damaged if they are coiled too tightly. The minimum bend radius for the link cable is 1.85 in. (47 mm). Caution – Coiling the link cables with a smaller bend radius than listed above will break the cables.
Carriers In the I/O boat, all PCI cards are mounted on carriers. When you insert the carrier and card into the boat and push the carrier handle into the closed position, the carrier mechanism automatically seats the PCI card. shows a carrier with an attached PCI card. FIGURE 1-12 Note –...
Carrier Features FIGURE 1-13 Figure Legend Carrier main body (metal) Carrier handle Section 1.3.2, “Card Locks” on page 1-19) Carrier plate (plastic) Card locks (see Carrier slot keyholes Turnaround area for card lock 1.3.1 Carrier Removal and Insertion A carrier operates by raising or lowering a PCI card into or out of a card socket. The vertical movement is approximately 0.4 in.
Page 32
After the carrier is out of the I/O boat, you can unlock the carrier handle by pushing in the metal latch while pushing the carrier handle into its closed position. Note that the closed position provides more vertical clearance for a PCI card when you install or remove the PCI card.
1.3.2 Card Locks A PCI card is attached to the carrier with screw-mounted locks or retainers ). Card locks hold a PCI card to its carrier and prevent the PCI card from FIGURE 1-15 shifting or tilting. The PCI card must be securely mounted in order for the carrier to correctly seat the PCI card in the card socket in the I/O boat.
There are three types of locks: Type A is a square-shaped lock. Three of these are supplied with each carrier. ■ Type B is an oblong lock that has an S-shaped profile. It provides a longer ■ extension (0.75 in., 19 mm) than type A. Type B can be used as a top or side mount, wherever a longer reach is needed.
1.3.3.1 Tightening Sequence for Wide Cards Card Lock Sequence for Wide Cards FIGURE 1-16 ● Rest the bottom edge of the PCI card on the type C card lock (number 1 in ) to ensure that the bottom of the PCI card is perfectly horizontal, FIGURE 1-16 then tighten the remaining card locks in the sequence shown.
1.3.4 Examples of PCI Card Installation shows the locks for a typical PCI card. FIGURE 1-15 However, cards can be much wider or narrower, or taller or shorter. FIGURE 1-18 , and show how cards can vary in height, width, and shape. FIGURE 1-19 FIGURE 1-20 Note –...
Page 37
Card and Lock Styles (Continued) TABLE 1-2 Suggested Lock Type PCI Card Shape Top Lock/Card Height Side Lock/Card Width Bottom Lock Example Low height Type A Type A FIGURE 1-19 2.0 in. (51mm) 5.75 in. (146 mm) (item 1) minimum card height minimum card width Type B 5.0 in.
1.3.4.1 Using Card Locks With Tall PCI Cards Lock Arrangements for Wide and Narrow PCI cards FIGURE 1-18 Figure Legend Tall and wide card 2 type A on top, 1 type A on right, 1 type C on bottom of the PCI card Tall and average width card 2 type A on top, 1 type A on right, 1 type C on bottom of the PCI card Tall and narrow card...
1.3.4.2 Using Card Locks With Low and Very Low Height PCI Cards Lock Arrangements for Low and Very Low Height PCI cards FIGURE 1-19 Figure Legend Low height card 2 type A on top, 1 type A on right side, 1 type C on lower right side of the PCI card Very low height card 1 type B on top, 1 type A on right edge, 1 type C on lower right side of the PCI card Chapter 1...
1.3.4.3 Using Card Locks With Unusual PCI Card Shapes Lock Arrangements for Unusually-Shaped Cards FIGURE 1-20 Figure Legend Very low height and narrow width card 1 type B on top, 1 type C on right side of the PCI card Irregularly-shaped card 1 type A and 1 type B on top, 1 type A on right side, 1 type C on lower right side of the PCI card...
Page 41
Excessive Force on a Lock Can Bend or Break the PCI Card FIGURE 1-21 Figure Legend Correct Incorrect Here are some rules to avoid the tilting of PCI cards: 1. You must have at least one lock on top of the card. If the top of the card is too low to accept a lock, the card cannot be used.
1.3.5.2 Hidden Problems The following features are located inside the front housing of the carrier. Because they are difficult to see, they can complicate the mounting of cards on carriers. Card alignment post ( ■ FIGURE 1-22 FIGURE 1-23 Card alignment tab ( ■...
Page 43
The post fits in a notch in the metal bracket of the PCI card (item 1 in ). If FIGURE 1-22 you do not position the post in the notch, the card mounting bracket might bend, so that the card lies at an angle on the carrier. The angle prevents the card from making proper electrical contact with the socket in the I/O boat.
Page 44
Card Alignment Post (Detail) FIGURE 1-23 Figure Legend Carrier front housing Card alignment post PCI card bracket 1-30 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
Page 45
Internal RFI Gasket An RFI gasket is located inside the carrier housing, next to the card alignment post. (A smaller RFI gasket is located on the outside of the carrier housing.) When you insert the PCI card into the carrier, be sure that the bottom of the metal card bracket does not scrape or loosen the bottom of the gasket ( FIGURE 1-24 The gasket material is flexible enough that you might not notice that the bracket has...
1.3.6 Carrier Keys Each carrier is keyed to a specific slot ( ) in the I/O boat. The key is an M2 FIGURE 1-25 screw on the top of each carrier. Each carrier slot (0 through 6) will accept only a carrier that has a key in the hole location for that slot.
External I/O Expansion Unit Configurations The External I/O Expansion Unit is available with one or two I/O boats. 1.4.1 Single Boat Configuration The base configuration for an External I/O Expansion Unit has a single boat, with a filler panel in the second boat bay. A link card in the host server connects to a link card in the External I/O Expansion Unit.
LEDs LEDs are located on the front ( ) and rear ( ) of the chassis and FIGURE 1-26 FIGURE 1-27 on individual PSUs. Appendix B for information about interpreting LED states. LEDs on the Front of the Chassis FIGURE 1-26 Figure Legend Chassis locate (LED and switch) PSU0 DC power...
LEDs on the Rear of the Chassis FIGURE 1-27 Figure Legend Chassis locate (LED and switch) I/O boat 1 ready to remove Chassis fault/service required I/O boat 1 fault/locate Chassis power I/O boat 1 DC power Chassis overtemp Link card data I/O boat 0 ready to remove Link card management I/O boat 0 fault/locate...
Temperature sensors are also located inside the I/O boats. FRU ID circuits are located on the link cards, the PSUs, the I/O boats, and on the chassis centerplane. Temperature data and FRU ID information is available on an I2C bus (Inter-IC bus) in the External I/O Expansion Unit and the link cards.
Site Preparation The following information summarizes installation requirements for the External I/O Expansion Unit. For additional specifications and compliance information, see Appendix 1.7.1 Physical Requirements The External I/O Expansion Unit with the cable management unit attached is 17.3 ■ in. (440 mm) wide and 39.4 in. (1000 mm) deep. The movement of air through the External I/O Expansion Unit chassis is from ■...
Service Information Service and installation information is also available on service labels that are located on the External I/O Expansion Unit top cover and on the dummy cards that are shipped with new carriers. Service Information Summary TABLE 1-4 Topic Comments Access •...
Service Information Summary (Continued) TABLE 1-4 Topic Comments Link cables • The link kit has one cable. The connector is designed in a way such that it cannot be connected upside down. Jumpers • The External I/O Expansion Unit does not have jumper pins. •...
Page 54
1-40 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
C H A P T E R Installing the External I/O Expansion Unit in a Rack The following topics are in this chapter: Section 2.1, “Tools” on page 2-1 ■ Section 2.2, “Installing the Mounting Brackets in a Rack” on page 2-2 ■...
Installing the Mounting Brackets in a Rack The External I/O Expansion Unit mounting kit ( ) includes a right-side FIGURE 2-1 mounting bracket and a left-side mounting bracket. The mounting brackets are adjustable for length and are shipped unassembled. The mounting kit includes two chassis lock brackets.
Page 57
If you are installing more than one External I/O Expansion Unit, install them ■ together. Do not install a shorter subassembly between External I/O Expansion Unit where the rear of the shorter subassembly might difficult to reach. If the rack is marked with rack units, place the mounting bracket so that the lower ■...
Page 58
Installing the Chassis Lock Brackets FIGURE 2-3 4. Loosen the screws ( ) that hold the sliding flanges to the mounting FIGURE 2-4 brackets. This action allows the rear flanges to adjust to fit different rack depths. External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
Page 59
Sliding Flange FIGURE 2-4 Figure Legend Sliding flange Flange crews Step 5. If your rack has threaded holes, continue at 6. If your rack has square holes, install cage nuts in the rack pillars. lists the locations for the cage nuts. Note that these are relative locations. TABLE 2-1 Adjust the actual hole locations as needed to leave space for a power distribution unit, additional External I/O Expansion Units, or other rack-mounted equipment.
Page 60
Cage Nut Locations TABLE 2-1 Rack Unit Hole No. Front Rear cage nut cage nut cage nut cage nut cage nut cage nut cage nut cage nut 7. Attach the mounting brackets to the rack ( FIGURE 2-5 a. At the front of the rack, orient the hooked portion of the mounting bracket away from you.
Installing the Mounting Brackets in a Rack FIGURE 2-5 8. Tighten the screws at the front of the rack. 9. Verify that the brackets at the rear of the rack can still slide slightly to the left and right. Installing the External I/O Expansion Unit in the Rack 1.
Page 62
of the chassis contact hooks that are located on the mounting brackets, wedging the mounting brackets tightly against the sides of the chassis. This wedging action stabilizes the External I/O Expansion Unit and is necessary to reduce the amount of vibration that occurs when the system is running. 3.
Installing the Cable Management Unit At the rear of the rack, install the cable management support brackets: 1. Use an antistatic strap for this procedure. 2. Place the support brackets just above the chassis mounting brackets ( FIGURE 2-7 and loosely attach the support brackets to the rack with two screws each. Do not tighten the screws yet.
Page 64
Installing the Support Brackets FIGURE 2-7 2-10 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
Page 65
Type A Cable Plate FIGURE 2-8 Type B Cable Plate FIGURE 2-9 Chapter 2 Installing the External I/O Expansion Unit in a Rack 2-11...
Page 66
3. Place the cable plate between the support brackets ( FIGURE 2-10 On each side of the cable plate, the forward tab rests on the bottom of the large cutout in the support bracket. The rear tab rests in a small depression in the top of the support bracket.
Installing the AC Cords 1. Use an antistatic strap for this procedure. 2. Attach an AC cord to an internal AC cable at the rear of the External I/O Expansion Unit, then connect the AC cord to an AC outlet. The PSUs should be connected to two independent external AC power sources so ■...
Page 68
AC Cables for Type A Cable Management Plate FIGURE 2-11 Figure Legend Internal AC cable, left Internal AC cable, right 2-14 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
Page 69
AC Cables and Cords for Type B Cable Plate FIGURE 2-12 Figure Legend Internal AC cable, left AC cord (connectors are placed in rectangular opening to reduce height) Internal AC cable, right Chapter 2 Installing the External I/O Expansion Unit in a Rack 2-15...
4. If you wish to test the External I/O Expansion Unit for basic electrical functionality, do the following: a. Turn on the PSU AC switches. The switches are located on the front of the power supplies. The LEDs on the power supplies should display the following indications after a short initialization period.
Page 71
Note – The minimum bend radius for the link cable is 1.85 in. (47 mm). 1. Use an antistatic strap for this procedure. 2. Install a low profile bracket on one link card, as follows. Note – The low profile bracket is used only in the host server. The link card in the I/O boat must use a full height bracket.
Page 72
b. Fit the low profile bracket over the cable connector and use the two No. 1 Phillips screws to attach the bracket to the card. See detail 2 in FIGURE 2-13 Note – A link kit might include an extra low profile bracket. If you are installing two I/O boats in the External I/O Expansion Unit, there might be two unused low profile brackets after you finish the installation.
Page 73
Note – If it is necessary to disconnect a link cable, pull the ring-shaped tab approximately 0.15 in./2 mm in the direction shown by the arrow in FIGURE 2-15 Caution – Do not pull the ring tab more than 0.15 in./2 mm, or it will break. When disconnecting the cable from a link card, pull the cable plug while pulling carefully on the ring tab.
Page 74
2-20 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
C H A P T E R Working With PCI Cards This chapter gives procedures for installing PCI and link cards, and related cables. Section 3.1, “Installing a PCI Card” on page 3-1 ■ Section 3.2, “Replacing a PCI Card” on page 3-9 ■...
Page 76
Unlocking and Removing a Carrier FIGURE 3-1 Figure Legend Carrier handle Carrier locking screw 4. Press lightly on the front of the carrier, and pull out the carrier handle until it clicks into the open position (approximately 1.5 in./38 mm). Pressing the front of the carrier prevents movement of the carrier when you pull the carrier handle.
Page 77
Locator Bar on Carrier FIGURE 3-2 5. Pull the carrier out of the slot and place it on an antistatic work surface. 6. Press the locking latch ( ) while you push the carrier handle into the FIGURE 3-3 closed position. This action lowers the carrier plate, giving you more room to move the PCI card into place on the carrier.
Page 78
7. Place the carrier on a padded static-safe surface and loosen the card lock screws. The screws are on the back of the carrier ( FIGURE 3-4 Screws for Card Locks FIGURE 3-4 Figure Legend Card lock screws 8. Turn the carrier over and remove the dummy card or PCI card. 9.
Page 79
Inserting the PCI Card FIGURE 3-5 Figure Legend See the following caution. Caution – Do not allow the bottom edge of the PCI card bracket to tear the RFI gasket in the front of the carrier ( FIGURE 3-5 b. A rectangular notch is located on the bottom front of the PCI card. Fit the notch over the card alignment tab (item 1 in ), then fit the top of the FIGURE 3-6...
Page 80
Placing the PCI Card on the Carrier FIGURE 3-6 Figure Legend Card alignment tab Card alignment post 11. Attach the PCI card to the carrier with one card lock: a. Slide a card lock to the right edge of the PCI card. b.
Page 81
Using Card Lock to Hold the PCI Card FIGURE 3-7 Figure Legend Card lock 12. Slide the remaining card locks to the edges of the PCI card and tighten the screws. The shape of a PCI card might prevent the use of some card locks. The minimum requirement is one card lock on the side of the PCI card and one card lock on the top.
Page 82
Caution – Do not apply excessive pressure on the upper card locks. Too much pressure will bend the PCI card bracket. See FIGURE 3-9 Excessive Pressure Bends the PCI Card Bracket FIGURE 3-9 Figure Legend Moderate pressure on an upper card lock holds the PCI card in a horizontal position. Excessive pressure bends the PCI card bracket.
17. Use the carrier locking screw located on the top of the handle to lock the carrier handle in place. 18. Add the PCI card to an Oracle Solaris domain. For details, refer to the service manual for your host server.
Page 84
3. Remove the carrier from the carrier slot in the I/O boat. a. Loosen the carrier locking screw on the carrier handle. b. Press lightly on the front of the carrier, and pull the carrier handle out fully. Caution – Pulling the carrier handle raises the carrier plate. If the carrier moves forward before the carrier plate is completely raised, the plastic locator bar ) might be damaged by hitting the lower front edge of the I/O boat.
Page 85
8. Place the PCI card on the carrier. a. Place the front of the card inside the housing ( FIGURE 3-12 Inserting the PCI Card FIGURE 3-12 Figure Legend See the following caution. Caution – Do not allow the bottom edge of the PCI card bracket to tear the RFI gasket in the front of the carrier ( FIGURE 3-5 b.
Page 86
Placing the PCI Card on the Carrier FIGURE 3-13 Figure Legend Card alignment tab Card alignment post 9. Attach the PCI card to the carrier with one card lock: a. Slide a card lock to the right edge of the PCI card. b.
Page 87
Using Card Lock to Hold the PCI Card FIGURE 3-14 Figure Legend Card lock 10. Slide the remaining card locks to the edges of the PCI card and tighten the screws. The shape of a PCI card might prevent the use of some card locks. The minimum requirement is one card lock on the side of the PCI card and one card lock on the top.
Page 88
Caution – Do not apply excessive pressure on the upper card locks. Too much vertical pressure will bend the PCI card bracket. See FIGURE 3-16 Excessive Pressure Bends the PCI Card Bracket FIGURE 3-16 Figure Legend Moderate pressure on an upper card lock holds the PCI card in a horizontal position. Excessive pressure on an upper card lock can bend the PCI card bracket.
Page 89
Locator Bar on Carrier (Carrier Handle Shown in the Closed Position) FIGURE 3-17 12. Carefully place the carrier in the card guides at the top and bottom of the carrier slot in the I/O boat. 13. Press on the metal front panel of the carrier to push the carrier fully into the carrier slot.
15. Use the carrier locking screw located on the top of the handle to lock the carrier handle in place. 16. Add the PCI card to an Oracle Solaris domain. For details, refer to the service manual for your host server.
Page 91
Example of Cable Management for the Type A Cable Plate FIGURE 3-19 Chapter 3 Working With PCI Cards 3-17...
Page 92
Example of Cable Management for the Type B Cable Plate FIGURE 3-20 3. Attach I/O cables from the PCI cards and their respective devices or network ports. 3-18 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
C H A P T E R Servicing and Replacing Components This chapter includes the service procedures for replacing field-replaceable units and installing optional assemblies. This chapter includes the following topics: Section 4.1, “Service Procedures Task Map” on page 4-2 ■...
Service Procedures Task Map Service Task Map TABLE 4-1 Task Comments PCI card • Install a new PCI card Section 3.1, “Installing a PCI Card” on page 3-1 • Replace an existing PCI card Section 3.2, “Replacing a PCI Card” on page 3-9 •...
Replacing a Power Supply Unit This is a hot replacement procedure, meaning that the PSU can be replaced while the system is still running, provided that a second PSU is working properly. 1. Use an antistatic strap for this procedure. 2.
Page 96
FIGURE 4-1 Figure Legend Handle Handle locking screw Warning labels 7. Set the PSU flat on an antistatic surface. 8. Insert the new PSU into the External I/O Expansion Unit. The pivoting flap automatically swings up and out of the way. Note –...
Preparing the Cable Plate for Service On the cable management unit, raise the cable plate ( ) to FIGURE 4-2 FIGURE 4-3 provide easier removal of a PCI carrier or I/O boat. 1. Use an antistatic strap for this procedure. 2.
Page 98
Type B Cable Plate FIGURE 4-3 The left side of shows the cable plate in its normal position. FIGURE 4-4 The right side of the illustration shows the service position. Cable Plate (Side Views of Normal and Service Positions) FIGURE 4-4 Figure Legend Cable plate in the normal position (lowered) Cable plate in the service position (raised)
Replacing a Carrier The External I/O Expansion Unit must be powered off during this procedure. A carrier is keyed to a specific slot in an I/O boat. The key is an M2 screw FIGURE 4-5 Screw Hole Locations for the Carrier Slot Key FIGURE 4-5 Figure Legend Keyholes 0-7...
Page 100
a. Loosen the cable plate locking screw(s) (item 3 in FIGURE 4-4 b. Pull the cable plate toward you approximately 0.5 in./13 mm, then raise it approximately 0.75 in./20 mm, until the bottom of the plate is above the tops of the support brackets.
Page 101
b. Examine the defective carrier and determine which card locks on the replacement carrier will be used. Move the card locks on the replacement carrier to approximately the same locations as on the defective carrier. One or more card locks on the replacement carrier might not be needed. Move the unused card locks out of the way or else remove them entirely from the carrier.
Replacing a Link Cable The External I/O Expansion Unit must be powered off during this procedure. Note – Ordering a complete set of replacements at one time can shorten the troubleshooting process. If you cannot determine if a link problem is caused by a link card or a link cable, order replacement link cards for the host and the I/O boat, along with a copper link cable or two optical link cables, as appropriate.
2. Stop the domains that are connected to the External I/O Expansion Unit. For information about stopping a domain, refer to the documentation for the server. 3. Turn off the server. For information about powering down a server, refer to the documentation for the server.
Page 104
4. Move the cable plate to the service position: a. Loosen the cable plate locking screw(s) (item 3 in FIGURE 4-4 b. Pull the cable plate toward you approximately 0.5 in./13 mm, then raise it approximately 0.75 in./20 mm, until the bottom of the plate is above the tops of the support brackets.
Caution – Do not press the plastic carrier handle during Step c. If you press the handle, the PCI card and the sliding carrier plate might suddenly fall, breaking the plastic locator bar ( ) by hitting it against the lower edge of the I/O boat. FIGURE 3-17 d.
Page 106
2. Turn off the domains that are connected to the External I/O Expansion Unit. For information about stopping a domain, refer to the documentation for the server. 3. Unpack the new I/O boat and set it flat on an antistatic surface. Caution –...
Page 107
c. Press on the metal front panel of the carrier to push the carrier fully into the carrier slot. Caution – Do not press the plastic carrier handle during Step c. If you press the handle, the PCI card and the sliding carrier plate might suddenly fall, breaking the plastic locator bar ( ) by hitting it against the lower edge of the I/O boat.
12. Move the cable plate back into the normal position ( ) and tighten the FIGURE 4-4 cable plate locking screw(s). 13. Attach the I/O cables to the PCI cards. 14. Attach the I/O cables to the cable management unit on the External I/O Expansion Unit.
Page 109
3. Turn off the AC switches on both PSUs at the front of the External I/O Expansion Unit. 4. Move the cable plate to the service position: a. Loosen the cable plate locking screw(s) (item 3 in FIGURE 4-4 b. Pull the cable plate toward you approximately 0.5 in./13 mm, then raise it approximately 0.75 in./20 mm, until the bottom of the plate is above the tops of the support brackets.
Page 110
Caution – The carrier can be damaged during removal and during insertion. For safe handling of the carrier, read the cautions in Section 3.1, “Installing a PCI Card” on page 3-1. 12. Install the link card and carrier in slot 0 of the new I/O boat: a.
17. Turn on the AC switches on the External I/O Expansion Unit PSUs. On the PSUs the AC power ( ) and DC power ( ) LEDs should light. For other LED indications, see Appendix 18. Start the domains that are connected to the External I/O Expansion Unit. For information about starting a domain, refer to the documentation for the server.
Page 112
Caution – Do not allow the rear of the I/O boat to drop. The connector at the rear of the boat can be damaged if it strikes the edge of the slot. Do not rest an I/O boat vertically. The plastic electrical connector on the back is breakable.
Caution – If the carrier handle does not close easily, the PCI card might not be seating properly in the card socket in the I/O boat. Remove the carrier and inspect the PCI card. If the bottom of the PCI card is not perfectly horizontal, loosen the card locks and realign the card on the carrier.
4.10.1 Locating the New System Serial Number Label The replacement chassis has a new system serial number. This serial number label is located inside the left I/O boat bay. In the following procedure, new serial number labels will be attached to the old bezel, and the old bezel will be used with the new chassis.
Page 115
8. Disconnect the link cable from the link card in slot 0 in each I/O boat 9. Disconnect the I/O cables from the PCI cards in the I/O boat(s). 10. Remove the I/O boat(s) from the External I/O Expansion Unit: a.
12. Remove the External I/O Expansion Unit chassis from the rack. a. Use a No. 2 Phillips screwdriver to remove the four mounting screws from the front of the chassis ( FIGURE 2-6 b. Pull the chassis out of the rack and place it on an antistatic workbench. The empty chassis weighs 30 lb (13.6 kg).
Page 117
Removing the Chassis Top Cover FIGURE 4-6 Figure Legend Top screw locations (10 total) Side screw locations (4 total) b. Set the cover aside. c. Use a No. 1 Phillips screwdriver to remove 4 screws from the top of the bezel ), then tilt the top of the bezel away from the External I/O FIGURE 4-7 Expansion Unit and lift it off.
Page 118
Bezel Screws FIGURE 4-7 Figure Legend Bezel screws 2. Install new serial number labels on the bezel. The current serial number labels are located at the left PSU opening of the bezel FIGURE 4-8 4-26 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
Page 119
Serial Number Labels FIGURE 4-8 Figure Legend Upper serial number label Lower serial number label 3. Use a No. 1 Phillips screwdriver to remove the top cover ( ) from the FIGURE 4-6 new chassis. 4. Set the cover aside. This cover will be used with the new chassis.
6. Install the new top cover on the replacement External I/O Expansion Unit chassis. Note – The new top cover does not have system serial number label. The new system serial number is located only on labels on the bezel ( ) and inside the FIGURE 4-8 left I/O boat bay of the FRU chassis.
Page 121
Uninstalling the Chassis Lock Brackets FIGURE 4-9 3. Set the old chassis aside for return or proper disposal. 4. Install the chassis lock brackets on the sides of the replacement chassis. 5. Slide the replacement chassis into the rack and lock the chassis in place, using two No.
Page 122
Installing the External I/O Expansion Unit in the Rack FIGURE 4-10 6. At the front of the chassis, install the PSUs: a. Pull the PSU handle to the unlocked position. b. Slide the PSU into the slot until the rear of the PSU contacts the centerplane. c.
Page 123
c. Install I/O boat 1 in the same manner. 8. Lower the cable plate from the service position to the normal position FIGURE 4-4 9. Attach the I/O cables to the PCI cards. 10. For each I/O boat, attach the link cable(s) to the link card in the boat. 11.
Page 124
4-32 External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
A P P E N D I X Specifications This guide provides the specifications and site requirements you need when planning for the installation of an External I/O Expansion Unit. Topics in this appendix include: Section A.1, “Physical Specifications” on page A-2 ■...
Physical Specifications Measure U.S. Metric Width 17.3 in. 440 mm Depth 29 in. 730 mm Depth with cable management unit 39 in. 1000 mm Height (4 rack units) 6.9 in. 175 mm Weight with 1 I/O boat and 2 PSUs 63 lb 28.6 kg Weight with 2 I/O boats and 2 PSUs...
Environmental Specifications Environmental Factor Operating Range Nonoperating Range Optimum Ambient 5˚C to 35˚C Unpacked: 21˚C to 23˚C temperature (41˚F to 95˚F) (70˚F to 74˚F) 0˚C to 50˚C (32˚F to 122˚F) Packed: ‡ -20˚C to 60˚C (-4˚F to 140˚F Relative 20% RH to 80% RH, Up to 93% RH, 45% RH to 50% RH, humidity...
Power Source Requirements Description Specification Input voltage range 100 VAC to 240 VAC, 50-60 Hz Maximum input current 8.0 A at 100 VAC 4.0 A at 220 VAC Rush current, per AC cord 30 A (peak) Maximum input power 600 W Phases Single Leakage current, per AC cord...
A P P E N D I X External I/O Expansion Unit LED Status Indicators The following tables list the status states for the LEDs on the front ( ) and FIGURE B-1 rear ( ) of the External I/O Expansion Unit. FIGURE B-2 Topics in this appendix include: Section B.1, “LED Locations”...
Page 132
LED Locations LEDs on the Front of the Chassis FIGURE B-1 Figure Legend Chassis locate (LED and switch) PSU0 DC power Chassis fault/service required PSU0 AC power Chassis power PSU1 ready to remove Chassis overtemp PSU1 fault/locate PSU0 ready to remove PSU1 DC power PSU0 fault/locate PSU1 AC power...
Page 133
LEDs on the Rear of the Chassis FIGURE B-2 Figure Legend Chassis locate (LED and switch) I/O boat 1 ready to remove Chassis fault/service required I/O boat 1 fault/locate Chassis power I/O boat 1 DC power Chassis overtemp Link card data I/O boat 0 ready to remove Link card management I/O boat 0 fault/locate...
LED States Note – Text shown in bold font indicates the normal operating state for each LED. External I/O Expansion Unit Chassis (Front) TABLE B-1 LED Name Color State Meaning Locate White Flash The unit is located here. (4 Hz) To turn off the flashing LED, press the button for 0.5 second or longer.
Page 135
I/O Boat TABLE B-3 LED Name Color State Meaning Power Green Boat power on, and boat is managed. Boat power off. Flash Boat power on, and boat is not managed. Fault/Locate Amber No fault. Fault present. Flash Locate on. Ready to remove Blue Not ready to remove.
Page 136
Link Card TABLE B-6 LED Name Color State Meaning Data (right LED) Green x8 PCI Express links. Link is down. Flash x1 or x4 PCI Express link (degraded state). (1Hz slow flash) Management (left LED) Green Management link is up. (Flashes in time with data traffic) Flash Management link is down.
A P P E N D I X Using the ILOM CLI To check the status of the External I/O Expansion Unit (I/O Box), use the command line interface (CLI) of the Integrated Lights Out Management (ILOM). See the examples in this appendix. Topics in this appendix include: Section C.1.1, “Starting the CLI”...
C.1.1 Starting the CLI ● To start the ILOM CLI, type: # spsh C.1.2 Listing Valid Targets In a Namespace ● To list all valid targets in the namespace, type: -> help targets The I/O Box and its FRUs are shown in the list of targets. C.1.3 Listing Targets Below a Namespace ●...
C.1.5 Exiting the CLI ● To exit from the CLI, type: -> exit Examples of CLI Commands for an I/O This section shows how ILOM CLI commands can be used with an I/O Box. To display information about an I/O Box, first navigate to the host server’s /SYS namespace: ->...
Page 140
Navigate to the I/O Box namespace: -> cd IOX@nnnn /SYS/IOX@nnnn Display information for IOX@nnnn. You can use the -display all option to increase the quantity of information: -> show -display all /SYS/IOX@nnnn Targets: ACTIVE OVERTEMP SERVICE LOCATE IOB0 IOB1 Properties: type = I/O Expansion Unit fru_description = Centerplane for IOBOX fru_manufacturer = manufacturer...
Page 141
Display information about boat 0 in the I/O Box by typing: -> show -display all /SYS/IOX@nnnn/IOB0 /SYS/IOX@X0Y1/IOB0 Targets: ACTIVE RDY2RM SERVICE T_CHIP T_HOTSPOT V_12V_MAIN V_12_0V V_1_0V V_1_5V V_3_3V V_3_3AUX I_DC LINK Properties: type = I/O Boat fru_description = PCIE Board for IOBOX fru_manufacturer = manufacturer fru_version = 05_51 fru_part_number = 5016939...
Page 142
Display information about power supply 1 in the I/O Box by typing: -> show -display all /SYS/IOX@nnnn/PS0 /SYS/IOX@X0Y1/PS0 Targets: DCOK POWER RDY2RM SERVICE T_AMBIENT T_CHIP T_HOTSPOT V_12V_ANODE V_12V_CATHODE V_ISHARE I_DC S_FAN_SET S_FAN_ACTUAL SWITCH Properties: type = Power Supply fru_name = A195 fru_description = A195 Power Supply fru_manufacturer = Tyco Electronics fru_version = 02_04...
Page 143
Display information about a specific temperature sensor in power supply 1 in the I/O Box by typing: -> show -display all /SYS/IOX@nnnn/PS1/T_AMBIENT /SYS/IOX@X0Y1/PS1/T_AMBIENT Targets: Properties: type = Temperature class = Threshold Sensor value = 25.398 degree C upper_nonrecov_threshold = N/A upper_critical_threshold = 37.00 degree C upper_noncritical_threshold = N/A lower_noncritical_threshold = N/A...
Page 144
External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
A P P E N D I X Troubleshooting with ILOM Diagnostic Messages Use this appendix to understand Integrated Lights Out Manager (ILOM) messages for SPARC Enterprise T5xxx servers. Topics in this appendix include: Section D.1, “ILOM Messages” on page D-2 ■...
ILOM Messages D.1.1 Types of ILOM Messages for External I/O Expansion Unit There are three categories of ILOM messages for the External I/O Expansion Unit, as shown in . For the External I/O Expansion Unit, all ILOM diagnostic TABLE D-1 messages begin with the word Ext.
Locating a Faulty FRU (Continued) TABLE D-2 Term Definition A link card, either in the server or in a boat. LINK When the link card is in a server I/O slot, a full address is typically /SYS/MB/PCIEn/LINK. When the link card is in a boat, a full address is typically /SYS/IOX@nnnn/IOBn/LINK.
D.2.2 When Service Is Not Required Messages that include the text string Ext info provide status information. There will be no entry under the ILOM show faulty command or ILOM BUI Fault Management tab for these messages. These messages are maintained in the ILOM event log and can be displayed either through the ILOM CLI or the Event Logs tab in the ILOM BUI.
Ext FRU Faults (listed in alphabetical order) (Continued) TABLE D-3 Message Link card in the server Ext FRU /SYS/MB/PCIEn/LINK SIGCON=0 I2C no device response Link card in the server Ext FRU /SYS/MB/PCIEn/LINK SIGCON=1 I2C no device response Link card in the server Ext FRU /SYS/MB/PCIEn/LINK XMTR transmitter fault Ext info Messages Service is not immediately required.
Ext info Messages (listed in alphabetical order) (Continued) TABLE D-4 Message Meaning PSU n, thermal information Ext info /SYS/IOX@nnnn/PSn THERMCTRL/T_AMBIENT has exceeded high warning threshold PSU n, thermal information Ext info /SYS/IOX@nnnn/PSn THERMCTRL/T_AMBIENT is within normal range PSU n, thermal information Ext info /SYS/IOX@nnnn/PSn THERMCTRL/T_HOTSPOT has exceeded high warning threshold PSU n, thermal information...
Page 152
Ext sensor Faults (listed in alphabetical order) (2 of 4) TABLE D-5 Message Section D.6.15, “Fan Controller Ext sensor /SYS/IOX@nnnn FANCTRL timeout. Fault not Timeout” on page D-19 isolated. Section D.6.1, “I2C_A Bus and I2C_B Ext sensor /SYS/IOX@nnnn I2C_A I2C bus fault. Fault not Bus”...
Page 153
Ext sensor Faults (listed in alphabetical order) (3 of 4) TABLE D-5 Message Section D.6.10, “Boat Presence Signal Ext sensor /SYS/IOX@nnnn/IOBn boat presence signal Failure” on page D-17 failure. Fault not isolated. Section D.6.6, “Interrupt Signal Ext sensor /SYS/IOX@nnnn/IOBn/LINK CONTROLLER I2C Failures”...
3. If thermal and electrical readings are present for the power supply, there is no fault. 4. If thermal and electrical readings are not present for the power supply, swap PS0 and PS1. 5. If the problem follows the power supply, replace the power supply. 6.
3. If FRUID ManR data is reported for the power supply, there is no fault. 4. If FRUID ManR data is not reported for the power supply, continue with the following steps. D.6.2.3 Clearing Other Faults If a new FRU was recently added to the system, remove the FRU to see if the ■...
Note – For this test, the link card in the server can be swapped with the link card in the boat. D.6.4 Management Bus Link Down The server has detected loss of both the management link and the data path to an I/O Box.
D.6.5.2 Resolution Swap the link cable. ■ Swap the link cards. ■ D.6.6 Interrupt Signal Failures I2C components in the FRUs generate interrupts to report hardware state changes and alarm events. Error messages are generated if a problem is discovered during an interrupt signal test, or if an interrupt cannot be cleared during normal interrupt processing.
If no other fault conditions are present, the following interrupt failures indicate a ■ component or connector failure. FRU swapping is necessary to determine which FRU contains the faulty signal. Swap FRUs to detect if the problem stays with the FRU bay or follows the FRU to the new location.
Caution – Boats in the same I/O Box are not allowed to be connected to different servers. There is a potential of data corruption in FRUID SEEPROMs on the centerplane and the power supplies. D.6.8.1 Cause The boats are connected to two different systems. ■...
D.6.9.2 Resolution Reconfigure the system so both boats are connected to the same server. ■ D.6.10 Boat Presence Signal Failure The fan controller on the centerplane monitors thermal sensors in the boats. If the boat is present but the fan controller does not detect it, then the error is generated. D.6.10.1 Cause Signal failure in the boat or the I/O Box centerplane.
D.6.12 Link Card Operating in Wrong Mode The downlink card is receiving packets over the management link, but the format of the data indicates the uplink card (boat link card) is functioning as a downlink card (server link card). A reserved pin in a boat link card slot is used by the link card to determine if the card is installed in the boat or in a host system.
D.6.14 Fan Controller Reset Upon software request, the bridge controller can assert a hardware reset signal to the centerplane. This operation will not result in error messages. Software will re- initialize all I2C devices after a controlled reset. D.6.14.1 Cause Inserting a boat into a chassis can result in an accidental hardware reset.
Cause Failed link card. ■ Resolution Replace the link card. ■ D.6.17 Bridge Controller Timeout If commands are not serviced within the expected time frame (with multiple retries), timeout errors are posted. Cause The I2C_LOCAL bus is faulted. ■ If a PCI Express link is not trained, the bridge controller will not service the ■...
A P P E N D I X Troubleshooting Hardware Problems This appendix discusses general hardware problems for the External I/O Expansion Unit from Oracle and Fujitsu:. For troubleshooting with specific ILOM error messages, see Appendix This appendix includes this topic: Section E.1, “Troubleshooting the External I/O Expansion Unit”...
Page 166
Troubleshooting (Continued) TABLE E-1 Area Symptom Comments • The PCI card does not fit the socket. The card might be the incorrect type for the I/O boat. • The PCI Express boat does not accept x16 PCI Express cards. This boat accepts only x8 PCI Express cards or smaller. Link A link problem is suspected.
Page 167
Troubleshooting (Continued) TABLE E-1 Area Symptom Comments PCI card or carrier is difficult • The card locks might be loose, allowing the card to tilt. PCI card to seat in the boat, or the Reposition the card and tighten the card locks. It is most card moves out of position important that the card lock(s) at the back of the card press...
Page 168
External I/O Expansion Unit Installation and Service Manual for SPARC Enterprise T5xxx Servers • April 2011...
Page 169
Index AC cable, internal, 1-3 electrical requirements, 1-38 AC power, 1-6 ESD precautions, 1-40 air flow direction, 1-39 External I/O Expansion Unit configurations, 1-33 alignment post, carrier, 1-28 alignment tab, carrier, 1-28 fans DC power shared, 1-6 direction of air flow, 1-39 behavior of fan, 1-6 fan operation, 1-6 bezel, in chassis replacement, 4-24...
Page 170
label, serial number, 4-26 rack unit markings, 2-3 labels for PCI carrier, 4-8 radius, minimum bend radius for link cables, 1-15 LED locations, front, 1-34 redundant PSUs, 1-5 LED locations, rear, 1-34 replacement chassis, 1-4 link cable minimum bend radius, 1-15 retainer, see PCI card lock link card not displayed in OpenBoot PROM RFI gasket, 1-31...
Need help?
Do you have a question about the SPARC Enterprise T5120 and is the answer not in the manual?
Questions and answers