Summary of Contents for Oracle Fabric Interconnect Series
Page 1
Oracle Fabric Interconnect Hardware User's Guide Part No: E50997-04 July 2016...
Page 3
Oracle. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except as set forth in an applicable agreement between you and Oracle.
Page 4
Oracle Corporation et ses affiliés déclinent toute responsabilité ou garantie expresse quant aux contenus, produits ou services émanant de tiers, sauf mention contraire stipulée dans un contrat entre vous et Oracle. En aucun cas, Oracle Corporation et ses affiliés ne sauraient être tenus pour responsables des pertes subies, des coûts occasionnés ou des dommages causés par l’accès à...
Pre-Installation Considerations .............. 22 ▼ Install the Fabric Interconnect F1-15 ............ 23 ▼ Attach the Right Angle Bracket ............ 31 ▼ Attach the Right Angle Bracket in a 27” Sun/Oracle Rack ...... 35 Installing the Cable Management Assembly ............ 37 Required Equipment ................ 37 ▼ Install the CMA ................ 38 Installing the Fabric Interconnect F1-4 in a 4-Post Rack ........
Page 6
Contents ▼ Attach the Right Angle Bracket in a 27-Inch Sun/Oracle Rack .... 55 Understanding Chassis Components and Indicators .......... 58 Fabric Interconnect Chassis Components ........... 58 Fabric Interconnect Control Panel ............ 61 Fabric Interconnect I/O Modules .............. 62 10 GE Ethernet I/O Module LED Codes ............ 65 10-Port GE Ethernet I/O Module LED Codes ..........
Page 7
Installation Considerations .............. 156 ▼ Power Down the Oracle Fabric Interconnect ........ 157 ▼ Uncable the Oracle Fabric Interconnect .......... 157 Removing the Oracle Fabric Interconnect from the Equipment Rack .... 157 ▼ Remove the Old Front Panel ............ 160 ▼ Remove the Old Mounting Brackets .......... 161 ▼...
Page 8
Contents Index ........................ 173 Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Using This Documentation Overview – Describes how to install and replace parts in the Oracle Fabric Interconnect F1- ■ 15 and Oracle Fabric Interconnect F1-4, and how to use and maintain the system. Audience – Technicians, system administrators, authorized service providers, and users ■...
Installing the Fabric Interconnect This chapter describes the physical installation and cabling of the Oracle Fabric Interconnect. Installation in a 4-post equipment rack and a standard Sun/Oracle 27” rack are described. This chapter contains the following sections: “Oracle Fabric Interconnect Overview” on page 11 ■...
The Oracle Fabric Interconnect F1-4 is a enterprise-class, 2 rack-unit Fabric Interconnect which provides server connectivity to network and storage resources. The Fabric Interconnect F1-4 supports up to 4 of the same 1 GE, 10 GE, and Fibre Channel modules as Oracle Fabric Interconnect F1-15 flagship product.
Site Preparation Site Preparation The Fabric Interconnect product family consists of the Fabric Interconnect F1-15 and the Fabric Interconnect F1-4. Before installing either system, make sure that the site meets the proper requirements. Fabric Interconnect F1-15 Site Criteria Verify that your installation site meets the following criteria for the Fabric Interconnect F1-15: Power: 100-127 / 200-240 VAC 12 A at 100 volts or 7.68 A at 180 volts 47- 63 Hz 1220 W ■...
Site Preparation The Oracle Fabric Interconnect has cable-management brackets that can be mounted Note - on the chassis as an option. It is your choice whether or not install the cable-management brackets. Service Clearance: 18 inches (45.7cm) (front and rear) ■...
Page 15
■ Client Servers: InfiniBand interface required ■ Select a location in the rack in which to install the Oracle Fabric Interconnect. The location must accept a chassis with the following dimensions, as shown in Figure 2, “Verifying Rack Space,” on page Height: 3 1/2 inches (8.9 cm)
If the shipping container is not damaged, you can unpack the Fabric Interconnect. See the appropriate section for your Fabric Interconnect: “Unpack the Fabric Interconnect F1-15” on page 17 ■ “Unpack the Fabric Interconnect F1-4” on page 19 ■ Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Note - deployments only. The Fabric Interconnect is orderable as a complete, engineered system called the Oracle Virtual Compute Appliance (VCA), which has different contents and installation procedures. If you are deploying a stand-alone (non-VCA) Fabric Interconnect, continue to use this documentation. If you are deploying an VCA Fabric Interconnect, do not use this documentation.
Page 18
37. However, to install I/O Modules, you must remove the module retention bracket. Unpack the Fabric Interconnect chassis and place it on a flat, stable surface capable of supporting up to 170 pounds (77.11kg). Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Unpack the Fabric Interconnect F1-4 Unpack the Fabric Interconnect F1-4 The Fabric Interconnect F1-4 is a 2 rack-unit Fabric Interconnect with capacity for 4 I/O modules and either 24 InfiniBand server connections at DDR speeds, or 20 InfiniBand server connections at QDR speeds. The Fabric Interconnect chassis weighs 45 - 53 pounds (20 - 24 kilograms) Caution - depending on its configuration.
Fabric Interconnect. The rail kit is useful as a “rack helper” to temporarily hold the chassis in place in the rack while attaching rack-mount hardware, Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Note - deployments only. The Fabric Interconnect is orderable as a complete, engineered system called the Oracle Virtual Compute Appliance (VCA), which has different contents and installation procedures. If you are deploying a stand-alone (non-VCA) Fabric Interconnect, continue to use this documentation. If you are deploying a VCA Fabric Interconnect, do not use this documentation.
7068910 Rev 03 or later. This number is silk- screened on the part. Otherwise, all the other hardware is the same. If you are installing the Fabric Interconnect in a 27” Sun Oracle rack, be aware that you will need to use this specific part.
Install the Fabric Interconnect F1-15 The Fabric Interconnect comes with a module retention bracket attached to the chassis. ■ Although you can rack install the Fabric Interconnect with the module retention bracket attached to the chassis, you will find it easier to remove the bracket before beginning the installation procedure.
Page 24
If you are using an untapped rack, for example a Sun/Oracle rack, insert the M6 cage nuts now if you have not already done so. Later in the procedure, you will then use M6 screws to secure the chassis to the rack.
Page 25
Install the Fabric Interconnect F1-15 Simultaneously pull the two pins inward (toward each other), and slide the module retention bracket off of the Fabric Interconnect chassis. See Figure 6, “Removing the Module Retention Bracket,” on page Removing the Module Retention Bracket FIGURE 6 The module retention bracket is used to hold modules and blanks in place while the Note -...
Page 26
25” pan-head screws (4 per rail) through the slotted rail piece into the piece with the threaded standoff. Do not completely tighten the screws. You want to leave some slack in the rails so that they can slide lengthwise to accommodate the Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 27
Install the Fabric Interconnect F1-15 depth of the rack. Figure 8, “Assembling the Rack Mount Rails,” on page 27 shows assembling the rack mount rails. Use short pan-head screws (10-32 x .25”) from the rail kit box. The rack mount rails have 3 slots. Figure 8, “Assembling the Rack Mount Rails,”...
Page 28
10-32 x .375” Phillips pan-head screws ■ 12-24 x .5” Phillips pan-head screws with lock washer already attached to the screw ■ M6 x 16 mm Phillips pan-head screws ■ Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 29
Install the Fabric Interconnect F1-15 Installing Assembled Rack Mount Rails into Equipment Rack FIGURE 9 When the rails are attached to the rack posts, tighten all screws that connect the pieces of the rail, as well as the screws that attach each assembled rail to the rack.
Page 30
The horizontal surface is required to support the Fabric Interconnect as it slides into the rack. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
4-post rack or a 27” Sun/Oracle rack: “Attach the Right Angle Bracket” on page 31 ■ “Attach the Right Angle Bracket in a 27” Sun/Oracle Rack” on page 35 ■ Attach the Right Angle Bracket Align the angle bracket with the screw holes on the outside of the side panels.
Page 32
Use eight (8) pan-head screws per side. Use the appropriate screw for your rack type: 10-32 x .375” Phillips pan-head screws ■ 12-24 x .5” Phillips pan-head screws with lock washers already attached to screws ■ M6 x 16 mm screws ■ Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 33
Attach the Right Angle Bracket If you are installing into a rack tapped for 12-24 screws, you must use the accompanying Note - #12 lock washers. For 12-24 racks, insert the screw into the lock washer before installing the screw into the rack. FIGURE 12 Attaching the Chassis to the Rack Posts When the chassis is completely installed, check all screws to make sure they are...
Page 34
It is not mandatory to remove the rail kit, but if you do, an extra 1U of vertical rack Note - space becomes available. As an option, you can install cable management assembly (CMA). If you want to install the CMA, see “Installing the Cable Management Assembly” on page Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Attach the Right Angle Bracket in a 27” Sun/Oracle Rack Attach the Right Angle Bracket in a 27” Sun/ Oracle Rack With the chassis on the rail kit, follow these steps to install the right angle brackets for a 27”...
Page 36
Rack Posts,” on page 36 shows attaching the chassis to the rack. The Oracle/Sun 27” rack is untapped, so you will need to use cage clips to attach Note - equipment. You will need to install the M6 cage nuts into the blanks in the rack, then use the M6 screws to secure the equipment into the rack.
Installing the Cable Management Assembly The cable management assembly shipped with the Fabric Interconnect F1-15 is the standard CMA used in Oracle 4-post equipment racks. The CMA consists of the following parts: two stand-off brackets, which attach to the equipment rack.
The type of screw that you will use depends on the type of rack you have. For example, if you have a Sun/Oracle rack, you will use the M6 cage nuts and the M6 screws to attach the CMA to the rack. Be aware that you will not use every type of screw, just the type appropriate for your equipment rack.
Page 39
Install the CMA Attachment Bracket sliding over the Long Rail FIGURE 16 Place the attachment plate on the flange side of the extender, opposite the attachment bracket. Use two M4 screws to sandwich the attachment bracket and plate to the extender, in the position farthest from the flange as shown in Figure 17, “Bracket and Plate Sandwiching to the Rail,”...
Page 40
Make sure that these rails are flush with the side of the Fabric Interconnect chassis, with the flange pointing outward as shown in Figure 18, “front brackets being installed,” on page Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 41
Install the CMA front brackets being installed FIGURE 18 Secure the assembly to the post with the correct screw type for the rack (10-32, 12-24, or M6) by inserting the screw through the slotted end of the CMA and into the rack or cage nut.
Page 42
Make sure not to exceed the allowed bend radius for any cables you route through the Note - comb. Also, wherever possible, make sure to drape equal numbers of cables in each slot to apply the cable weight equally along the CMA. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 43
Install the CMA Connecting Cables and Draping Them Through the Comb FIGURE 20 When all cables are connected, align the captive screws in the top of the CMA with the screw holes in the CMA's comb as shown in Figure 21, “Installing the Top of the CMA,”...
Page 44
Installing the Top of the CMA FIGURE 21 Using a Phillips-head screwdriver, tighten the screws to secure the plate to the comb as shown in Figure 22, “Completing Assembly of the CMA,” on page Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Installing the Fabric Interconnect F1-4 in a 4-Post Rack Completing Assembly of the CMA FIGURE 22 Installing the Fabric Interconnect F1-4 in a 4-Post Rack This section documents how to install the Fabric Interconnect F1-4 in a four-post equipment rack. The procedure in this section documents installing a Fabric Interconnect F1-4 by using the mounting hardware that comes with the Fabric Interconnect.
Note - deployments only. The Fabric Interconnect is orderable as a complete, engineered system called the Oracle Virtual Compute Appliance (VCA), which has different contents and installation procedures. If you are deploying a stand-alone (non-VCA) Fabric Interconnect, continue to use this documentation. If you are deploying an VCA Fabric Interconnect, do not use this documentation.
Page 47
Installing the Fabric Interconnect F1-4 in a 4-Post Rack Power Cord Anchor FIGURE 23 The power cord management system consists of: two (2) plastic anchors, one per mounting bracket. Each anchor has slots at the bottom and ■ top. two (2) tie wraps, one for each power cord. Each tie wrap is reusable and has a tab that ■...
Be aware of the following considerations before beginning the installation procedure. You will find it helpful to determine the depth of the rack to know where to attach the side ■ panels to the chassis. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Install the Fabric Interconnect F1-4 For a 27-inch rack (a standard Sun/Oracle rack), you need a separate mounting bracket for ■ the Oracle Fabric Interconnect F1-4. Additionally, the to-chassis mounting hardware for the Oracle Fabric Interconnect F1-4 is metric, the to-chassis mounting hardware for the Oracle Fabric Interconnect F1-15 is English.
Page 50
Get eight mounting screws for the Fabric Interconnect's mounting brackets, which are the M3.5 Phillips countersink screws. If you are using an untapped rack, for example a Sun/oracle rack, insert all the required M6 cage nuts now if you have not already done so.
Page 51
Install the Fabric Interconnect F1-4 Use four (4) M3.5 x .6 x 6 mm Phillips countersink screws per bracket. Figure 24, “Installing the Rack Depth Measures,” on page 51 shows installing the Note - rack-depth measures for one depth. Your rack depth might differ, so the rack-depth measures might need to slide forward or backward depending on the depth of your rack.
Page 52
Fabric Interconnect F1-4 into the correct location. Choose either of the following workflows based on the whether you have a standard 4-post rack or a 27” Sun/Oracle rack: “Attach the Right Angle Bracket” on page 31 ■ Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Attach the Right Angle Bracket “Attach the Right Angle Bracket in a 27” Sun/Oracle Rack” on page 35 ■ Attach the Right Angle Bracket Slide the Fabric Interconnect into the rack until the front mounting brackets make contact with the front rack posts. See Figure 26, “Attaching the Rack...
Page 54
Align the slots in the rack-depth rails with the screw holes in the right angle brackets, and insert the 6 M3.5 x .6 x 6 mm Phillips flat-head screws through the Oracle Fabric Interconnect Hardware User's Guide • July 2016...
“Installing the Cable Management Assembly” on page The same CMA is supported for both the Oracle Fabric Interconnect F1-15 and Oracle Fabric Interconnect F1-4, so use the same procedure. Where illustrations show the Oracle Fabric Interconnect F1-15, you will substitute the Oracle Fabric Interconnect F1-4.
Page 56
Insert the M6 pan-head rack screw through the slotted edge of the angle bracket, and into the chassis. The Sun/Oracle rack is untapped, so you should have already installed the M6 cage nuts into the rack. For the Sun/Oracle rack, use only the M6 screws to attach the chassis to the rack.
Page 57
“Installing the Cable Management Assembly” on page The same CMA is supported for both the Oracle Fabric Interconnect F1-15 and Oracle Fabric Interconnect F1-4, so use the same procedure. Where illustrations show the Oracle Fabric Interconnect F1-15, you will substitute the Oracle Fabric Interconnect F1-4.
Figure 30, “Fabric Interconnect F1-15 Front and Rear Panel Components,” on page 59 shows the Fabric Interconnect F1-15's chassis components on the front and rear panels. The following illustration shows a Fabric Interconnect F1-15 supporting DDR connectivity. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 59
Understanding Chassis Components and Indicators Fabric Interconnect F1-15 Front and Rear Panel Components FIGURE 30 The Fabric Interconnect F1-15 can also support InfiniBand connections at Quad Data Rate (QDR) speeds. Figure 31, “QDR Fabric Interconnect F1-15 Rear Panel Components,” on page shows the Fabric Interconnect F1-15's rear panel with a QDR Fabric Board installed.
Page 60
The front of a QDR Fabric Interconnect F1-4 has the same standard equipment—for example, power supplies, fans, and so on are the same between a QDR Fabric Interconnect F1-4 and DDR Fabric Interconnect F1-4. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Understanding Chassis Components and Indicators QDR Fabric Interconnect F1-4 Rear Panel Components FIGURE 33 Fabric Interconnect Control Panel The Fabric Interconnect control panel contains LEDs that provide indicators of the system's status as well as a power LED and power button for hard power on and off. Figure 34, “Fabric Interconnect F1-15 Control Panel,”...
For network connectivity, Gigabit Ethernet modules are supported in either 1 Gigabit per ■ second or 10 Gigabit per second bandwidths. Gigabit I/O Modules offer different numbers of ports for various connectivity options: Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 63
Understanding Chassis Components and Indicators 10 GE Module, which has one 10 Gbps Ethernet port. The 10 GE Module supports full ■ duplex 10 Gbps Ethernet traffic. The 10 GE module supports copper and fiber cables. This module is hot swappable and supports robust network software features. 10-Port GE Module, which has 10 individual ports.
Page 64
Figure 37, “Fabric Interconnect F1-4 I/O Modules and Indicators,” on page 64 shows I/O modules in a Fabric Interconnect F1-4. I/O Modules are installed horizontally in the rear of the Fabric Interconnect. Fabric Interconnect F1-4 I/O Modules and Indicators FIGURE 37 Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Understanding Chassis Components and Indicators 10 GE Ethernet I/O Module LED Codes Figure 38, “LED Codes — 10 Gigabit Ethernet Module,” on page 65 shows the 10 Gigabit Ethernet module LED codes. The 10 Gigabit Ethernet module is supported by all models of Fabric Interconnect F1-15 and Fabric Interconnect F1-4.
LED blinks yellow whenever traffic exists on the link. If the link is up but has no traffic activity, the LED glows steady yellow (not blinking). If the port is disabled, the link is down, or the port is not receiving power, the LED is dark. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 67
Understanding Chassis Components and Indicators The traffic activity LED (yellow LED) states are conditional on the presence of a Note - physical link. They are valid only if the Link LED is solid green, which indicates the port is enabled and link is present. If the port is not enabled and link is not present, traffic activity cannot occur, and as a result, the traffic activity LED will be dark.
Solid blue Solid green I/O card is down (power to card is off) Blinking blue Solid green I/O card is shutting down These are specific LED codes for each Fibre Channel: Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Installing an I/O Module Yellow Green Amber Description Power off Power on (before FC initialization) Flash Simultaneously Power on (after FC initialization) Flash in series: yellow, green, amber FW fault 1 Gb link up/active 2 Gb link up/active 4 Gb link up/active Fabric Interconnect Front Panel LED Codes Figure 42, “LED Codes —...
This procedure is supported by all models of Fabric Interconnect F1-15. When tightening screws, make sure to tighten them to lightly snug. Do not Caution - overtighten screws or they can break. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Modules. I/O module blanks also have the locking mechanism. When you are initially installing I/O Modules, make sure to use the lock and unlocked positions appropriately—for example, make sure that you lock the modules when you are done installing them. Oracle recommends that you keep all module blanks. Installing the Fabric Interconnect...
Page 72
Using a slotted screwdriver, unlock the I/O module blank by turning it 90 degrees counterclockwise (1) as shown in Figure 44, “Removing an I/O Module Blank,” on page Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 73
Install I/O Modules in a Fabric Interconnect F1‐4 Removing an I/O Module Blank FIGURE 44 Grasp the module handle and remove the I/O module blank (2). Get the I/O module you will be installing, and make sure that the I/O module is in the unlocked position.
Page 74
Using a slotted screwdriver, lock the I/O Module into the chassis (2) by turning the lock 90 degrees clockwise. Repeat this procedure as needed to remove all the I/O module blanks and install all the I/O modules. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Different servers have different methods for installing an HCA card. This section documents one HCA installation procedure for a common server. You can use the procedure as a general guideline for installing the HCA card, but Oracle recommends you also consult the documentation from the server's manufacturer for specific installation information.
Page 76
HCA's InfiniBand port is at the rear of the server chassis. In this position, the components on the HCA card will be facing down, and the HCA's InfiniBand port will be facing out of the rear of the chassis. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Client servers are connected to the Fabric Interconnect through the Fabric Interconnect's Fabric Board. The Fabric Interconnect has models which feature different Fabric Board options. Fabric Interconnects Oracle offers Fabric Interconnects with InfiniBand Fabric Boards: The Fabric Interconnect F1-15 with a QDR Fabric Board ■...
Page 78
InfiniBand interface (or optional HCA card) to one of the 24 server port on the Fabric Board. Additional cables may be ordered from Oracle. Figure 49, “InfiniBand Ports for Fabric Interconnect F1-15 Client Server Connections,” on page shows an InfiniBand port connection to a Fabric Interconnect F1-15 with a DDR Fabric Board.
Page 79
Connecting the Fabric Interconnect to Other Equipment InfiniBand Ports for Fabric Interconnect F1-15 Client Server Connections FIGURE 49 Figure 50, “InfiniBand Ports for Fabric Interconnect F1-4 Client Server Connections,” on page shows an InfiniBand port connection on the Fabric Interconnect F1-4 with a DDR Fabric Board.
Fibre Channel cables, as required for the Fabric Interconnect F1-15 or Fabric Interconnect F1-4. Figure 51, “Fabric Interconnect F1-15 Fibre Channel and Gigabit Ethernet Modules and Connectors,” on page 81 shows storage and network connections for a Fabric Interconnect F1-15. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 81
Connecting the Fabric Interconnect to Other Equipment Fabric Interconnect F1-15 Fibre Channel and Gigabit Ethernet Modules and Connectors FIGURE 51 Figure 51, “Fabric Interconnect F1-15 Fibre Channel and Gigabit Ethernet Modules and Connectors,” on page 81 shows an example of the 10-Port GE module, but the 10 GE module (which has one 10 GE port) is also supported.
Page 82
Connecting the Fabric Interconnect to Other Equipment Fabric Interconnect F1-4 Fibre Channel Module and Connectors FIGURE 52 Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 83
Connect the 4-Port 10 GE Module to Other Equipment Connect the 4-Port 10 GE Module to Other Equipment The 4-Port 10 GE module can be connected to network equipment through different methods, which are determined by the type of cables and connectors you ordered. If you are using SFP+ passive or active copper cables, the port transceivers are integrated ■...
Page 84
Inserting the Optical Transceiver into the 4-Port 10 GE Module FIGURE 53 Insert one end of the optical cable into the SFP+ transceiver. See Figure 54, “Inserting the Optical Cable into the Optical Transceiver,” on page Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 85
Connect the 4-Port 10 GE Module to Other Equipment Inserting the Optical Cable into the Optical Transceiver FIGURE 54 Connect the other end of the cable to a 10 Gbps port on a peer device, such as a 10 Gbps Ethernet switch. Check the Link LED on the 4-Port 10 GE module's port to make sure that the port detects the cable.
This section documents the Management module and its port connections. Some ports on the Management module are inactive, so you should not use them. When the Fabric Interconnect is shipped from the factory, inactive ports are capped with a black rubber plug. Oracle recommends that you leave these plugs in place.
Page 87
Connect to a Fabric Interconnect F1-4's Management Module Fabric Interconnect F1-15 Management Module for Connection to Management FIGURE 55 Console There are two ways to connect to the serial port on the Fabric Interconnect Management module: Connect directly to the Serial 1 port on the Management module. This ■...
Page 88
Connect a remote system to the Serial 1 port using a serial console terminal ■ concentrator (for example, Cyclades). Verify that the serial port on your system is set up as follows: 115200 baud ■ 8 bits, no parity ■ VT100 encoding ■ Oracle Fabric Interconnect Hardware User's Guide • July 2016...
The Serial 2 port is used for debug purposes only. Note - AC Power Your system is shipped with power cords that are suitable for your location. However, Oracle recommends that you inspect the cords to ensure that they are of the correct type. See the appropriate section: “Connect Power to the Fabric Interconnect F1-15”...
Page 90
Connect Power for the Fabric Interconnect F1-4 Fabric Interconnect F1-15 Power Supplies for Connecting to AC Facility Power FIGURE 57 On the opposite side of Oracle Fabric Interconnect, simultaneously switch all power supplies to the “On” position. Connect Power for the Fabric Interconnect F1-4 The Fabric Interconnect F1-4's power supplies and power plugs are located on the front of the chassis.
Page 91
Connect Power for the Fabric Interconnect F1-4 Get the two cable tie wraps from the accessory kit. Notice that one side is smooth and one side has teeth. You will find it helpful to orient the cable tie as follows: - the smooth side is facing the Note - chassis and the ribbed side is facing you.
Page 92
Connect the dual power supplies to a facility AC power source. The Fabric Interconnect F1-4's power supplies do not have a power switch, so the Fabric Interconnect F1-4 will power up when the power cord is connected to facility power. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 93
Connect Power for the Fabric Interconnect F1-4 When the Oracle Fabric Interconnect F1-4 has I/O modules and is cabled to facility power, network, storage, and server resources, you can power it on and perform initial configuration through the Oracle Configuration Wizard. See “Performing Initial Configuration”...
Interconnect. The setup procedure is offered as an example of a typical, basic configuration and is not intended to be a comprehensive reference. This chapter documents the following topics: Log in to the Oracle Fabric Interconnect through the serial port and perform the initial ■ system configuration using the Oracle Configuration Wizard. See “Run the Oracle...
■ information to Oracle Technical Assistance for proactive system monitoring. By default, the wizard sets this to “true” so that diagnostic information is sent to Oracle Technical Assistance. In the Oracle First Boot script, the diagnostic information is controlled through...
Page 97
Would you like to use the Oracle Configuration Wizard? [Y/n] y Hostname: ovn123 Domain: oracle.com (or blank)Is this Interconnect to be designated as the IB subnet manager? [Y/n] y Do you want this Interconnect to send diagnostic data to Oracle periodically? [Y/n] y...
When you have completed the configuration, the Oracle Configuration Wizard closes and returns you to the admin@<system_name> prompt. Change the Root Password When you have completed the Oracle Configuration Wizard, log in to the Fabric Interconnect, and change the root password: Log in to the Fabric Interconnect as root by entering:...
Management module for Oracle Fabric Interconnect F1-4 is not. Also, with Oracle XgOS version 3.6.0 and later, the USB drive on the Management Module is now disabled by default for compliance with security mandates in data centers that require high security.
Compatibility of Fabric Interconnect F1‐15 and F1-4 Hardware Oracle has designed the Fabric Interconnect F1-4 with forward compatibility in mind. As a result, the I/O Modules have been designed for interchangeability between the Fabric Interconnect F1-15 and the Fabric Interconnect F1-4: 10 GE modules ■...
The ID indicator LED (ID LED) on the Fabric Interconnect front panel provides a visual indicator of the Fabric Interconnect if it needs service. You can turn on the ID LED to allow a Oracle-certified technician to find the Fabric Interconnect that needs service. See Figure 60, “ID Button and ID LED Location,”...
Page 102
Activate and Deactivate the ID LED Log in to the Oracle Fabric Interconnect that needs service. Activate the ID LED by issuing the set system id-led command. For example: set system id-led on When you issue this command, the amber ID LED illuminates. The service technician should be able to locate the Fabric Interconnect by looking for Fabric Interconnect with the glowing amber LED.
Removing Fan Modules httpproxy-port 3128 phone-home-frequency weekly ------------------------------------------------------------------------------- 1 record displayed In this example, the bold text highlights the current state of the ID LED. True indicates that the LED is on (illuminated), and false indicates that the LED is off (dark). If the LED is on (the locator-led-on parameter shows true), deactivate it when the service call is completed.
Page 104
Grasp the handle and slide the fan module out of the chassis. See Figure 62, “Removing a Fabric Interconnect F1-15 Fan Module,” on page 104. FIGURE 62 Removing a Fabric Interconnect F1-15 Fan Module Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Remove Fabric Interconnect F1-4 Fan Modules Installation is the reverse of this procedure. Remove Fabric Interconnect F1-4 Fan Modules Fabric Interconnect F1-4 fan modules are located on the front of the chassis. To remove a Fabric Interconnect F1-4 fan module, follow this procedure: Use a Phillips screwdriver to release the two spring-loaded securing screws located at the top corners of the each fan module (1).
PSU at a time without powering down the Fabric Interconnect. For this procedure, you will need a #2 Phillips (cross-head) screwdriver. When tightening screws, make sure to tighten them to lightly snug. Do not Caution - overtighten screws or they can break. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Remove Fabric Interconnect F1-15 Power Supplies When replacing the Power Supplies: - wear an ESD wrist strap - When the hardware Caution - is out of its shipping package or not in the Fabric Interconnect chassis, make sure you place it in an ESD safe bag or on a grounded work surface such as a rubberized work mat.
Fabric Interconnect F1-4 power cords have their own cable management system which anchors them to a pre-installed part that sits on the front mounting brackets. The Fabric Interconnect F1-4 power cord management system consists of: two (2) plastic anchors, one per side ■ Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 109
The Fabric Interconnect F1-4 can operate fully on one PSU. However, if both PSUs must be removed at the same time, the system will shut down due to having no power. In cases where both PSUs must be removed, Oracle recommends swapping one power supply at a time (if possible) to avoid service interruptions.
Page 110
FIGURE 67 Removing a Fabric Interconnect F1-4 PSU Both PSUs are oriented in the same direction, so both locking tabs pull to the right. Note - Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Install Fabric Interconnect F1-4 Power Supplies Install Fabric Interconnect F1-4 Power Supplies When a PSU is disconnected, you must wait for at least 30 seconds before re-inserting Note - it. This 30-second wait period is required to properly raise alarms and update the PSU part number, serial number, and other pertinent information for the system's hardware inventory.
“Upgrading a DDR Fabric Board to a QDR Fabric Board” on page 128 ■ “Upgrade a Fabric Interconnect F1-15 to a QDR Fabric Board” on page 129 ■ “Upgrade a Fabric Interconnect F1-4 to a QDR Fabric Board” on page 134 ■ Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Replace the Fabric Interconnect F1-15 DDR Fabric Board Replace the Fabric Interconnect F1-15 DDR Fabric Board To replace the Fabric Interconnect F1-15 fabric board, follow this procedure: Power down the Fabric Interconnect. Disconnect all cables attached to the server ports, as well as any other cables that could interfere with the Fabric Board's removal.
To replace the Fabric Interconnect F1-4 Fabric Board, you will need a #2 Phillips-head screwdriver. To remove the Fabric Interconnect F1-4 Fabric Board, follow this procedure: Power down the Fabric Interconnect. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 115
Replace the Fabric Interconnect F1-4 DDR Fabric Board Disconnect all cables attached to the server ports, as well as any other cables that could interfere with the Fabric Board's removal. Using a screwdriver, loosen the captive screws in the release levers by turning them counterclockwise (1).
Removing the Fabric Interconnect F1-4 Fabric Board Insert a Fabric Interconnect F1-4 DDR Fabric Board Orient the Fabric Board so that the InfiniBand server ports are facing you and the sheet metal side is facing down. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 117
Insert a Fabric Interconnect F1-4 DDR Fabric Board Make sure that the Fabric Board handles are in the open position, not flat against the faceplate. Align the Fabric Board with the empty slot, and gently slide it into the chassis, making sure to support the Fabric Board from underneath (the sheet metal side).
“Upgrading a DDR Fabric Board to a QDR Fabric Board” on page 128. To replace the Fabric Interconnect F1-15 QDR Fabric Board, follow this procedure: Power down the Fabric Interconnect. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 119
Replace the Fabric Interconnect F1-15 QDR Fabric Board Disconnect all cables attached to the server ports, as well as any other cables that could interfere with the Fabric Board's removal. Using a screwdriver, loosen the captive screws in the release levers by turning them counterclockwise (1).
Page 120
Supporting the QDR Fabric Board under the sheet metal bottom, hold the Fabric Board level and align it with the empty Fabric Board bay in the Fabric Interconnect. Figure 77, “Inserting the QDR Fabric Board into a Fabric Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 121
Replace the Fabric Interconnect F1-15 QDR Fabric Board Interconnect F1-15,” on page 121 shows aligning and inserting the QDR Fabric Board. Caution - Pay attention to the leading edges of the Fabric Board—the edges that will contact and insert into the chassis first. These edges must be kept horizontal. If you tip one side of the Fabric Board, it possible to insert the Fabric Board incorrectly.
Page 122
(2). When tightening screws, make sure to tighten them to lightly snug. Do not Caution - overtighten screws or they can break. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Replace the Fabric Interconnect F1-4 QDR Fabric Board Reconnect the Fabric Interconnect to facility power, then power on the Fabric Interconnect, and allow it to completely boot up. The Fabric Board will boot, and when the board completes is bootup and passes to runtime, the Power LED will glow.
Page 124
Fabric Board out of the chassis, making sure to support the Fabric Board from underneath (the sheet metal side). See Figure 80, “Removing the Fabric Interconnect F1-4 Fabric Board,” on page 125. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 125
Replace the Fabric Interconnect F1-4 QDR Fabric Board Removing the Fabric Interconnect F1-4 Fabric Board FIGURE 80 Remove the QDR Fabric Board from the packaging, and orient the module so that it is right side up. Orient the QDR Fabric Board so that the sheet metal surface is on the bottom, and pivot the QDR Fabric Board's handles so that they are pointing directly towards you—not flush with the module.
Page 126
Fabric board level while you slide it in. When the notches and guides are properly positioned, press the Fabric Board's handles inward to seat the Fabric Board (1) as shown in Figure 82, “Seating a Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 127
Replace the Fabric Interconnect F1-4 QDR Fabric Board QDR Fabric Board into a Fabric Interconnect F1-4,” on page 127. The Fabric Board is properly seated when it and its handles lie flush against the chassis. FIGURE 82 Seating a QDR Fabric Board into a Fabric Interconnect F1-4 Do not force the levers.
ESD-safe workspace for the Fabric Boards when they are not in the Fabric Interconnect. ■ The ESD bag that ships with Oracle modules is an option if a grounded workspace is not available. Oracle recommends reading this entire procedure to get an overview of it before Note - performing the hardware upgrade.
Fabric Board you will be replacing was using DDR cables. * Bringing the Fabric Interconnect with the new QDR Fabric Board online in a data center managed by Oracle Fabric Manager. These additional considerations are addressed in the QDR Fabric Board Upgrade Guide, Release XgOS 3.0.0.
Page 130
Figure 84, “Removing the DDR Fabric Board from a Fabric Interconnect F1-15,” on page 131 shows removing the Fabric Board. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 131
Upgrade a Fabric Interconnect F1-15 to a QDR Fabric Board Removing the DDR Fabric Board from a Fabric Interconnect F1-15 FIGURE 84 Set the DDR Fabric Board on an ESD-safe work surface or in a ESD bag. Remove the QDR Fabric Board from the packaging, and orient the module so that it is right side up.
Page 132
Do not force the Fabric Board into place. If it does not slide in easily, remove it and Caution - attempt to insert it again. Make sure that the leading edges are level, and you are keeping the Fabric Board level while you slide it in. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 133
Upgrade a Fabric Interconnect F1-15 to a QDR Fabric Board When the notches and guides are properly positioned, press the Fabric Board's handles inward to seat the Fabric Board (1) as shown in Figure 86, “Seating the QDR Fabric Board into a Fabric Interconnect F1-15,” on page 133.
You do not need to return the old Fabric Board. You can keep it as a spare part. But if you choose to return the old Fabric Board, Oracle can dispose of it. If you choose to return the old Fabric Board, you can use the shipping container for the QDR Fabric Board to return the DDR Fabric Board.
Page 135
Upgrade a Fabric Interconnect F1-4 to a QDR Fabric Board first Fabric Interconnect. When the first Fabric Interconnect boots up with a QDR Fabric Board, move all traffic off of the second Fabric Interconnect and upgrade that Fabric Interconnect to a QDR Fabric board. In a single Fabric Interconnect environment, stop all traffic going through ■...
Page 136
At the far end of each of the Fabric Board's handles, notice there is a notch. This notch will meet a part in the chassis, and allow you to lock the Fabric Board into the system midplane and power channel. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 137
Upgrade a Fabric Interconnect F1-4 to a QDR Fabric Board Supporting the QDR Fabric Board under the sheet metal bottom, hold the Fabric Board level and align it with the empty Fabric Board bay in the Fabric Interconnect. Figure 89, “Inserting a QDR Fabric Board into a Fabric Interconnect F1-4,”...
Page 138
When tightening screws, make sure to tighten them to lightly snug. Do not overtighten screws or they can break. Reconnect the Fabric Interconnect to facility power, then power on the Fabric Interconnect, and allow it to completely boot up. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
You do not need to return the old Fabric Board. You can keep it as a spare part. But if you choose to return the old Fabric Board, Oracle can dispose of it. If you choose to return the old Fabric Board, you can use the shipping container for the QDR Fabric Board to return the DDR Fabric Board.
Using the screwdriver, turn the locking handle (1) located at the bottom edge of each module 90 degrees counterclockwise. See Figure 91, “Fabric Interconnect F1-15 I/O Module Locking Handle,” on page 141. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Install Fabric Interconnect F1-15 I/O Modules Fabric Interconnect F1-15 I/O Module Locking Handle FIGURE 91 Grasp the locking handle (2) and slide the module out of the chassis. Install Fabric Interconnect F1-15 I/O Modules When re-installing the I/O modules, ensure that the module's sheet metal side is facing Note - the left edge of the slot.
While performing any hardware procedures, including swapping or upgrading a module, Note - make sure to observe proper ESD precautions. To upgrade to a 4-Port 10 GE Module follow this procedure: Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 143
Upgrade to a 4-Port 10 GE Module With the previous module installed, set each physical interface supporting a vNIC to none. For example: set vnic <vnic-name> -if=none set vnic <vnic-name> -if=none set vnic <vnic-name> -if=none set vnic <vnic-name> -if=none When all vNIC interfaces are set to none, turn off the I/O Module. For example: set ethernet-card 4 down Disconnect cables from the 10-Port 1 GE module.
Before removing an I/O module, use the XgOS to shutdown the I/O module. Using a slotted screwdriver, unlock the I/O module by turning it 90 degrees counterclockwise. See (1) in Figure 93, “Removing a Fabric Interconnect F1-4 I/O Module,” on page 145. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 145
Remove Fabric Interconnect F1-4 I/O Modules Removing a Fabric Interconnect F1-4 I/O Module FIGURE 93 Grasp the module handle and remove the I/O module, while keeping the module level (2) as shown in Figure 93, “Removing a Fabric Interconnect F1-4 I/O Module,”...
When the I/O Module is almost completely inserted, you will feel some resistance, which is the I/O Module's connector meeting its midplane socket. Continue to apply gentle pressure until the connector is seated in the socket. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Removing and Installing the Management Module When the module is completely inserted, it should lie flush with the face of the Fabric Interconnect F1-4. Using a slotted screwdriver, lock the I/O Module into the chassis by turning the lock 90 degrees clockwise (2) as shown in Figure 94, “Installing a Fabric Interconnect F1-4 I/O Module,”...
Slide the module into the slot until you feel the connectors begin to seat. Carefully, but firmly, press the Management module into the slot until the module's faceplate is flush with those of the other I/O modules. Tighten the screw. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Some notes about disabling the USB drive: Upgrading to Oracle XgOS 3.6.0 or later from a previous version of software does not ■ cause the USB to be deactivated. You must explicitly disable the USB drive by using the procedure in this section.
Page 150
If the USB device is not active and you need to activate it, you can use this procedure Note - and select Enable USB Function to re-enable the USB drive. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
■ Remove and Install a Fabric Interconnect F1-15 Front Panel Assembly To remove a front panel assembly from Oracle Fabric Interconnect F1-15, follow this procedure: Power down the Fabric Interconnect. Using the screwdriver, release the two spring-loaded screws (1) securing the release levers to the chassis.
Page 152
(2), then use the levers to carefully slide the assembly out of the chassis (3). See Figure 97, “Removing the Fabric Interconnect F1-15 Front Panel Assembly,” on page 153. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Remove a Fabric Interconnect F1-4 Front Panel Assembly Removing the Fabric Interconnect F1-15 Front Panel Assembly FIGURE 97 Remove a Fabric Interconnect F1-4 Front Panel Assembly To remove a Fabric Interconnect F1-4 front panel assembly, you will need a #2 Phillips-head screwdriver.
Page 154
Panel disengage from the chassis. Holding the lever arms, apply even pressure to both handles and slide the Front Panel towards you (3) until it is completely removed from the Fabric Interconnect F1-4. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Install a Fabric Interconnect F1-4 Front Panel Assembly Install a Fabric Interconnect F1-4 Front Panel Assembly Orient the Front Panel so that the sheet metal side is facing up, and align the left and right sides of the Front Panel with their guides on the left and right sides of the chassis.
“Uncable the Oracle Fabric Interconnect” on page 157 ■ “Prepare for the Procedure” on page 158 ■ “Remove the Oracle Fabric Interconnect from a Four-Post Equipment Rack” on page 158 ■ “Remove the Oracle Fabric Interconnect from a 27-Inch Sun/Oracle Equipment ■...
Oracle Fabric Interconnect to replace the old mounting brackets with the new ones. As part of unracking the Oracle Fabric Interconnect, you will need to uncable the system ■ and remove facility power. Consequently, you will need to schedule a service window to accommodate a service interruption.
Also, you will find it helpful to have a cart or sturdy table nearby on which you can place the Oracle Fabric Interconnect when it is remove from the equipment rack. Always follow safety and safe-handling considerations and prerequisites for the procedure.
Page 159
Remove the Oracle Fabric Interconnect from a 27-Inch Sun/Oracle Equipment Rack You will find it helpful to have a cart or sturdy table nearby on which you can place the Oracle Fabric Interconnect when it is removed from the equipment rack.
To gain access to the old mounting brackets, which need to be removed, you must disconnect the old front panel. Disconnect the existing front panel as needed for your model of Oracle Fabric Interconnect: “Remove and Install a Fabric Interconnect F1-15 Front Panel Assembly” on page 151.
Remove the Old Mounting Brackets Remove the Old Mounting Brackets Remove the screws that attach the old mounting bracket to the chassis. Remove the old mounting brackets. Performing Service Tasks...
With the old front panel and old mounting brackets removed, you can now begin installing the new parts. You can differentiate the old and new mounting brackets by the presence of a threaded standoff on the new mounting bracket's face. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
Page 163
Install the New Mounting Brackets Align the new mounting brackets over the screw holes in the chassis. Performing Service Tasks...
Make sure that the ejector levers are fully extended so that they are pointing towards you. Gently slide the FP-2 into the chassis, making sure to apply even pressure so that the FP-2 does not twist or rotate while you are inserting it. Oracle Fabric Interconnect Hardware User's Guide • July 2016...
“Install the Oracle Fabric Interconnect” on page 165. Install the Oracle Fabric Interconnect After rack installation, the Oracle Fabric Interconnect you will need to return to this procedure and see “Return the Oracle Fabric Interconnect to Service” on page 166.
For an Oracle Fabric Interconnect F1-15 in a four-post rack or a 27-inch Sun/Oracle rack, ■ “Installing the Fabric Interconnect F1-15 in a 4-Post Rack” on page For an Oracle Fabric Interconnect F1-4 in a four -post rack or a 27-inch Sun/Oracle rack, ■ see, “Install the Fabric Interconnect F1-4”...
Page 167
Return the Oracle Fabric Interconnect to Service Supported transceivers and cables are available from Oracle or a third-party. If you will be purchasing transceivers and cables from a third party vendor, make sure to order one of the supported types listed in Table 2, “...
Glossary ACL (Access A collection of rule sets, rules, conditions, and actions that are compared against network Control List) traffic to determine if an action should be taken on the traffic. ACL action A single action that can be configured in an ACL rule. The action is the enforced result after network traffic is successfully matched against one or more ACL conditions.
Page 170
Fabric Interconnect InfiniBand Fabric board. I/O Slot A single slot in the Oracle Fabric Interconnect chassis. A slot can be populated with an I/O card, pre-provisioned for a specific I/O card, or not populated with a module.
Page 171
vNIC User An internal or external representation of a person. Users either exist locally or remotely through LDAP, Active Directory, or RADIUS. By default, an “admin” user is created locally. vHBA Virtual Host Bus Adapter, which is a Fibre Channel storage connection, provided without a physical HBA.
Need help?
Do you have a question about the Fabric Interconnect Series and is the answer not in the manual?
Questions and answers