Related Manuals for Oracle Exalogic EL X2-2
Summary of Contents for Oracle Exalogic EL X2-2
- Page 1 Oracle® Exalogic Elastic Cloud Multirack Cabling Guide Release EL X2-2, X3-2, X4-2, X5-2, and X6-2 E89294-01 August 2017...
- Page 2 Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.
-
Page 3: Table Of Contents
Contents Preface Audience Revision History Documentation Accessibility Related Documents Conventions Multirack Cabling Overview About the Multirack Cabling Service Multirack Cabling Scenarios Port Usage in InfiniBand Switches 1.3.1 Port Usage in NM2-GW 1.3.2 Port Usage in NM2-36P Multirack Cabling Tasks Safety Considerations Equipment Safety Precautions InfiniBand Cable Cautions 2.2.1... - Page 4 Multirack Cabling Tasks Shutting Down the Affected Exalogic Machines and Exadata Database Machines Handling Inter-Switch Cables Extracting Spare InfiniBand Cables Unbundling the Inter-Switch Cables Applying Cable and Rack Labels Connecting the Cables Bundling the Cables Inspecting the Cables Verifying the Multirack Systems Connecting Two Exalogic Machines Before You Begin Connecting Two Exalogic Machine Full Racks Together...
- Page 5 6.2.2.6 Connecting Leaf Switches of Exadata Database Machine to Spine Switch of Exalogic Machine Connecting an Exalogic Machine Quarter Rack and an Exadata Database Machine Quarter Rack Multirack Cabling Diagrams for Exalogic-to-Exalogic Scenarios Connecting Two Exalogic Eighth Racks Connecting an Exalogic Eighth Rack and Exalogic Quarter Rack Connecting an Exalogic Eighth Rack and Exalogic Half Rack Connecting an Exalogic Eighth Rack and Exalogic Full Rack Connecting Two Exalogic Quarter Racks...
- Page 7 List of Figures Port Usage/Connection Groups for NM2-GW in Multirack Scenarios Port Usage/Connection Groups for NM2-36P in Multirack Scenarios Connectivity Between Two Exalogic Machine Full Racks in a Multirack Scenario Connectivity Between Exalogic Machine Full Rack and Exadata Database Machine Full Rack in a Multirack Scenario Two Exalogic Eighth Racks Exalogic Eighth Rack and Exalogic Quarter Rack...
- Page 8 viii...
- Page 9 List of Tables Remove Factory-Installed Cross-Connects in Exalogic Machines Connect 3-Meter Cables from Each Leaf Switch to the Spine Switch Connect 5-Meter Cable One Rack's Leaf Switches to the Second Rack's Spine Switch Factory-Installed Cross-Connects in Exalogic Machine to be Removed Factory-Installed Cross-Connects in Exadata Database Machine to be Removed Connect 3-Meter Cables from Each Leaf Switch to the Spine Switch in Exalogic Machine Full Rack...
-
Page 11: Preface
Note: All hardware-related specifications in this guide are based on information for a typical deployment provided by Oracle at the time this guide was written. Oracle is not responsible for hardware problems that may result from following the average specifications in this document. For detailed information about preparing your site for multirack cabling and setup, consult your Oracle representative. -
Page 12: Documentation Accessibility
For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup? ctx=acc&id=docacc Access to Oracle Support Oracle customers that have purchased support have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/ or visit lookup?ctx=acc&id=info http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs... -
Page 13: Multirack Cabling Overview
Multirack Cabling Overview This chapter describes the Exalogic Machine multirack cabling service. It contains the following sections: • About the Multirack Cabling Service • Multirack Cabling Scenarios • Port Usage in InfiniBand Switches • Multirack Cabling Tasks 1.1 About the Multirack Cabling Service The multirack cabling service allows you to extend the capability of the Exalogic rack systems to beyond just a single rack. -
Page 14: Port Usage In Infiniband Switches
Oracle Support. For detailed information about the hardware components of the Exalogic machine and the rack layout, see the Oracle Exalogic Elastic Cloud Machine Owner's Guide. 1.3 Port Usage in InfiniBand Switches All configurations of the Exalogic machine (full, half, quarter, and eighth rack) include Sun Network QDR InfiniBand Gateway Switches. -
Page 15: Multirack Cabling Tasks
Procedures in this guide assume that the racks are next to each other. Before you begin installing an Exalogic machine, perform pre–installation planning. Complete the checklists in the Oracle Exalogic Elastic Cloud Machine Owner's Guide. Verify the site space, access route, power, grounding, air flow, and network requirements that are provided in the Oracle Exalogic Elastic Cloud Machine Owner's Guide. - Page 16 Chapter 1 Multirack Cabling Tasks • Perform the cabling tasks: – Shutting Down the Affected Exalogic Machines and Exadata Database Machines – Handling Inter-Switch Cables – Extracting Spare InfiniBand Cables – Unbundling the Inter-Switch Cables – Applying Cable and Rack Labels –...
-
Page 17: Safety Considerations
Safety Considerations This chapter describes important safety considerations that you should review before Exalogic machine cabling begins. The following topics are covered: • Equipment Safety Precautions • InfiniBand Cable Cautions • Preparing the Cabling Site • Observing the Safety Guidelines and Warnings 2.1 Equipment Safety Precautions Observe the following precautions when cabling Exalogic machines. -
Page 18: Infiniband Cable Path Lengths
Note - Overhead cabling details are not included in this guide. For details on overhead cabling, contact a certified service engineer. 2.3 Preparing the Cabling Site It is the customer's responsibility to prepare the site prior to multirack cabling of Exalogic machines. Refer to the Oracle Exalogic Elastic Cloud Machine Owner's... -
Page 19: Observing The Safety Guidelines And Warnings
Guide. Go to http://docs.sun.com to download these guides. Note: Contact your service representative to confirm that your equipment is qualified for installation and use in an Exalogic machine. Oracle is not liable for any issues when you install or use non-qualified equipment. - Page 20 Chapter 2 Observing the Safety Guidelines and Warnings...
-
Page 21: Preparing For Multirack Cabling
For example, U24 indicates rack unit 24. U1 is located at the bottom of the rack, and U42 is at the top of the rack. For more information about Exalogic machine rack layout, see the Oracle Exalogic Elastic Cloud Machine Owner's Guide. -
Page 22: Performing A Cable Inventory
Chapter 3 Performing a Cable Inventory 3.3 Performing a Cable Inventory Collect and inventory the cables you ordered, if any. You should inventory spare cables from the kit and the existing multirack cables. Each Exalogic machine includes a kit of spare parts. Some spare cables are used for multirack cabling. -
Page 23: Multirack Cabling Tasks
Before you can set up the multirack cabling, you must first shut down the affected Exalogic machines and Exadata Database Machines. Consider the operational impact of shutting down the systems, and take appropriate action. Refer to the Oracle Exalogic Elastic Cloud Machine Owner's Guide and the Exadata Database Machine documentation for shutdown procedures. -
Page 24: Unbundling The Inter-Switch Cables
Chapter 4 Unbundling the Inter-Switch Cables 4.4 Unbundling the Inter-Switch Cables Unbundle the inter-switch cables. Remove the cable ties holding the cables together, such as hook-and-loop fasteners. Lay the cables out gently in collections of like length. Observe safety precautions. Be careful when handling the cables to avoid cable damage. -
Page 25: Verifying The Multirack Systems
In addition, Oracle recommends that you verify the InfiniBand topology after connecting multiple racks. For information about verifying the InfiniBand fabric and discovering the topology, see the Oracle Exalogic Elastic Cloud Machine Owner's Guide and the Oracle Exadata Database Machine... - Page 26 Chapter 4 Verifying the Multirack Systems...
-
Page 27: Connecting Two Exalogic Machines
Connecting Two Exalogic Machines This chapter describes how to connect two Exalogic machines on the same InfiniBand fabric. It includes the following topics: • Before You Begin • Connecting Two Exalogic Machine Full Racks Together 5.1 Before You Begin Before you start connecting multiple Exalogic machines together, ensure that you have completed all of the multirack cabling tasks described in Multirack Cabling Tasks. -
Page 28: Connect Two Exalogic Machine Full Racks Together
Chapter 5 Connecting Two Exalogic Machine Full Racks Together Figure 5-1 Connectivity Between Two Exalogic Machine Full Racks in a Multirack Scenario Sun Network QDR InfiniBand Gateway Switch (NM2-GW) 4x4 (four switches in each Exalogic machine full rack) cross-connects are recabled to their respective local spine switches Sun Datacenter InfiniBand Switch 36 (NM2-36P). -
Page 29: Remove The Factory-Installed Cables
For example, U24 indicates rack unit 24. U1 is located at the bottom of the rack, and U42 is at the top of the rack. For more information about Exalogic machine rack layout, see the Oracle Exalogic Elastic Cloud Machine Owner's Guide. -
Page 30: Remove Cables Between Gateway Switches
Chapter 5 Connecting Two Exalogic Machine Full Racks Together 5.2.2.2 Remove Cables Between Gateway Switches You must remove the cables connected in ports 4A, 4B, 5A, 5B, 6A, and 6B between gateway switches in both the racks. 5.2.2.3 Connect Each Leaf Switch to the Spine Switch Table 5-2 lists the 3-meter cables that should be used to connect leaf switches (NM2- GW) to the spine switch (NM2-36P) in the Exalogic machine rack. -
Page 31: Connect Each Leaf Switch In One Exalogic Rack To Spine Switches In The Other Exalogic Rack
Chapter 5 Connecting Two Exalogic Machine Full Racks Together 5.2.2.4 Connect Each Leaf Switch in One Exalogic Rack to Spine Switches in the Other Exalogic Rack Table 5-3 lists the 5-meter cables (or longer) that should be used to connect leaf switches (NM2-GW) from each Exalogic machine full rack to the spine switches (NM2-36P) in the other Exalogic machine full rack. - Page 32 Chapter 5 Connecting Two Exalogic Machine Full Racks Together...
-
Page 33: Connecting Exalogic And Exadata Machines
Connecting Exalogic and Exadata Machines This chapter describes how to connect Exalogic machine racks and Exadata Database Machine racks on the same InfiniBand fabric. It includes the following topics: • Before You Begin • Connecting an Exalogic Machine Full Rack and an Exadata Database Machine Full Rack •... -
Page 34: Connecting An Exalogic Machine Full Rack And An Exadata Database Machine Full Rack
Chapter 6 Connecting an Exalogic Machine Full Rack and an Exadata Database Machine Full Rack Figure 6-1 Connectivity Between Exalogic Machine Full Rack and Exadata Database Machine Full Rack in a Multirack Scenario Each Sun Network QDR InfiniBand Gateway Switch (NM2-GW) in the Exalogic machine full rack is connected to its local spine switch (Sun Datacenter InfiniBand Switch 36 (NM2-36P). -
Page 35: Removing Factory-Installed Cables (Cross-Connects) In Exalogic Machine Full Rack
U24 indicates rack unit 24. U1 is located at the bottom of the rack, and U42 is at the top of the rack. For more information about Exalogic machine rack layout, see the Oracle Exalogic Elastic Cloud Machine Owner's Guide. Table 6-1... -
Page 36: Removing Factory-Installed Cables (Cross-Connects) In Exadata Database Machine Full Rack
Chapter 6 Connecting an Exalogic Machine Full Rack and an Exadata Database Machine Full Rack Table 6-1 (Cont.) Factory-Installed Cross-Connects in Exalogic Machine to be Removed Cable Coming from Port Connecting to Slot Port Length Slot (Exalogic (Exalogic Machine) Machine) R1-U23 R1-U26 1 meter... -
Page 37: Connecting Each Leaf Switch In Exalogic Machine Full Rack To Its Spine Switch
Chapter 6 Connecting an Exalogic Machine Full Rack and an Exadata Database Machine Full Rack Note: R2 in Table 6-2 indicates an Exadata Database Machine full rack that is being connected to an Exalogic machine full rack (R1). 6.2.2.3 Connecting Each Leaf Switch in Exalogic Machine Full Rack to Its Spine Switch Table 6-3 lists the 3-meter cables that should be used to connect leaf switches (NM2-... -
Page 38: Connecting Each Leaf Switch In Exadata Database Machine Full Rack To Its Spine Switch
Chapter 6 Connecting an Exalogic Machine Full Rack and an Exadata Database Machine Full Rack Note: R1 in Table 6-3 indicates an Exalogic machine full rack that is being connected to an Exadata Database Machine full rack. 6.2.2.4 Connecting Each Leaf Switch in Exadata Database Machine Full Rack to Its Spine Switch Table 6-4 lists the 3-meter cables that should be used to connect leaf switches to the... -
Page 39: Connecting Leaf Switches Of Exadata Database Machine To Spine
Chapter 6 Connecting an Exalogic Machine Full Rack and an Exadata Database Machine Full Rack Table 6-5 Connect 5-Meter Cables from Each Leaf Switch to the Spine Switch in Exalogic Machine Full Rack Cable Coming from Port Connecting to Slot Port Length Slot (Exalogic... -
Page 40: Connecting An Exalogic Machine Quarter Rack And An Exadata Database Machine Quarter Rack
Chapter 6 Connecting an Exalogic Machine Quarter Rack and an Exadata Database Machine Quarter Rack Table 6-6 Connect 5-Meter Cables from Each Leaf Switch of Exadata Database Machine to the Spine Switch of Exalogic Machine Cable Coming from Port Connecting to Slot Port Length Slot (Exadata... - Page 41 Chapter 6 Connecting an Exalogic Machine Quarter Rack and an Exadata Database Machine Quarter Rack Table 6-7 (Cont.) Cross-Connections Between Exalogic Machine Quarter Rack and Exadata Database Machine Quarter Rack Exadata Database Machine Exalogic Machine Leaf Switch Port Leaf Switch Port R2-U24 R1-U16...
- Page 42 Chapter 6 Connecting an Exalogic Machine Quarter Rack and an Exadata Database Machine Quarter Rack 6-10...
-
Page 43: Multirack Cabling Diagrams For Exalogic-To-Exalogic Scenarios
Multirack Cabling Diagrams for Exalogic-to- Exalogic Scenarios This appendix contains multirack cabling diagrams for the following scenarios: • One Exalogic machine with another Exalogic machine, as listed in Table A-1. • Eight Exalogic machines, as described in Connecting Eight Exalogic Full Racks. -
Page 44: Connecting An Exalogic Eighth Rack And Exalogic Quarter Rack
Appendix A Connecting an Exalogic Eighth Rack and Exalogic Quarter Rack Figure A-1 Two Exalogic Eighth Racks Figure A-1, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch (leaf switches). A.2 Connecting an Exalogic Eighth Rack and Exalogic Quarter Rack Figure A-2 illustrates how an Exalogic eighth rack is connected to an Exalogic quarter rack on the same InfiniBand fabric. -
Page 45: Connecting An Exalogic Eighth Rack And Exalogic Half Rack
Appendix A Connecting an Exalogic Eighth Rack and Exalogic Half Rack A.3 Connecting an Exalogic Eighth Rack and Exalogic Half Rack Figure A-3 illustrates how an Exalogic eighth rack is connected to an Exalogic half rack on the same InfiniBand fabric. Figure A-3 Exalogic Eighth Rack and Exalogic Half Rack Figure... -
Page 46: Connecting Two Exalogic Quarter Racks
Appendix A Connecting Two Exalogic Quarter Racks Figure A-4 Exalogic Eighth Rack and Exalogic Full Rack Figure A-4, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch (leaf switches) A.5 Connecting Two Exalogic Quarter Racks Figure A-5 illustrates how two Exalogic quarter racks are connected on the same InfiniBand fabric. -
Page 47: Connecting An Exalogic Quarter Rack And An Exalogic Half Rack
Appendix A Connecting an Exalogic Quarter Rack and an Exalogic Half Rack Figure A-5, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch (leaf switches). A.6 Connecting an Exalogic Quarter Rack and an Exalogic Half Rack Figure A-6 illustrates how an Exalogic quarter rack is connected to an Exalogic half rack on the same InfiniBand fabric. -
Page 48: Connecting Two Exalogic Half Racks
Appendix A Connecting Two Exalogic Half Racks Figure A-7 Exalogic Quarter Rack and Exalogic Full Rack Figure A-7, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch (leaf switches) A.8 Connecting Two Exalogic Half Racks Figure A-8 illustrates how two Exalogic half racks are connected on the same InfiniBand fabric. -
Page 49: Connecting An Exalogic Half Rack And An Exalogic Full Rack
Appendix A Connecting an Exalogic Half Rack and an Exalogic Full Rack Figure A-8, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch (leaf switches). NM2-36P represents a Sun Datacenter InfiniBand Switch 36 (spine switch). In this configuration, four up-links from the gateway switch (leaf switch) are cabled to the Sun Datacenter InfiniBand Switch 36 (spine switches). -
Page 50: Connecting Eight Exalogic Full Racks
Appendix A Connecting Eight Exalogic Full Racks Figure A-10 Two Exalogic Full Racks Figure A-10, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch (leaf switches). NM2-36P represents a Sun Datacenter InfiniBand Switch 36 (spine switch). This scenario is explained in detail in Connecting Two Exalogic Machine Full Racks Together. - Page 51 Appendix A Connecting Eight Exalogic Full Racks Figure A-11 Eight Exalogic Full Racks Figure A-11, each leaf switch is connected to each spine switch (1 cable per leaf switch per spine switch).
- Page 52 Appendix A Connecting Eight Exalogic Full Racks A-10...
-
Page 53: Multirack Cabling Diagrams For Exalogic-To-Exadata Scenarios
Multirack Cabling Diagrams for Exalogic-to- Exadata Scenarios This appendix contains multirack cabling diagrams for the Exalogic-to-Exadata scenarios listed in Table B-1. Table B-1 Exalogic-to-Exadata Multirack Cabling Scenarios Exalogic Exadata Section Machine Database Rack Type Machine Rack Type Connecting an Exalogic Eighth Rack and an Exadata Quarter Rack Connecting an Exalogic Eighth Rack and an Exadata Half Rack Full... -
Page 54: Connecting An Exalogic Eighth Rack And An Exadata Half Rack
Appendix B Connecting an Exalogic Eighth Rack and an Exadata Half Rack Figure B-1 Exalogic Eighth Rack and Exadata Quarter Rack Figure B-1, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch. NM2-36P represents a Sun Datacenter InfiniBand Switch 36. B.2 Connecting an Exalogic Eighth Rack and an Exadata Half Rack Figure B-2... -
Page 55: Connecting An Exalogic Eighth Rack And An Exadata Full Rack
Appendix B Connecting an Exalogic Eighth Rack and an Exadata Full Rack B.3 Connecting an Exalogic Eighth Rack and an Exadata Full Rack Figure B-3 illustrates how an Exalogic eighth rack and an Exadata full rack are connected on the same InfiniBand fabric. Figure B-3 Exalogic Eighth Rack and Exadata Full Rack Figure... -
Page 56: Connecting An Exalogic Quarter Rack And An Exadata Half Rack
Appendix B Connecting an Exalogic Quarter Rack and an Exadata Half Rack Figure B-4 Exalogic Quarter Rack and Exadata Quarter Rack Figure B-4, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch. NM2-36P represents a Sun Datacenter InfiniBand Switch 36. This scenario is explained in detail in Connecting an Exalogic Machine Quarter Rack and an Exadata Database Machine Quarter... -
Page 57: Connecting An Exalogic Quarter Rack And An Exadata Full Rack
Appendix B Connecting an Exalogic Quarter Rack and an Exadata Full Rack Figure B-5 Exalogic Quarter Rack and Exadata Half Rack Figure B-5, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch. NM2-36P represents a Sun Datacenter InfiniBand Switch 36. B.6 Connecting an Exalogic Quarter Rack and an Exadata Full Rack Figure B-6... -
Page 58: Connecting An Exalogic Half Rack And An Exadata Quarter Rack
Appendix B Connecting an Exalogic Half Rack and an Exadata Quarter Rack B.7 Connecting an Exalogic Half Rack and an Exadata Quarter Rack Figure B-7 illustrates how an Exalogic half rack and an Exadata quarter rack are connected on the same InfiniBand fabric. Figure B-7 Exalogic Half Rack and Exadata Quarter Rack Figure... -
Page 59: Connecting An Exalogic Half Rack And An Exadata Full Rack
Appendix B Connecting an Exalogic Half Rack and an Exadata Full Rack Figure B-8 Exalogic Half Rack and Exadata Quarter Rack Figure B-8, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch. NM2-36P represents a Sun Datacenter InfiniBand Switch 36. B.9 Connecting an Exalogic Half Rack and an Exadata Full Rack Figure B-9... -
Page 60: Connecting An Exalogic Full Rack And An Exadata Quarter Rack
Appendix B Connecting an Exalogic Full Rack and an Exadata Quarter Rack B.10 Connecting an Exalogic Full Rack and an Exadata Quarter Rack Figure B-10 illustrates how an Exalogic full rack and an Exadata quarter rack are connected on the same InfiniBand fabric. Figure B-10 Exalogic Full Rack and Exadata Quarter Rack Figure... -
Page 61: Connecting An Exalogic Full Rack And An Exadata Full Rack
Appendix B Connecting an Exalogic Full Rack and an Exadata Full Rack Figure B-11 Exalogic Full Rack and Exadata Half Rack Figure B-11, NM2-GW represents a Sun Network QDR InfiniBand Gateway Switch. NM2-36P represents a Sun Datacenter InfiniBand Switch 36. B.12 Connecting an Exalogic Full Rack and an Exadata Full Rack Figure B-12... - Page 62 Appendix B Connecting an Exalogic Full Rack and an Exadata Full Rack In this configuration, the cabling within the Exadata machine is preserved. Only the traffic on the Exalogic machine and the Exalogic-to-Exadata traffic use the spine switches. The Exadata machine's local traffic is restricted to the leaf switches of the Exadata machine.
- Page 63 Index Multi-rack cabling (continued) overview, Multi-rack cabling Index-1...
- Page 64 Index Index-2...