Page 1: Windows 2000
Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Second Edition (June 2001) Part Number 225081-002 Compaq Computer Corporation...
Page 2 All other product names mentioned herein may be trademarks of their respective companies. Compaq shall not be liable for technical or editorial errors or omissions contained herein. The information in this document is provided “as is” without warranty of any kind and is subject to change without notice.
Page 3: Table Of Contents
Compaq Authorized Reseller................xx Chapter 1 Clustering Overview Clusters Defined ...................... 1-2 Availability ......................1-4 Scalability ........................ 1-4 Compaq Parallel Database Cluster Overview............1-5 Chapter 2 Cluster Architecture Compaq ProLiant Servers..................2-2 High-Availability Features of ProLiant Servers ..........2-3 Shared Storage Components..................2-3 MA8000/EMA12000 Storage Subsystem ............
Page 4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Hub ...................... 2-6 KGPSA-BC and KGPSA-CB Host Adapter ............. 2-6 Gigabit Interface Converter-Shortwave ............2-7 Fibre Channel Cables..................2-7 Configuring and Cabling the MA8000/EMA12000 Storage Subsystem Components ......................
Page 5 Compaq Insight Manager XE ................3-3 Compaq Options ROMPaq ................3-4 Compaq StorageWorks Command Console ............. 3-4 Compaq StorageWorks Secure Path for Windows 2000 ........3-4 Compaq Operating System Dependent Modules..........3-5 Oracle Software ....................... 3-5 Oracle8i Server Enterprise Edition ..............3-6 Oracle8i Server ....................
Page 6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Chapter 5 Installation and Configuration Reference Materials for Installation................. 5-1 Installation Overview....................5-2 Installing the Hardware.................... 5-3 Setting Up the Nodes ..................5-3 Installing the KGPSA-BC and KGPSA-CB Host Adapters......5-4 Installing GBIC-SW Modules for the Host Adapters ........
Page 7 Contents vii Verifying the Hardware and Software Installation ..........5-55 Cluster Communications ................5-55 Access to Shared Storage from All Nodes............5-55 OSDs ......................5-55 Other Verification Tasks ................5-56 Power Distribution and Power Sequencing Guidelines ......... 5-56 Overview ......................5-56 Server Power Distribution ................
Page 8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Chapter 7 Troubleshooting Basic Troubleshooting Tips ..................7-2 Power ........................ 7-2 Physical Connections ..................7-2 Access to Cluster Components ................. 7-3 Software Revisions ................... 7-3 Firmware Revisions ..................7-4 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages ..
Page 9 Run and Validate Secure Path Manager On All Nodes ...........A-5 Run Disk Management On All Nodes ..............A-5 Run and Validate the StorageWorks Command Console From All Storage Subsystems ......................A-6 Perform Cluster Software and Firmware Checks ............A-6 Perform Cluster Hardware Checks ................A-7 Contact Your Compaq Support Representative ............A-8 Glossary Index...
Page 10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide List of Figures Figure 1-1. Example of a two-node PDC/O5000 cluster ........1-3 Figure 2-1. SCSI bus numbers for I/O modules in the controller enclosure ..2-10 Figure 2-2.
Page 11 Contents xi Figure 5-4. UltraSCSI cabling between a controller enclosure and three disk enslosures ....................5-12 Figure 5-5. UltraSCSI cabling between a controller enclosure and six disk enclosures......................5-13 Figure 5-6. Dual redundancy configuration for a redundant Fibre Channel Fabric ....................... 5-15 Figure 5-7.
Page 12: About This Guide
Purpose This administrator guide provides information about the planning, installation, configuration, implementation, management, and troubleshooting of the Compaq Parallel Database Cluster Model PDC/O5000 on Oracle8i software running on the Microsoft Windows 2000 Advanced Server operating system. Audience The expected audience of this guide consists primarily of MIS professionals whose jobs include designing, installing, configuring, and maintaining Compaq Parallel Database Clusters.
Page 13: Scope
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Scope This guide offers significant background information about clusters as well as basic concepts associated with designing clusters. It also contains detailed product descriptions and installation steps.
Page 14: Referenced Manuals
Referenced Manuals For additional information, refer to documentation related to the specific hardware and software components of the Compaq Parallel Database Cluster. These related manuals include, but are not limited to: Documentation related to the ProLiant servers you are clustering (for...
Page 15: Supplemental Documents
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Supplemental Documents The following technical documents contain important supplemental information for the Compaq Parallel Database Cluster Model PDC/O5000: Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000, at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 16: Symbols In Text
About This Guide xvii Symbols in Text These symbols may be found in the text of this guide. They have the following meanings: WARNING: Text set off in this manner indicates that failure to follow directions in the warning could result in bodily harm or loss of life. CAUTION: Text set off in this manner indicates that failure to follow directions could result in damage to equipment or loss of information.
Page 17: Rack Stability
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Any surface or area of the equipment marked with these symbols indicates the presence of a hot surface or hot component. If this surface is contacted, the potential for injury exists.
Page 18: Compaq Technical Support
. This service is available 24 hours a day, 7 days a week. Outside North America, call the nearest Compaq Technical Support Phone Center. Telephone numbers for world wide Technical Support Centers are listed on the Compaq website. Access the Compaq website by logging on to the Internet at www.compaq.com...
Page 19: Compaq Authorized Reseller
Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Compaq Authorized Reseller For the name of your nearest Compaq Authorized Reseller: In the United States, call 1-800-345-1518. In Canada, call 1-800-263-5868. Elsewhere, see the Compaq website for locations and telephone...
Page 20: Clustering Overview
These clustering solutions use industry-standard hardware and software, thereby providing key clustering features at a lower price than proprietary clustering systems. Before examining the features and benefits of the Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000), it is helpful to understand the concepts and terminology of clustered systems.
Page 21: Clusters Defined
1-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Clusters Defined A cluster is an integration of software and hardware products that enables a set of loosely coupled servers and shared storage subsystem components to present a single system image to clients and to operate as a single system.
Page 22: Figure 1-1. Example Of A Two-Node Pdc/O5000 Cluster
Clustering Overview 1-3 Client LAN Switch/Hub Cluster Interconnect Host Adapters (2) Host Adapters (2) Node 1 Node 2 Fibre Channel SAN Fibre Channel SAN Switch/Storage Hub #2 Switch/Storage Hub #1 P1 P2 P1 P2 P1 P2 P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 1-1.
Page 23: Availability
1-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Availability When computer systems experience outages, the amount of time the system is unavailable is referred to as downtime. Downtime has several primary causes: hardware faults, software faults, planned service, operator error, and environmental factors.
Page 24: Compaq Parallel Database Cluster Overview
The PDC/O5000 combines the popular Microsoft Windows 2000 Advanced Server operating system and the industry-leading Oracle8i Parallel Server with award-winning Compaq ProLiant servers and shared storage subsystems.
Page 25: Cluster Architecture
Chapter Cluster Architecture The Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000) is an integration of a number of different hardware and software products. This chapter discusses how these products play a role in bringing a complete clustering solution to your computing environment.
Page 26: Compaq Proliant Servers
2-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Gigabit Interface Converter-Shortwave (GBIC-SW) modules Fibre Channel cables Cluster interconnect components Ethernet NIC adapters Ethernet cables Ethernet switches/hubs The software products include: Microsoft Windows 2000 Advanced Server with Service Pack 1 or later...
Page 27: High-Availability Features Of Proliant Servers
Error Checking and Correcting (ECC) memory. The high-availability features of ProLiant servers are a critical foundation of Compaq clustering products. Table 2-1 lists the high-availability features found in many ProLiant servers. Table 2-1...
Page 28: Ma8000/Ema12000 Storage Subsystem
2-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide MA8000/EMA12000 Storage Subsystem The MA8000/EMA12000 Storage Subsystem is the shared storage solution for the PDC/O5000. Each storage subsystem consists of one controller enclosure and up to six disk enclosures.
Page 29: Fibre Channel San Switch
Fibre Channel SAN Switch IMPORTANT: For detailed information about cascading two Fibre Channel SAN Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations for the Fibre Channel SAN Switch. Fibre Channel SAN Switches are installed between cluster nodes and shared storage subsystems in clusters to create redundant Fibre Channel Fabrics.
Page 30: Storage Hub
2-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide For further information, refer to these manuals provided with each Fibre Channel SAN Switch: Compaq StorageWorks Fibre Channel SAN Switch 8 Installation and Hardware Guide Compaq StorageWorks Fibre Channel SAN Switch 16 Installation and...
Page 31: Gigabit Interface Converter-Shortwave
MA8000/EMA12000 Storage Subsystem Components Configuring LUNS for Storagesets If you install the optional Compaq StorageWorks Large LUN utility on your PDC/O5000 cluster, you can distribute a maximum of 256 LUNs among the four array controller ports in each storage subsystem.
Page 32: Scsi Cabling Examples
2-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide If you do not install the optional Large LUN utility in you cluster, there are two accepted methods for distributing LUNs for storagesets: You can distribute eight LUNs between the same-numbered ports on both array controllers so that a total of 16 LUNs are assigned across all four controller ports in each storage subsystem.
Page 33: Ultrascsi Cables
UltraSCSI Cables Compaq recommends using the shortest cable length possible to connect disk enclosures to the controller enclosure. The maximum supported cable length is 2 meters (6.6 ft). Refer to the documentation provided with your storage subsystem for further information.
Page 34: Figure 2-1. Scsi Bus Numbers For I/O Modules In The Controller Enclosure
2-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide SCSI Bus 3 SCSI Bus 4 SCSI Bus 1 SCSI Bus 5 SCSI Bus 2 SCSI Bus 6 Controller Enclosure (rear view) Figure 2-1. SCSI bus numbers for I/O modules in the controller enclosure...
Page 35: Figure 2-2. Ultrascsi Cabling Between A Controller Enclosure And Three Disk Enclosures
Cluster Architecture 2-11 Dual-Bus I/O Module Disk Enclosure Disk Enclosure Disk Enclosure I/O Modules (6) Controller Enclosure Figure 2-2. UltraSCSI cabling between a controller enclosure and three disk enclosures Using Single-Bus I/O Modules in the Disk Enclosures Figure 2-3 shows UltraSCSI cables installed between a controller enclosure and six disk enclosures.
Page 36: Connecting Emus Between Ma8000/Ema12000 Storage Subsystems
2-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Single-Bus I/O Single-Bus I/O Module Module Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure Disk Enclosure I/O Modules (6) Controller Enclosure Figure 2-3. UltraSCSI cabling between a controller enclosure and six disk...
Page 37: I/O Path Configurations For Redundant Fibre Channel Fabrics
GBIC-SW modules installed in host adapters, Fibre Channel SAN Switches, and array controllers IMPORTANT: For detailed information about cascading two Fibre Channel SAN Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations for the Fibre Channel SAN Switch.
Page 38: Figure 2-4. Two-Node Pdc/O5000 With A Four-Fabric Redundant Fibre Channel Fabric
2-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide A redundant Fibre Channel Fabric consists of from two to four individual fabrics. The number of fabrics present is determined by the number of host adapters installed in the redundant Fibre Channel Fabric: two host adapters create two fabrics and four host adapters create four fabrics.
Page 39: Multiple Redundant Fibre Channel Fabrics
Fabric. The maximum number of redundant Fibre Channel Fabrics you can install in a PDC/O5000 is restricted by the number of host adapters your Compaq servers support. Refer to the Compaq server documentation for this information. Figure 2-5 shows a two-node PDC/O5000 with two redundant Fibre Channel Fabrics.
Page 40: Maximum Distances Between Nodes And Shared Storage Subsystem Components
2-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Subsystem #1 Storage Subsystem #2 P1 P2 P1 P2 P1 P2 P1 P2 Redundant Fibre Channel Fabric #1 Fibre Channel Fibre Channel SAN Switch #1...
Page 41: I/O Data Paths In A Redundant Fibre Channel Fabric
Cluster Architecture 2-17 Figure 2-6 illustrates these maximum cable distances for a redundant Fibre Channel Fabric. 500 meter maximum Host Adapters (2) Host Adapters (2) Node 1 Node 2 Fibre Channel Fibre Channel SAN Switch # 1 SAN Switch # 2 500 meter maximum P1 P2...
Page 42: Figure 2-7. Host Adapter-To-Fibre Channel San Switch I/O Data Paths
2-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapter-to-Fibre Channel SAN Switch I/O Data Paths Figure 2-7 highlights the I/O data paths that run between the host adapters and the Fibre Channel SAN Switches. Each host adapter has its own I/O data path.
Page 43: Figure 2-8. Fibre Channel San Switch-To-Array Controller I/O Data Paths
Cluster Architecture 2-19 Fibre Channel SAN Switch-to-Array Controller I/O Data Paths Figure 2-8 highlights the I/O data paths that run between the Fibre Channel SAN Switches and the two dual-port array controllers in each MA8000/EMA12000 Storage Subsystem. There is one I/O connection path to and from each array controller port.
Page 44: I/O Path Definitions For Redundant Fibre Channel Fabrics
2-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Definitions for Redundant Fibre Channel Fabrics In a redundant Fibre Channel Fabric, devices are accessed by the operating system using conventional SCSI addressing terminology. An I/O path consists of the complete physical interconnections from a given host (node) to a specific RAID storageset in a shared storage subsystem.
Page 45: I/O Path Configuration Examples For Redundant Fibre Channel Fabrics
Cluster Architecture 2-21 I/O Path Configuration Examples for Redundant Fibre Channel Fabrics Every I/O path connection between host adapters and array controller ports is active at the same time. An active port does not become inactive unless it fails, whereupon all I/O activity on the failed path is automatically switched over to the pre-defined backup I/O path.
Page 46: Figure 2-9. Dual Redundancy Configuration For A Redundant Fibre Channel Fabric
2-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (2) Host Adapters (2) Node 1 Node 2 Fibre Channel Fibre Channel SAN Switch # 1 SAN Switch # 2 P1 P2 P1 P2...
Page 47 Cluster Architecture 2-23 Figure 2-10 shows an acceptable method for cabling the I/O path components in a quad redundancy configuration. In this example, the following connections are made: Each of four host adapters in each node connects to the same-numbered Fibre Channel SAN Switch (host adapter #1 to Fibre Channel SAN Switch #1, host adapter #2 to Fibre Channel SAN Switch #2, host adapter #3 to Fibre Channel SAN Switch #3, host adapter #4 to Fibre...
Page 48: Figure 2-10. Quad Redundancy Configuration For A Redundant Fibre Channel Fabric
2-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (4) Node 1 Node 2 Node 3 Node 4 Fibre Channel Fibre Channel SAN Switch #1 SAN Switch #2 Fibre Channel Fibre Channel SAN Switch #3...
Page 49: Summary Of I/O Path Failure And Failover Scenarios For Redundant Fibre Channel Fabrics
Cluster Architecture 2-25 Summary of I/O Path Failure and Failover Scenarios for Redundant Fibre Channel Fabrics Table 2-3 identifies possible I/O path failure events for both dual redundancy or quad redundancy configurations in a redundant Fibre Channel Fabric and the failover response implemented by Secure Path for each failure. Table 2-3 I/O Path Failure and Failover Scenarios for Redundant Fibre Channel Fabrics Description of Failure...
Page 50: I/O Path Configurations For Redundant Fibre Channel Arbitrated Loops
2-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Configurations for Redundant Fibre Channel Arbitrated Loops Overview of FC-AL SAN Topology Fibre Channel standards define a multi-layered architecture for moving data across the storage area network (SAN). This layered architecture can be implemented using the Fibre Channel Fabric or the Fibre Channel Arbitrated Loop (FC-AL) topology.
Page 51: Figure 2-11. Two-Node Pdc/O5000 With A Four-Loop Redundant Fibre Channel Arbitrated Loop
Cluster Architecture 2-27 Host Adapters (2) Host Adapters (2) Node 1 Node 2 Storage Hub #2 Storage Hub #1 P1 P2 P1 P2 P1 P2 P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 2-11. Two-node PDC/O5000 with a four-loop redundant Fibre Channel Arbitrated Loop Used in conjunction with the I/O path failover capabilities of the Secure Path software, this redundant FC-AL configuration gives cluster resources...
Page 52: Multiple Redundant Fibre Channel Arbitrated Loops
2-28 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Multiple Redundant Fibre Channel Arbitrated Loops The PDC/O5000 supports the use of multiple redundant FC-ALs within the same cluster. You would install additional redundant FC-ALs in a PDC/O5000 to: Increase the amount of shared storage space available to the cluster’s...
Page 53: Figure 2-12. Two-Node Pdc/O5000 With Two Redundant Fibre Channel
Cluster Architecture 2-29 Figure 2-12 shows a two-node PDC/O5000 that contains two redundant FC-ALs. In this example, each redundant FC-AL has its own pair of host adapters in each node, a pair of Storage Hubs, and two MA8000/EMA12000 Storage Subsystems. In Figure 2-12, the hardware components that constitute the second redundant FC-AL are shaded.
Page 54: Maximum Distances Between Nodes And Shared Storage Subsystem Components
2-30 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Maximum Distances Between Nodes and Shared Storage Subsystem Components By using standard short-wave Fibre Channel cables with Gigabit Interface Converter-Shortwave (GBIC-SW) modules, the following maximum distances apply: Each MA8000/EMA12000 Storage Subsystem’s controller enclosure...
Page 55: Figure 2-13. Maximum Distances Between Pdc/O5000 Cluster Nodes And Shared Storage Subsystem Components In A Redundant Fc-Al
Cluster Architecture 2-31 Figure 2-13 illustrates these maximum cable distances for a redundant FC-AL. 500 meter maximum Host Adapters (2) Host Adapters (2) Node 1 Node 2 Storage Hub #1 Storage Hub #2 500 meter maximum P1 P2 P1 P2 P1 P2 P1 P2 Storage Subsystem #1...
Page 56: I/O Data Paths In A Redundant Fc-Al
2-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide I/O Data Paths in a Redundant FC-AL Within a redundant FC-AL, an I/O path connection exists between each host adapter and all four array controller ports in each storage subsystem.
Page 57: Figure 2-15. Storage Hub-To-Array Controller I/O Data Paths
Cluster Architecture 2-33 Secure Path monitors the status of the components along each active path. If Secure Path detects the failure of a host adapter, Fibre Channel cable, or Storage Hub along an active path, it automatically transfers all I/O activity on that path to the defined backup path.
Page 58: I/O Path Definitions For Redundant Fc-Als
2-34 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide If any component along an active path fails, Secure Path detects the failure and automatically transfers all I/O activity to the defined backup path. I/O Path Definitions for Redundant FC-ALs In a redundant FC-AL, devices are accessed by the operating system using conventional SCSI addressing terminology.
Page 59: I/O Path Configuration Examples For Redundant Fc-Als
Cluster Architecture 2-35 While cluster nodes use the unit number (Dxxyy) to identify and access a storageset or a single disk drive unit in the shared storage subsystems, the array controllers use a Port-Target-LUN (PTL) address to identify and access these resources in their storage subsystem.
Page 60: Figure 2-16. Dual Redundancy Configuration For A Redundant Fc-Al
2-36 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Host Adapters (2) Host Adapters (2) Node 1 Node 2 Storage Hub #2 Storage Hub #1 P1 P2 P1 P2 P1 P2 P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 2-16.
Page 61: Figure 2-17. Quad Redundancy Configuration For A Redundant Fc-Al
Cluster Architecture 2-37 The two odd-numbered Storage Hubs connect to port 1 of the array controllers. Storage Hub #1 connects to array controller A, port 1 in each storage subsystem. Storage Hub #3 connects to array controller B, port 1 in each storage subsystem. The two even-numbered Storage Hubs connect to port 2 of the array controllers.
Page 62: Summary Of I/O Path Failure And Failover Scenarios For Redundant Fc-Als
2-38 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Summary of I/O Path Failure and Failover Scenarios for Redundant FC-ALs Table 2-4 identifies possible I/O path failure events for both dual redundancy and quad configurations for a redundant FC-AL and the failover response implemented by Secure Path for each failure.
Page 63: Cluster Interconnect Requirements
TCP/IP. When configuring the Ethernet cluster interconnect, be sure to enable TCP/IP. NOTE: Refer to the technical white paper Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server (ECG062/0299) for detailed information about configuring redundant Ethernet cluster interconnects. This document is available at www.compaq.com/support/techpubs/whitepapers...
Page 64 2-40 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Ethernet Cluster Interconnect Components The following components are used in a redundant Ethernet cluster interconnect: Two Ethernet adapters in each cluster node Ethernet cables and switches or hubs...
Page 65: Figure 2-18. Redundant Ethernet Cluster Interconnect For A Two-Node Pdc/O5000 Cluster
Cluster Architecture 2-41 Ethernet Cluster Interconnect Diagrams Figure 2-18 shows the redundant Ethernet cluster interconnect components used in a two-node PDC/O5000 cluster. Ethernet Switch/Hub #1 for Cluster Interconnect Ethernet Switch/Hub #2 Crossover for Cluster Interconnect Cable Dual-port Dual-port Ethernet Ethernet Adapters (2) Adapters (2) Node 1...
Page 66: Local Area Network
2-42 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Local Area Network NOTE: For the PDC/O5000, the client LAN and the cluster interconnect must be treated as separate networks. Do not use either network to handle the other network’s traffic.
Page 67: Cluster Software Components
Compaq operating system dependent modules (OSDs) and this Oracle software: Oracle8i Enterprise Edition with the Oracle8i Parallel Server Option NOTE: For information about currently-supported software revisions for the PDC/O5000, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 68: Microsoft Windows 2000 Advanced Server
3-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Microsoft Windows 2000 Advanced Server This version of the PDC/O5000 supports Microsoft Windows 2000 Advanced Server with Service Pack 1 or later. NOTE: The PDC/O5000 does not work in conjunction with Microsoft Cluster Server. Do not install Microsoft Cluster Server on any of the cluster nodes.
Page 69: Compaq System Configuration Utility
Forwards server alert fault conditions Remotely controls servers The Integrated Management Log is used to collect and feed data to Compaq Insight Manager. This log is used with the Compaq Integrated Management Display (IMD), the optional Remote Insight controller, and SmartStart.
Page 70: Compaq Options Rompaq
3-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Compaq Options ROMPaq The Compaq Options ROMPaq diskettes allow a user to upgrade the ROM Firmware images for Compaq System product options, such as array controllers, disk drives, and tape drives used for non-shared storage.
Page 71: Compaq Operating System Dependent Modules
Cluster Software Components 3-5 You can also install the optional Compaq StorageWorks Large LUN utility with the Secure Path software. This utility allows cluster nodes to access up to 64 LUNs per target (array controller port). Compaq Operating System Dependent Modules Compaq supplies low-level services, called operating system dependent modules (OSDs), which are required by Oracle8i Parallel Server.
Page 72: Oracle8I Server Enterprise Edition
3-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Oracle8i Server Enterprise Edition The Oracle8i Server Enterprise Edition provides the following: Oracle8i Server Oracle8i Parallel Server Option Oracle8i Enterprise Manager Oracle8i Server Oracle8i Server is the database application software and must be installed on each node in the PDC/O5000.
Page 73: Oracle8I Enterprise Manager
Oracle8i Parallel Server that does not strictly conform to the configurations certified with this process. For a complete list of certified Compaq servers, see the Compaq Parallel Database Cluster Model PDC/05000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 74 3-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To implement this application and connection failover, a software interface between the Oracle software and the client must be written. Such a software interface would be responsible for detecting when the client’s cluster node is no longer available and then connecting the client to one of the remaining, operational cluster nodes.
Page 75: Cluster Planning
Chapter Cluster Planning Before connecting any cables or powering on any hardware on your Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000), it is important that you understand how all the various cluster components fit together to meet your operational requirements. The major...
Page 76: Site Planning
4-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Site Planning You must carefully select and prepare the site to ensure a smooth installation and a safe and efficient work environment. To select and prepare a location for...
Page 77: Planning Shared Storage Components For Redundant Fibre Channel Fabrics
NOTE: Certain restrictions apply to the server models and server configurations that are supported by the Compaq Parallel Database Cluster. For a current list of PDC/O5000- certified servers and details on supported configurations, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000. This document is available on the Compaq website at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 78: Planning Shared Storage Components For Redundant Fibre Channel Arbitrated Loops
4-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To determine which Fibre Channel SAN Switch model is appropriate for a redundant Fibre Channel Fabric in your cluster, you need to know the following: Whether you are implementing dual redundancy or quad redundancy in the redundant Fibre Channel Fabric.
Page 79 Cluster Planning 4-5 MA8000/EMA12000 Storage Subsystems, each of which holds up to 72 disk drives. Two dual-port HSG80 Array Controllers installed in each storage subsystem’s controller enclosure. NOTE: For more information about redundant Fibre Channel Arbitrated Loops (FC-ALs) in a PDC/O5000, see Chapter 2, “Cluster Architecture.” The Storage Hubs are available in two models: Storage Hub 7 (7 ports) Storage Hub 12 (12 ports)
Page 80: Planning Cluster Interconnect And Client Lan Components
4-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Planning Cluster Interconnect and Client LAN Components PDC/O5000 clusters running Oracle8i Parallel Server can use a redundant Ethernet cluster interconnect. A redundant cluster interconnect is required because it provides fault tolerance along the cluster interconnect paths.
Page 81: Planning Cluster Configurations For Redundant Fibre Channel Fabrics
Cluster Planning 4-7 Planning Cluster Configurations for Redundant Fibre Channel Fabrics A redundant Fibre Channel Fabric is two or more Fibre Channel SAN Switches installed between host adapters in cluster nodes and array controllers in the shared storage subsystems. Each redundant Fibre Channel Fabric in a PDC/O5000 provides I/O path connections between each cluster node and the two ports on both array controllers in each MA8000/EMA12000 Storage Subsystem.
Page 82: Figure 4-1. Dual Redundancy Configuration For A Redundant Fibre Channel Fabric
4-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 4-1 shows the correct method for cabling the I/O path components for dual redundancy in a redundant Fibre Channel Fabric. In Figure 4-1, both Fibre Channel SAN Switches connect to port 1 on one array controller and to port 2 on the other array controller in each storage subsystem.
Page 83: Planning Quad Redundancy Configurations
Cluster Planning 4-9 Planning Quad Redundancy Configurations For the highest possible level of I/O path redundancy, you can configure your cluster for quad redundancy. This configuration provides four host adapters in each of at least four nodes and four Fibre Channel SAN Switches for each redundant Fibre Channel Fabric.
Page 84: Figure 4-2. Quad Redundancy Configuration For A Redundant Fibre Channel Fabric
4-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The two even-numbered Fibre Channel SAN Switches connect to port 2 of the array controllers. Fibre Channel SAN Switch #2 connects to array controller A, port 2 in each storage subsystem. Fibre Channel SAN Switch #4 connects to array controller B, port 2 in each storage subsystem.
Page 85: Planning Cluster Configurations For Redundant Fibre Channel Arbitrated Loops
Cluster Planning 4-11 Planning Cluster Configurations for Redundant Fibre Channel Arbitrated Loops A redundant Fibre Channel Arbitrated Loop (FC-AL) is two or more Storage Hubs installed between host adapters in cluster nodes and array controllers in the shared storage subsystems. Each redundant FC-AL in a PDC/O5000 provides I/O path connections between each cluster node and the two ports on both array controllers in each MA8000/EMA12000 Storage Subsystem.
Page 86: Figure 4-3. Dual Redundancy Configuration For A Redundant Fc-Al
4-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 4-3 shows the correct method for cabling the I/O path components for dual redundancy in a redundant FC-AL. In Figure 4-3, both Storage Hubs connect to port 1 on one array controller and to port 2 on the other array controller in each storage subsystem.
Page 87: Planning Quad Redundancy Configurations
Cluster Planning 4-13 Planning Quad Redundancy Configurations For the highest possible level of I/O path redundancy, you can configure your cluster for quad redundancy. This configuration provides four host adapters in each of at least four nodes and four Storage Hubs for each redundant FC-AL. The recommended quad redundancy configuration includes these I/O path components: At least four cluster nodes...
Page 88: Figure 4-4. Quad Redundancy Configuration For A Redundant Fc-Al
4-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The two even-numbered Storage Hubs connect to port 2 of the array controllers. Storage Hub #2 connects to array controller A, port 2 in each storage subsystem. Storage Hub #4 connects to array controller B, port 2 in each storage subsystem.
Page 89: Raid Planning For The Ma8000/Ema12000 Storage Subsystem
Cluster Planning 4-15 RAID Planning for the MA8000/EMA12000 Storage Subsystem Storage subsystem performance is one of the most important aspects of tuning database cluster servers for optimal performance. Efforts to plan, configure, and tune a PDC/O5000 cluster should focus on getting the most out of each disk drive and having an appropriate number of shared storage drives in the cluster.
Page 90: Supported Raid Levels
4-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The PDC/O5000 implements RAID at the hardware level, which is faster than software RAID. When you implement RAID on shared storage arrays, you use the hardware RAID to perform such functions as making copies of the data or calculating checksums.
Page 91: Raw Data Storage And Database Size
Cluster Planning 4-17 Raw Data Storage and Database Size Raw data storage is the amount of storage available before any RAID levels have been configured. It is called raw storage because RAID volumes require some overhead. The maximum size of a database stored in a RAID system will always be less than the amount of raw data storage available, except for RAID 0, where no storage overhead is required.
Page 92: Selecting The Appropriate Raid Level
4-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Selecting the Appropriate RAID Level Many factors will affect the RAID levels you select for your cluster database. These include the specific availability, performance, reliability, and recovery capabilities required from the database.
Page 93: Planning The Grouping Of Physical Disk Storage Space
Cluster Planning 4-19 Planning the Grouping of Physical Disk Storage Space Figure 4-5 shows an example of how the physical storage space for one 12-drive array in an MA8000/EMA12000 Storage Subsystem disk enclosure could be grouped for an Oracle8i Parallel Server database. Shared Disk Storage Array Disk Disk...
Page 94: Disk Drive Planning
4-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide logical partitions. For detailed information about these procedures, refer to “Configuring a Storageset” and “Creating Partitions” in Chapter 5, “Installation and Configuration.” You use either the CLI window for the HSG80 Array Controllers or the Compaq StorageWorks Command Console (SWCC) to create, configure, and partition the logical disks.
Page 95: Network Planning
Ethernet Cluster Interconnect When an Ethernet cluster interconnect is installed between cluster nodes, the Compaq operating system dependent modules (OSDs) require a unique entry in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc for each network port on each node.
Page 96: Client Lan
4-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The following list identifies the format of the hosts and lmhosts files for a four-node PDC/O5000 cluster with an Ethernet cluster interconnect: IP address node1 IP address...
Page 97: Installation And Configuration
Chapter Installation and Configuration Reference Materials for Installation The Compaq Parallel Database Cluster PDC/O5000 (referred to here as the PDC/O5000) is a combination of several individually available products. As you set up your cluster, have the following materials available during installation.
Page 98: Installation Overview
5-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide StorageWorks Secure Path Software CD Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD Oracle8i Enterprise Edition CD Oracle8i Parallel Server Setup and Configuration Guide...
Page 99: Installing The Hardware
Installing the Fibre Channel Software Setup (FCSS) utility Installing the Secure Path Driver, Secure Path Agent, and Secure Path Manager Installing the optional Compaq StorageWorks Large LUN utility Specifying preferred_path units Creating partitions Installing and configuring the Compaq operating system dependent...
Page 100: Installing The Kgpsa-Bc And Kgpsa-Cb Host Adapters
5-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide software, review the section in this chapter entitled “Installing Operating System Software” to understand the idiosyncrasies of configuring a cluster. NOTE: Certain restrictions apply to the server models and server configurations that are supported by the Compaq Parallel Database Cluster.
Page 101: Cabling The Host Adapters To Fibre Channel San Switches Or Storage Hubs
SAN Switch/Storage SAN Switch/Storage Hub #1 Hub #2 Figure 5-1. Connecting host adapters to Fibre Channel SAN Switches or Storage Hubs For more information about Fibre Channel SAN Switches and Storage Hubs, refer to the Compaq documentation provided with them.
Page 102: Installing The Cluster Interconnect Adapters
5-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Installing the Cluster Interconnect Adapters A PDC/O5000 running Oracle8i software has an Ethernet cluster interconnect. Both redundant and non-redundant cluster interconnects can be used. Ethernet Cluster Interconnect For a non-redundant Ethernet cluster interconnect, install one single-port or dual-port Ethernet adapter into each cluster node.
Page 103 Installation and Configuration 5-7 To install GBIC-SW modules: 1. Verify that a GBIC-SW module is installed into both ports on each array controller. 2. Insert a GBIC-SW module into the correct port on each Storage Hub or Fibre Channel SAN Switch (two modules for each array controller). 3.
Page 104: Figure 5-2. Redundant Ethernet Cluster Interconnect For A Two-Node Pdc/O5000 Cluster
5-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-2 shows the redundant Ethernet cluster interconnect components used in a PDC/O5000 with two or more nodes. Ethernet Switch/Hub #1 for Cluster Interconnect Ethernet Switch/Hub #2...
Page 105 Installing redundant crossover cables directly between the nodes is not allowed. For information on configuring Ethernet connections in a redundant client LAN, including enabling failover from one Ethernet path to another, see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at www.compaq.com/support/techpubs/whitepapers...
Page 106: Installing Hardware Into An Ma8000/Ema12000 Storage Subsystem
5-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Installing Hardware Into an MA8000/EMA12000 Storage Subsystem Installing Hardware into a Controller Enclosure The controller enclosure for the MA8000/EMA12000 Storage Subsystem houses the two HSG80 Array Controllers, one cache module for each controller, an environmental monitoring unit (EMU), one or two power supplies, and three dual-speed fans.
Page 107: Cabling The Controller Enclosure To Disk Enclosures
Installation and Configuration 5-11 Cabling the Controller Enclosure to Disk Enclosures Compaq recommends using the shortest UltraSCSI cable length possible to connect disk enclosures to the controller enclosure. The maximum supported cable length is 2 meters (6.6 ft). Refer to the documentation provided with your storage subsystem for further information.
Page 108: Disk Enslosures
5-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-4 shows UltraSCSI cables installed between a controller enclosure and three disk enclosures. When three or fewer disk enclosures are present in a storage subsystem, you can install one dual-bus I/O module in each disk enclosure.
Page 109: Cabling Emus To Each Other
Figure 5-5. UltraSCSI cabling between a controller enclosure and six disk enclosures Cabling EMUs to Each Other For information about connecting the EMUs between enclosures in an MA8000/EMA12000 Storage Subsystem and between different subsystems, refer to the Compaq documentation provided with the storage subsystem.
Page 110: Cabling Array Controllers To Fibre Channel San Switches
5-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cabling Array Controllers to Fibre Channel SAN Switches In a PDC/O5000 with a redundant Fibre Channel Fabric, a dual redundancy configuration is the minimum allowable configuration that provides redundancy along the I/O paths.
Page 111: Cabling Array Controllers To Storage Hubs
Installation and Configuration 5-15 Host Adapters (2) Host Adapters (2) Node 1 Node 2 Fibre Channel Fibre Channel SAN Switch # 1 SAN Switch # 2 P1 P2 P1 P2 P1 P2 P1 P2 Storage Subsystem #1 Storage Subsystem #2 Figure 5-6.
Page 112: Installing Operating System Software
5-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Storage Hub still has access to port 1 on one array controller and port 2 on the other array controller in each storage subsystem. This ensures that the host adapters can still access the full range of LUNs.
Page 113: Guidelines For Clusters
Installation and Configuration 5-17 Guidelines for Clusters Installing clustering software requires several specific steps and guidelines that might not be necessary when installing software on a single server. Be sure to read and understand the following items before proceeding with the specific software installation steps in “Automated Installation Steps.”...
Page 114 5-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cluster-Specific SmartStart Installation The SmartStart Installation Poster describes the general flow of configuring and installing software on a single server. The installation for a PDC/O5000 will be very similar.
Page 115 Installation and Configuration 5-19 8. Double-click on TCP/IP and the Internet Protocol (TCP/IP) Properties screen is displayed. Enter the IP address as appropriate for each of the port connections. 9. Enter unique IP addresses and node names for each node in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc.
Page 116 5-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide e. With the client LAN NIC selected, click Move Up to position this NIC to the top of the list. f. Click OK on the dialog box and restart the node when prompted.
Page 117: Setting Up And Configuring An Ma8000/Ema12000 Storage Subsystem
Installation and Configuration 5-21 Setting up and Configuring an MA8000/EMA12000 Storage Subsystem Designating a Server as a Maintenance Terminal To set up and configure an MA8000/EMA12000 Storage Subsystem in a PDC/O5000, you must connect a cluster server (node) to an HSG80 Array Controller in that storage subsystem through the controller’s maintenance port.
Page 118: Installing The Storageworks Command Console (Swcc) Client
5-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 2. Connect each storage subsystem’s controller enclosure and disk enclosures to a guaranteed power source. 3. Power on all storage subsystem enclosure components as specified in the user documentation for the storage subsystem.
Page 119 To set port topologies and Arbitrated Loop Physical Address (ALPA) settings for both array controllers, perform these steps: NOTE: Refer to the Compaq StorageWorks documentation for more detailed information about changing the port topology settings. 1. Click on the Windows taskbar Start button.
Page 120 5-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide d. Verify that both controllers are in multibus failover mode by entering the following commands and viewing the results: show this_controller show other_controller e. Close the CLI window.
Page 121 Installation and Configuration 5-25 set this_controller port_2_topology=loop_hard set other_controller port_2_topology=loop_hard IMPORTANT: The ALPA addresses of port 1 on both array controllers must be identical; the ALPA addresses of port 2 on both array controllers must be identical. Use only the specified hexadecimal numerals for the ALPA addresses. For the array controllers, first use the middle-range ALPA addresses.
Page 122 5-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 17. If “this_controller” is in a redundant Fibre Channel Fabric, verify the following: a. Port_1_topology=fabric b. Port_2_topology=fabric 18. If “this_controller” is in a redundant FC-AL, verify the following: a.
Page 123: Verifying Array Controller Properties
5. Verify the devices discovered are consistent with the installed disk drives. The disk drives will be named in the DISK##### format. Refer to the Compaq StorageWorks documentation for the naming conventions for storage subsystem devices. 6. Close the CLI window.
Page 124: Table 5-1 Controller Properties
5-28 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Table 5-1 Controller Properties Description Values General Allocation class: 0 SCSI version: SCSI-2 Host Ports Host Port 1 Requested Topology: Loop Hard Host Port 1 Requested Port Address: (user defined)
Page 125: Configuring A Storageset
Installation and Configuration 5-29 To change the default connection names: 1. From CLI, issue the show connection command to display connection names and other connection information. The default connection names are listed under the “Connection Name” column. 2. For each host adapter, determine the correct new connection name using the format SVRnTOPn or SVRnBOTn, where SVRn identifies the node number, TOPn identifies the port number (1 or 2) in the top array controller, and BOTn identifies the port number (1 or 2) in the bottom...
Page 126 5-30 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 4. Click Next for step 3 of the wizard. 5. Select the maximum capacity for the virtual disk. IMPORTANT: Secure Path requires maximum capacity partitions. 6. Click Next for step 4 of the wizard.
Page 127 Installation and Configuration 5-31 12. Click Finish to create the virtual disk. The Storage window displays the virtual disk you created. The hourglass on the disk icon indicates that the storageset is being initialized. The drives you used to create the RAID set are highlighted in the Devices window.
Page 128: Installing Secure Path Software For Windows 2000
5-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. If not, type the following and press Enter for other units (virtual disks) as necessary: set unit-number enable_access_path=ALL Example: Set D1 enable_access_path=ALL 12. Close the CLI window.
Page 129: Description Of The Secure Path Software
Installing the Host Adapter Drivers To install the host adapter drivers: 1. From the Start menu, select Programs. Open Administrative Tools. 2. Select Computer Management. 3. Double-click on System Tools. 4. Double-click on Display Adapters. 5. Double-click on Compaq LP6000/LP7000/LP8000. A Properties screen appears.
Page 130: Installing The Fibre Channel Software Setup (Fcss) Utility
5-34 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 6. Select the Driver tab. Click Update driver. The Update Device Driver Wizard appears. 7. Select Display a list of the know drivers for the device so I can choose a specific driver.
Page 131: Installing The Secure Path Drivers, Secure Path Agent, And Secure Path Manager
Subsystem in the PDC/O5000. Do not perform these steps for other storage subsystems. To install the Secure Path Raidisk filter driver, RAIDisk driver, and agent: 1. Insert the Compaq StorageWorks Secure Path Software CD into your CD-ROM drive. 2. If you have CD AUTORUN enabled on your server, the Secure Path setup program will start automatically.
Page 132: Specifying The Preferred_Path For Storage Units
5-36 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To install Secure Path Manager: 1. Insert the StorageWorks Secure Path Software CD in your CD-ROM drive. 2. If you have CD AUTORUN enabled on your server, the Secure Path setup program will start automatically.
Page 133 Installation and Configuration 5-37 If you do not install the optional Large LUN utility on your cluster, there are two accepted methods for distributing LUNs for storagesets: You can distribute eight LUNs between the same-numbered ports on both array controllers so that a total of 16 LUNs are assigned across all four controller ports in each storage subsystem.
Page 134: Powering Up All Other Fibre Channel San Switches Or Storage Hubs
5-38 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 3. Verify that the above changes were made by entering the following command: show units full 4. Repeat steps 1 through 3 for each storage subsystem in the PDC/O5000.
Page 135: Installing Compaq Osds
Installing Compaq OSDs Use the Oracle Universal Installer (OUI) program to install the Compaq operating system dependent modules (OSDs) for Oracle8i Parallel Server. Compaq supplies an Oracle8i software package with the OUI that installs Compaq OSDs for an Ethernet cluster interconnect.
Page 136: Verifying Cluster Communications
5-40 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Sets up registry entries for file location and standard registry parameters for the OSDs on all cluster nodes. From one node in the cluster, the OUI installs and configures OSDs on the...
Page 137: Mounting Remote Drives And Verifying Administrator Privileges
Installation and Configuration 5-41 For example, the command ping node1 should return the following type of information. Pinging node1.loc1.yoursite.com [127.0.0.1] with 32 bytes of data: Reply from 127.0.0.1: bytes=32 time<10ms TTL=128 Reply from 127.0.0.1: bytes=32 time<10ms TTL=128 Reply from 127.0.0.1: bytes=32 time<10ms TTL=128 Reply from 127.0.0.1: bytes=32 time<10ms TTL=128 Mounting Remote Drives and Verifying Administrator Privileges...
Page 138: Installing The Ethernet Osds
5-42 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide If the command is not successful, make sure the administrative shares on remote node drives are present. Installing the Ethernet OSDs To install the OSDs for an Ethernet cluster interconnect: 1.
Page 139 Installation and Configuration 5-43 4. Click Next to continue. The License screen appears. 5. Click Accept to accept the terms of the license agreement. The File Locations screen appears.
Page 140 5-44 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 6. Click Next to continue. The Installation Types screen appears. 7. Select Typical to install the Ethernet OSDs into the default location, C:\Compaq\OPS. Skip to step 12.
Page 141 Installation and Configuration 5-45 9. If you want to accept the default location, skip to step 13. 10. If you want to specify an alternate location and you know the exact directory path, specify it by typing over the default location and skip to step 12.
Page 142 5-46 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. If you want to specify an alternate location but do not know the exact directory path, click Change Location. The Choose Directory screen appears.
Page 143 Installation and Configuration 5-47 12. Browse to the location where you want to install the OSDs. Click OK. The Component Locations screen appears indicating the alternate location you specified.
Page 144 5-48 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 13. Click Next to continue. The Cluster Members screen appears. 14. The Cluster Members screen indicates the host name of the installing machine. Define the node names for the client LAN by adding the node names you specified in the hosts and lmhosts files for the client LAN for all other cluster nodes.
Page 145 Installation and Configuration 5-49 15. Click Next to continue. The Cluster Members screen reappears. 16. The Cluster Members screen now shows the node names you specified on the first Cluster Members screen with “_san” appended to each name, indicating it as a cluster interconnect node name. If necessary, retype the cluster interconnect node names to match the names you entered into the hosts and lmhosts files for the Ethernet cluster interconnect.
Page 146 5-50 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 17. Click Next to continue. The OSD Summary screen appears.
Page 147 Installation and Configuration 5-51 18. If the information on the OSD Summary screen is not correct, click Previous to modify the previous screens. When the information is correct, click Next to continue. The Summary screen appears. 19. Click Install to install the components listed in the summary. At this point, the OUI verifies that the installing node can communicate with other cluster nodes through both the client LAN and cluster interconnect node names.
Page 148: Installing Oracle Software
5-52 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 21. When the installation completes, the End of Installation screen appears. 22. Click Exit to quit the OUI or Next Install to install another product. 23. Start the cluster manager, OracleCMService, on all nodes in the cluster.
Page 149: Configuring Oracle8I Software
Installation and Configuration 5-53 Oracle8i Parallel Server Manager Oracle8i Enterprise Manager NOTE: If the Oracle Universal Installer does not offer the option to install Oracle8i Parallel Server, it could mean that the OSDs did not install successfully or that the OracleCMService is not running.
Page 150: Additional Notes On Configuring Oracle Software
5-54 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide online changes, Oracle8i Server can find the correct data files and can properly start the Oracle database. You must install Object Link Manager before installing the Oracle database software to ensure that the symbolic links are defined before creating a database.
Page 151: Verifying The Hardware And Software Installation
If the Oracle service did not start successfully, make sure that the OracleCMService is started and the SID is in the Oracle registry. There are two logs that can provide information about why the Oracle service did not start. These logs are: \Compaq\OPS\cmsrvr.log \Compaq\OPS\nm.log...
Page 152: Other Verification Tasks
5-56 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Other Verification Tasks Perform these other verification tasks: Verify that Object Link Manager has been installed and is running. Verify that the OPS_CMDISK symbolic link has actually been set.
Page 153: Server Power Distribution
Installation and Configuration 5-57 Server Power Distribution Figure 5-8 shows an example of server power distribution for a three-node cluster. Power supply #1 of each server is connected to PDU #1. Power supply #2 of each server is connected to PDU #2. PDU #1 is connected to UPS #1, and PDU #2 is connected to UPS #2.
Page 154: Power Sequencing
5-58 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Power Sequencing The following power sequencing information applies to existing clusters that are up and running. The cluster installation steps presented previously in this chapter might have instructed you to perform a different sequence.
Page 155: Cluster Management
This chapter describes these management activities for the Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000). The topics addressed in this chapter include:...
Page 156: Cluster Management Concepts
6-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management Concepts Powering Off a Node Without Interrupting Cluster Services At some time during the life of your cluster you will need to perform an operation on a cluster node that will require it to be powered off.
Page 157: Managing Network Clients Connected To A Cluster
Cluster Management 6-3 4. Determine how critical it is to repair the problem. a. If the problem is not considered to be critical, wait until a non-peak time to service the problem. b. If the problem is critical but does not affect all cluster nodes, shut down the Oracle instances on the affected nodes and wait until a non-peak time to service the problem.
Page 158: Management Applications
6-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The following log files are used: The Node Manager (NM), provided by Compaq, logs messages to C:\compaq\ops\nm.log By default, the OracleCMService sends messages to C:\compaq\ops\cmsrvr.log Management Applications...
Page 159: Monitoring Storage Subsystem Hardware
Cluster Management 6-5 Compaq Insight Manager XE is a web-based management system that can also be used to monitor cluster hardware components. Compaq Insight Manager XE is an optional CD available upon request from the Compaq System Management website at www.compaq.com/sysmanage...
Page 160 6-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Using the Compaq StorageWorks Command Console The StorageWorks Command Console (SWCC) is a graphical user interface used to create and manage storage subsystems for the PDC/O5000. While the...
Page 161: Monitoring The Database
They can operate on Windows NT Server, Windows 95, or Windows 2000. Before installing Compaq Insight Manager, Compaq Insight Manager XE, or Oracle8i Enterprise Manager, it is recommended that you read through the corresponding documentation to determine how to set up and configure each...
Page 162: Software Maintenance
6-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Software Maintenance Deinstalling the OSDs At some point, you might want to deinstall the operating system dependent modules (OSDs). Some example situations are: When a node is being permanently removed from the cluster...
Page 163 Cluster Management 6-9 3. If autorun is enabled, the Oracle Universal Installer Welcome screen appears. 4. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD drive for d. The Oracle Universal Installer Welcome screen appears.
Page 164 6-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 5. Click Deinstall Products. Next, a window containing an inventory of the installed components appears. The following shows the Inventory screen that is displayed with an Ethernet cluster interconnect.
Page 165: Upgrading Oracle8I Server
Significant changes in Oracle8i Server will likely affect the operation of the cluster. Before upgrading to any new release of Oracle8i Server, consult your Compaq and Oracle service representatives. Make sure that the new release is certified and supported on the PDC/O5000, and get assistance to determine which procedure you should follow to perform the upgrade properly.
Page 166 6-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 2. Click Deinstall Products. The Inventory screen showing a list of installed components is displayed: 3. Checkmark Compaq Ethernet OSDs and click Remove. The Oracle Universal Installer removes all Ethernet OSDs.
Page 167: Deinstalling A Partial Osd Installation
Deinstalling a Partial OSD Installation While not a common occurrence, the Oracle Universal Installer might not be able to complete the installation of the Compaq Ethernet OSDs. An incomplete OSD installation could occur for a variety of reasons, for example: A cable is no longer seated tightly into its connector.
Page 168: Managing Changes To Shared Storage Components
6-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Managing Changes to Shared Storage Components Adding New Storagesets to Increase Shared Storage Capacity During the life of your PDC/O5000, you might need to expand the capacity of your shared storage subsystem.
Page 169: Replacing A Failed Drive In A Storage Subsystem
Cluster Management 6-15 Expanding a Logical drive Within a Storageset Expanding the size of a logical drive within a storageset is a complex process and should be attempted only by the site database administrator. The major steps in expanding a logical drive within a storageset are: 1.
Page 170: Replacing A Host Adapter
6-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To replace a failed drive in a storage subsystem: 1. Identify the failed drive by typing the following CLI command and pressing Enter: show failedset The command returns the name of the failed disk drive.
Page 171 8. From the Device Manager, open the entry SCSI Adapter and RAID Controllers and verify that the Compaq KGPSA device driver is started. If it is not started, reload the driver. 9. Connect the new host adapter by Fibre Channel cable to its Storage Hub...
Page 172 6-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 10. From CLI, use the Rename command to rename the connections between the host adapter and the ports on each storage subsystem array controller. When you replaced the host adapter, the connection names you defined were replaced by default connection names for the new host adapter.
Page 173: Adding A Shared Storage Subsystem
IMPORTANT: Certain restrictions apply to the server models and server configurations that are supported by the Compaq Parallel Database Cluster. For a current list of PDC/O5000-certified servers and details on supported configurations, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000.
Page 174: Removing The Node
6-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Removing the Node 1. Record pertinent information that will be needed when configuring the replacement node. This information includes: The IP address and network name of the client LAN adapters...
Page 175 IMPORTANT: Do not connect the server to the Storage Hubs or Fibre Channel SAN Switches yet. 2. Start the replacement node with the Compaq SmartStart and Support Software CD in the CD-ROM drive. 3. Use the Compaq System Configuration Utility to configure the hardware settings of the node and its adapters.
Page 176 6-22 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 11. Using Fibre Channel cables, connect the host adapters in the replacement node to the appropriate Storage Hubs or Fibre Channel SAN Switches. 12. If, in the process of replacing the node, you have also replaced the host adapters, do the following for each new host adapter: a.
Page 177 Cluster Management 6-23 13. Restart the replacement node. 14. Verify Ethernet connectivity between the replacement node and the existing cluster nodes. Run the ping utility from the replacement node, pinging the client LAN adapters on all other nodes. If Ethernet is used as the cluster interconnect, ping the cluster interconnect adapters of all other nodes.
Page 178: Adding A Cluster Node
6-24 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 2. If the replacement node is the node on which you originally installed the OSDs, do the following on all nodes: a. Shut down the Oracle instance.
Page 179: Preparing The New Node
IMPORTANT: Do not connect the server to the Storage Hubs or Fibre Channel SAN Switches yet. 2. Start the new node with the Compaq SmartStart and Support Software CD in the CD-ROM drive. 3. Use the Compaq System Configuration Utility to configure the hardware settings of the node and its adapters.
Page 180 6-26 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide 8. Install the host adapter drivers. 9. Install the FCSS utility. 10. Install and configure Secure Path for Windows 2000. 11. Using Fibre Channel cables, connect the host adapters in the new node to the correct Storage Hubs or Fibre Channel SAN Switches.
Page 181: Preparing The Existing Cluster Nodes
Cluster Management 6-27 14. Restart the new node. 15. Verify Ethernet connectivity between the new node and the existing cluster nodes. Run the ping utility from the replacement node, pinging the client LAN adapters on all other nodes. If Ethernet is used as the cluster interconnect, ping the cluster interconnect adapters of all other nodes.
Page 182 6-28 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To install the cluster software, perform the following steps: 1. To add a node to the cluster, you must disassemble the existing cluster and then reform it to include the new node. Do the following on all original cluster nodes (not the new node): a.
Page 183: Monitoring Cluster Performance
Compaq Insight Manager can be used to gather low-level, in-depth hardware statistics for the server and networking hardware components used in the cluster.
Page 184: Using Secure Path Manager
6-30 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Using Secure Path Manager StorageWorks Secure Path is a high availability software product that provides continuous data access for MA8000/EMA12000 Storage Subsystems. Redundant hardware, advanced RAID technology, and automated failover capability are used to enhance fault tolerance and availability.
Page 185 Cluster Management 6-31 Monitoring Storage Systems The Storage System View of Secure Path Manager displays each storage subsystem, array controllers, and associated storagesets. When you highlight a storageset in Storage System View, Secure Path Manager displays information about the physical I/O paths from each node to that storageset. The following information is displayed for each path: The Secure Path host system (node) that has an established access path to the storageset.
Page 186 6-32 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Make an I/O path Offline. Use this action when you want to prevent that path from being used for any I/O operations under any circumstances. Make an I/O path Online. Use this action when you want to return a path that is currently in the “Alt-Offline”...
Page 187: Uninstalling Secure Path
Cluster Management 6-33 Uninstalling Secure Path To remove Secure Path and Fibre Channel software and hardware from your cluster: 1. Open the Control Panel and choose Add/Remove Programs. 2. Select SANWorks Secure Path Server and click Add/Remove. 3. Select SANWorks Secure Path Client and click Add/Remove. 4.
Page 188: Troubleshooting
Chapter Troubleshooting This chapter addresses potential problems you might encounter as you install, configure, test, and operate your Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000). This chapter is organized as follows: Basic Troubleshooting Tips Troubleshooting Oracle8i and operating system dependent module...
Page 189: Basic Troubleshooting Tips
7-2 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Basic Troubleshooting Tips Power Verify that the cluster nodes, storage subsystems, and Fibre Channel SAN Switches or Storage Hubs for the storage area network (SAN) and the switches or hubs for the cluster interconnect are all powered on.
Page 190: Access To Cluster Components
Oracle8i Server with Oracle8i Parallel Server Option MA8000/EMA12000 Storage Subsystem drivers Compaq Insight Manager Compaq Array Configuration Utility Compaq Secure Path for Windows 2000 Compaq Ethernet OSDs Cluster interconnect drivers Microsoft Windows 2000 Advanced Server with Service Pack 1 or later...
Page 191: Firmware Revisions
7-4 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide For information about currently supported software revisions for the PDC/O5000, refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000. This document is available at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 192: Troubleshooting Oracle8I And Osd Installation Problems And Error Messages
Troubleshooting 7-5 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages Potential Difficulties Installing the OSDs with the Oracle Universal Installer The Oracle Universal Installer (OUI) uses an inventory file to track which components of the OSDs have been installed on each of the cluster nodes.
Page 193: Unable To Start Oraclecmservice
7-6 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Refer to “Deinstalling a Partial OSD Installation” in Chapter 6, “Cluster Management,” for information about how to remove a partial OSD installation. Unable to Start OracleCMService Verify that the cluster interconnect is properly configured.
Page 194: Unable To Start The Database
Check the System log and Applications log for errors. Check the OracleCMService error log for information. The default location for the error log is C:\compaq\ops\cmsrvr.log. If not found in the default location, look in the same directory in which the OSDs were installed.
Page 195: Unable To Ping The Cluster Interconnect Or The Client Lan
7-8 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide The cluster interconnect must be on a separate subnet than the client LAN. Check this by viewing the IP addresses assigned to all interconnect adapters and to all client LAN adapters.
Page 196: Troubleshooting Client-To-Cluster Connectivity Problems
Troubleshooting 7-9 If you are using two dual-port Ethernet adapters in each node to connect to both the client LAN and the Ethernet cluster interconnect, make sure that the cables are installed between the correct adapter ports and switches. Make sure the OSDs have been properly installed on the affected node or nodes by running the OUI on the cluster’s primary node.
Page 197: Troubleshooting Shared Storage Subsystem Problems
7-10 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Make sure that the Oracle TNSListener is running on the cluster nodes. From the Services screen, verify that it is started. Make sure that the Oracle Listener files are set up on the cluster nodes by using the Oracle TNSPing utility.
Page 198 Troubleshooting 7-11 To fix this problem, first do the following: 1. From the Event Viewer, locate the event log entry marked CPQKGPSA to find the host adapter device driver errors. 2. Restart the server. 3. If the driver error message persists, reinstall the KGPSA-BC Host Adapter device driver.
Page 199: Verifying Connectivity To A Redundant Fibre Channel Fabric
7-12 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Verifying Connectivity to a Redundant Fibre Channel Fabric Verify that the array controllers and the cluster nodes are properly connected to all paths of the redundant Fibre Channel Fabric as described in this section.
Page 200: Verifying Connectivity To A Redundant Fibre Channel Arbitrated Loop
Troubleshooting 7-13 The topology for Port 1 on each array controller should be: Fabric Up state (not the Standby state) FABRIC Server Node Connectivity to Fibre Channel Fabric Connection Paths To verify that your servers are connected to all Fibre Channel Fabric paths, open a CLI window and issue the CLI command show connection.
Page 201 7-14 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide For the array controller pair installed in each storage subsystem, verify that the two ports on each array controller are connected to different Storage Hubs. If an MA8000/EMA12000 Storage Subsystem has three or fewer disk enclosures, verify that a dual-bus I/O module with two SCSI cables is installed in each disk enclosure.
Page 202: A Cluster Node Cannot Connect To The Shared Drives
Troubleshooting 7-15 A Cluster Node Cannot Connect to the Shared Drives Verify connections and accessibility between the servers, the Fibre Channel SAN Switches or Storage Hubs, and the MA8000/EMA12000 Storage Subsystem controller enclosures. See “Verifying Cluster Communications” in Chapter 5, “Installation and Configuration.” Disk Management Shows Storagesets With the Same Label (Dual Image) To enhance I/O throughput, both ports (1 and 2) on an HSG80 Array...
Page 203: Devices On One I/O Connection Path Cannot Be Seen By The Cluster Nodes
7-16 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide To determine whether or not the storagesets can be viewed by a KGPSA-BC device driver, do the following: 1. From the Device Manager, select SCSI Adapters.
Page 204 Fibre Channel Fabric path. Do the following: Confirm that the Compaq KGPSA device driver was properly installed. Refer to “Installing Secure Path Software for Windows 2000” in Chapter 5, “Installation and Configuration.”...
Page 205: Troubleshooting Secure Path
7-18 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Verify that the Fibre Channel cables have been properly installed from the Storage Hubs to both ports on the array controller. Use the CLI to obtain status information about the array controller.
Page 206 Troubleshooting 7-19 You can distribute eight LUNs across all four array controller ports. For example, assign LUNs 0 and 1 to array controller A port 1, LUNS 2 and 3 to array controller A port 2, LUNS 4 and 5 to array controller B port 1, and LUNs 6 and 7 to array controller B port 2.
Page 207: Secure Path Manager Shows Reversed Locations For Top And Bottom Array Controllers
7-20 Compaq Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Secure Path Manager Shows Reversed Locations for Top and Bottom Array Controllers The Secure Path Manager GUI detects and displays the two HSG80 Array Controllers in each MA8000/EMA12000 Storage Subsystem according to their serial numbers, not their slot assignments.
Page 208: The Addition Of New Luns Causes An Error
Troubleshooting 7-21 The Addition of New LUNs Causes an Error In a PDC/O5000 cluster running Windows 2000 Advanced Server, the addition of new LUN(s) may cause the Add/Remove Hardware Wizard to display an error upon initializing the LUN(s) and yellow warning indicators to be displayed from the Device Manager.
Page 209: Diagnosing And Resolving Shared Disk Problems
Diagnosing and Resolving Shared Disk Problems Introduction When one or more nodes in a Compaq Parallel Database Cluster Model PDC/O5000 (referred to here as the PDC/O5000) cannot properly join the cluster, you need to consider two possible causes: A problem related to the shared disk storage in the cluster’s...
Page 210 Disk On All Nodes Affected Problems Nodes Problem Persists Problem Perform Cluster Software Persists and Firmware Checks Problem Persists Perform Cluster Hardware Checks Problem Persists Contact Your Compaq Support Representative Figure A-1. Tasks for diagnosing and resolving shared storage problems...
Page 211: Run Object Link Manager On All Nodes
“Restart All Affected Nodes in the Cluster” in this appendix for further instructions. For instructions on installing and using Object Link Manager, see the readme file: \Object Link Manager\readme.txt This file is located on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD.
Page 212: Restart All Affected Nodes In The Cluster
1. Verify that the Object Link Manager service is properly installed on the node. For instructions on installing and using Object Link Manager, refer to this readme file contained on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 \Object Link Manager\readme.txt...
Page 213: Run And Validate Secure Path Manager On All Nodes
Secure Path Manager screens. 2. Verify that the version number for Secure Path Manager is correct. Refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000 for version information. 3. Verify that Secure Path Manager has been properly installed on the node.
Page 214: Run And Validate The Storageworks Command Console From All Storage Subsystems
4. Repeat steps 1 through 3 for all nodes in the cluster. Perform Cluster Software and Firmware Checks Use the Compaq SmartStart and Support Software CD to perform these checks on the cluster software and firmware: 1. Verify that all necessary drivers have been installed. For example, verify that the correct driver for the host adapter is installed and running on all cluster nodes.
Page 215: Perform Cluster Hardware Checks
NOTE: Refer to the Compaq Parallel Database Cluster Model PDC/O5000 Certification Matrix for Windows 2000 for information about supported software and firmware versions for this PDC, at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 216: Contact Your Compaq Support Representative
A-8 Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Contact Your Compaq Support Representative If you are unable to resolve the problem, contact your Compaq support representative for assistance. In North America, call the Compaq Technical Support Phone Center at 1-800-OK-COMPAQ.
Page 217: Glossary
Glossary Availability A measure of how well a computer system can deliver services to its clients continuously. Availability is typically expressed as a percentage, with 100 percent being the best possible rating. Client LAN A local area network dedicated to servicing requests between network client machines and between the cluster nodes and the client machines.
Page 218 2 Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Fabric A complex set of interconnections through which there can be multiple and (to the user) unknown paths from point to point. The term “fabric” is used to refer to the X or Y portion of the internal or external network;...
Page 219 Glossary 3 Interconnect A physical connection between cluster nodes that transmits intra-cluster communication, e.g. the cluster heartbeat. The Compaq Parallel Database Cluster interconnect is dedicated to servicing requests only between cluster nodes. Integrated An Oracle software component that coordinates database locks Distributed Lock among cluster nodes.
Page 220 Resource A software or hardware entity upon which a client/server application or service is dependent. As it pertains to the Compaq Parallel Database Cluster, a cluster resource must have the ability to be managed by the cluster and must reside on one of the cluster nodes.
Page 221 Glossary 5 Shared storage A storage system that can be simultaneously accessed by multiple nodes in a PDC cluster. The MA8000/EMA12000 Storage Subsystem is the shared storage subsystem for the PDC/O5000. Storageset A group of disk drives configured with RAID techniques to operate as a single container and ensure that every partitioned storageset whether it uses 10 or 14 physical disk driveslooks like a single storage unit to the cluster nodes.
Page 222: Index
Oracle8i 5-41 Software 5-35 ALPAs 5-23 certification application failover software for Oracle8i software 3-7 Oracle8i 3-7 CIM See Compaq Insight Manager arbitrated loop physical CLI Window 3-4 addresses 5-23 client LAN ASR timer value 5-4 adapters installation for Oracle availability 8.1.7 5-6...
Page 223 RAID planning 4-15 distributed lock shared disk drives 4-20 management 2-39 site planning 4-2 Ethernet cluster cluster software for Oracle8i interconnect 2-39 Compaq software options 2-39 overview 3-2 troubleshooting 7-7 Oracle software 3-6 type supported 1-3 overview 3-1 cluster management...
Page 224 Oracle8i paths 7-13 overview 3-5 clusters Compaq Options ROMPaq, availability in 1-4 overview 3-4 defined 1-2 Compaq Parallel Database Cluster, overview 1-1 overview 1-5 PDC/O5000 example 1-2 Compaq SmartStart and Support scalability in 1-4 software 3-2 Command Line Interface...
Page 225 4 Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide Compaq System Configuration Utility 3-3 error logs 6-4 Compaq StorageWorks Command Ethernet adapters Console 3-4 planning 4-6 Compaq StorageWorks Fibre Ethernet client LAN Channel Storage Hubs cabling 5-9...
Page 226 Index 5 installing dual redundancy multiple redundant Fibre configurations 5-15 Channel Fabrics multiple Fibre Channel example 2-15 Arbitrated Loops 2-28 quad redundancy configuration multiple redundant FC-AL example 2-22 example 2-29 redundant fabrics 2-14 quad redundancy configuration redundant Fibre Channel Fabrics 2-13 example 2-36 redundant Fibre Channel topology overview 2-13...
Page 227 6 Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide verifying connectivity to Fibre GBIC-SW modules for Channel Fabrics 7-13 Oracle8i 5-4 hosts files Object Link Manager for for Ethernet cluster Oracle8i 5-53 interconnect 4-21 Oracle8i software 5-52 for Oracle 8.1.7 5-40 installation for Oracle 8.1.7 for Oracle8i 5-19...
Page 228 Oracle8i 5-55 verifying array controller logical drives properties 5-27 expanding capacity 6-15 Integrated Management Log, using logical partitions with Compaq Insight in extended partitions 5-38 Manager 3-3 Oracle8i Parallel Server database resources 5-38 logical unit number 2-5 KGPSA Host Adapters...
Page 229 A-4 storage units 5-36 running to check for shared storage power disk problems A-3 distribution 5-57 operating system dependent StorageWorks Command modules See Compaq Console 2-4 operating system dependent troubleshooting 7-10 modules using dual-bus I/O operating system software modules 2-10...
Page 230 7-6 selecting RAID levels 4-18 shared disk drives 4-20 OracleNMService, troubleshooting 7-6 shared storage components for OSDs See Compaq operating redundant FC-ALs 4-4 system dependent modules shared storage components for OUI See Oracle Universal Installer redundant Fibre Channel overview...
Page 231 5-28 performing cluster software replacing cluster nodes 6-19 and firmware checks A-6 RJ-45 receptacle xvii rerunning and validating Object ROMPaq See Compaq Options Link Manager A-4 Rompaq restarting all affected nodes A-4 running and validating Secure Path Manager A-5...
Page 232 7-3 required by Secure Path 5-33 required for OSDs for storage hubs powering on 5-21, 5-38 Oracle8i 5-40 Storage Hubs See Compaq technical support xix StorageWorks Fibre Channel telephone numbers xx Storage Hubs text conventions xvi description 2-6...
Page 233 12 Parallel Database Cluster Model PDC/O5000 for Oracle8i and Windows 2000 Administrator Guide KGPSA-BC driver access to shared storage 7-15 uninterruptible power source 5-56 NM.DLL 7-7 upgrading nodes cannot Oracle8i Server 6-11 communicate 7-7 Oracle8i installation 7-5 OracleCMService 7-6 OracleNMService 7-6 verifying IP addresses Secure Path errors 7-18 for Oracle8i 5-19...

This manual is also suitable for:

Proliant 6500 Proliant 8500 6400r - proliant - 1 gb ram Pdc/o5000

Compaq 112726-001 - ProLiant - 6500 Administrator's Manual

About this Guide

Chapter 1 Clustering Overview

Chapter 2 Cluster Architecture

Chapter 3 Cluster Software Components

Chapter 4 Cluster Planning

Chapter 5 Installation and Configuration

Chapter 6 Cluster Management

Chapter 7 Troubleshooting

Appendix A Diagnosing and Resolving Shared Disk Problems

Quick Links

Windows 2000

Troubleshooting

Related Manuals for Compaq 112726-001 - ProLiant - 6500

Summary of Contents for Compaq 112726-001 - ProLiant - 6500