Page 1: Windows 2000
Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Second Edition (June 2001) Part Number 225082-002 Compaq Computer Corporation...
Page 2 All other product names mentioned herein may be trademarks of their respective companies. Compaq shall not be liable for technical or editorial errors or omissions contained herein. The information in this document is provided “as is” without warranty of any kind and is subject to change without notice.
Page 3: Table Of Contents
Symbols in Text.......................xvii Symbols on Equipment..................xviii Rack Stability ......................xix Getting Help ......................xix Compaq Technical Support ................xix Compaq Website....................xx Compaq Authorized Reseller................xx Chapter 1 Clustering Overview Clusters Defined ...................... 1-2 Availability ......................1-3 Scalability ........................ 1-3 Compaq Parallel Database Cluster Overview............1-4...
Page 4 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Chapter 2 Cluster Architecture Compaq ProLiant Servers ..................2-2 High-Availability Features of ProLiant Servers ..........2-3 Shared Storage Components ..................2-3 RA4000 Array....................2-4 RA4100 Array....................2-4 RA4000 Array Controllers................
Page 5 Compaq Insight Manager ................. 3-4 Compaq Insight Manager XE ................3-4 Compaq Options ROMPaq ................3-4 Compaq StorageWorks Secure Path for Windows 2000 ........3-5 Compaq Operating System Dependent Modules..........3-5 Oracle Software ....................... 3-6 Oracle8i Server Enterprise Edition ..............3-6 Oracle8i Server ....................
Page 6 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Cluster Planning continued Planning Cluster Configurations for Redundant Fibre Channel Arbitrated Loops ........................4-11 Sample Small Configuration in a Redundant FC-AL ........4-11 Sample Large Configuration in a Redundant FC-AL ........4-13 RAID Planning.......................
Page 7 Contents vii Installing Compaq OSDs ..................5-30 Verifying Cluster Communications ..............5-31 Mounting Remote Drives and Verifying Administrator Privileges ....5-32 Installing the Ethernet OSDs ................5-33 Installing Oracle Software ..................5-44 Configuring Oracle Software................. 5-45 Installing Object Link Manager ................5-45 Additional Notes on Configuring Oracle Software ........
Page 8 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management continued Replacing a Cluster Node ..................6-17 Removing the Node ..................6-17 Adding the Replacement Node ............... 6-18 Adding a Cluster Node................... 6-21 Preparing the New Node................. 6-22 Preparing the Existing Cluster Nodes .............
Page 9 Run and Validate Secure Path Manager On All Nodes ...........A-5 Run Disk Management On All Nodes ..............A-5 Run and Validate the Array Configuration Utility On All Nodes......A-6 Perform Cluster Software and Firmware Checks ............A-6 Perform Cluster Hardware Checks ................A-7 Contact Your Compaq Support Representative ............A-8 Glossary Index...
Page 10 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide List of Figures Figure 1-1. Example of a two-node Compaq Parallel Database Model PDC/O2000 cluster ................. 1-2 Figure 2-1. Two-node PDC/O2000 with a two-fabric redundant Fibre Channel Fabric ....................
Page 11 Contents xi Figure 2-25. Active/active configuration with two RA4000/RA4100 Arrays ..2-60 Figure 2-26. Active/active configuration with three RA4000/RA4100 Arrays....................... 2-61 Figure 2-27. Active/active configuration with four RA4000/RA4100 Arrays....................... 2-62 Figure 2-28. Active/active configuration with five RA4000/RA4100 Arrays ..2-63 Figure 2-29.
Page 12: List Of Tables
Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide List of Tables Table 2-1 High-Availability Components of ProLiant Servers........ 2-3 Table 2-2 Features of Active/Standby and Active/Active Configurations for Redundant Fibre Channel Fabrics ..............2-17...
Page 13: About This Guide
Purpose This administrator guide provides information about the planning, installation, configuration, implementation, management, and troubleshooting of the Compaq Parallel Database Cluster Model PDC/O2000 running Oracle8i software on the Microsoft Windows 2000 Advanced Server operating system. Audience The expected audience of this guide consists primarily of MIS professionals whose jobs include designing, installing, configuring, and maintaining Compaq Parallel Database Clusters.
Page 14: Scope
Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Scope This guide offers significant background information about clusters as well as basic concepts associated with designing clusters. It also contains detailed product descriptions and installation steps.
Page 15: Referenced Manuals
Referenced Manuals For additional information, refer to documentation related to the specific hardware and software components of the Compaq Parallel Database Cluster. These related manuals include, but are not limited to: Documentation related to the ProLiant servers you are clustering...
Page 16: Supplemental Documents
Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Oracle8i documentation, including: Oracle8i Parallel Server Setup and Configuration Guide Oracle8i Parallel Server Concepts Oracle8i Parallel Server Administration, Deployment, and Performance Oracle Enterprise Manager Administrator’s Guide...
Page 17: Text Conventions
About This Guide xvii Text Conventions This document uses the following conventions to distinguish elements of text: User Input, GUI Text a user types or enters appears in boldface. Selections Items a user selects from a GUI, such as tabs, buttons, or menu items, also appear in boldface.
Page 18: Symbols On Equipment
Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Symbols on Equipment These icons may be located on equipment in areas where hazardous conditions may exist. Any surface or area of the equipment marked with these symbols indicates the presence of electrical shock hazards.
Page 19: Rack Stability
Compaq Technical Support In North America, call the Compaq Technical Phone Support Center at 1-800-OK-COMPAQ. This service is available 24 hours a day, 7 days a week. For continuous quality improvement, calls may be recorded or monitored. Outside North America, call the nearest Compaq Technical Support Phone Center.
Page 20: Compaq Website
Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Compaq Website The Compaq website has information on this product as well as the latest drivers and Flash ROM images. You can access the Compaq website by logging on to the Internet at www.compaq.com...
Page 21: Clustering Overview
These clustering solutions use industry-standard hardware and software, thereby providing key clustering features at a lower price than proprietary clustering systems. Before examining the features and benefits of the Compaq Parallel Database Cluster Model PDC/O2000 (referred to here as the PDC/O2000), it is helpful to understand the concepts and terminology of clustered systems.
Page 22: Clusters Defined
1-2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Clusters Defined A cluster is an integration of software and hardware products that enables a set of loosely coupled servers and shared storage subsystem components to present a single system image to clients and to operate as a single system.
Page 23: Availability
Clustering Overview 1-3 Availability When computer systems experience outages, the amount of time the system is unavailable is referred to as downtime. Downtime has several primary causes: hardware faults, software faults, planned service, operator error, and environmental factors. Minimizing downtime is a primary goal of a cluster. Simply defined, availability is the measure of how well a computer system can continuously deliver services to clients.
Page 24: Compaq Parallel Database Cluster Overview
1-4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Compaq Parallel Database Cluster Overview As traditional clustering technology has moved into the open systems of client/server computing, Compaq has provided innovative, customer-focused solutions. The PDC/O2000 moves client/server computing one step closer to the capabilities found in expensive, proprietary cluster solutions, at a fraction of the cost.
Page 25: Cluster Architecture
Chapter Cluster Architecture The Compaq Parallel Database Cluster Model PDC/O2000 (referred to here as the PDC/O2000) is an integration of a number of different hardware and software products. This chapter discusses how these products play a role in bringing a complete clustering solution to your computing environment.
Page 26: Compaq Proliant Servers
2-2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Cluster interconnect components Ethernet NIC adapters Ethernet cables Ethernet switches/hubs The software products include: Microsoft Windows 2000 Advanced Server with Service Pack 1 or later Compaq drivers and utilities Oracle8i Enterprise Edition with the Oracle8i Parallel Server Option Refer to Chapter 3, “Cluster Software Components,”...
Page 27: High-Availability Features Of Proliant Servers
Error Checking and Correcting (ECC) memory. The high-availability features of ProLiant servers are a critical foundation of Compaq clustering products. Table 2-1 lists the high-availability features found in many ProLiant servers. Table 2-1...
Page 28: Ra4000 Array
2-4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Storage Hubs or FC-AL Switches for each redundant Fibre Channel Arbitrated Loop (FC-AL) Fibre Host Adapters Gigabit Interface Converter-Shortwave (GBIC-SW) modules Fibre Channel cables RA4000 Array The RA4000 Array is one shared storage solution for the PDC/O2000.
Page 29: Ra4000 Array Controllers
The array controller that is installed in the active slot is automatically assigned active status by Compaq Secure Path, without the need for any further configuration. To determine which of the two array controllers in an RA4000/RA4100 Array is currently active, find the controller on which the ninth green LED is lit;...
Page 30: Fibre Channel San Switches
2-6 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Access to the same logical disks is provided to both RA4000 Array Controllers to allow for successful failovers. In this configuration, both the active and standby array controllers are configured to receive and transmit data for the same logical disks.
Page 31: Fc-Al Switches
FC-AL Switches IMPORTANT: For detailed information about cascading two FC-AL Switches, refer to the latest Compaq StorageWorks documentation. This guide does not document cascaded configurations for the FC-AL Switch. FC-AL Switches can be installed between cluster nodes and shared storage subsystems in PDC/O2000 clusters to create redundant Fibre Channel Arbitrated Loops (FC-ALs).
Page 32: Fibre Host Adapters
2-8 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide The PDC/O2000 allows the use of either the Storage Hub 7 (with 7 ports) or the Storage Hub 12 (with 12 ports). Using the Storage Hub 7 limits the size of the PDC/O2000 cluster.
Page 33: Fibre Channel Cables
Cluster Architecture 2-9 Fibre Channel Cables Shortwave (multi-mode) fibre optic Fibre Channel cables are used to connect the nodes, the Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches, and RA4000/RA4100 Arrays in a PDC/O2000 cluster. I/O Path Configurations for Redundant Fibre Channel Fabrics Overview of Fibre Channel Fabric SAN Topology Fibre Channel standards define a multi-layered architecture for moving data...
Page 34: Figure 2-1. Two-Node Pdc/O2000 With A Two-Fabric Redundant Fibre Channel Fabric
2-10 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide One or more RA4000/RA4100 Arrays, each containing two single-port RA4000 Array Controllers GBIC-SW modules installed in the Fibre Host Adapters, Fibre Channel SAN Switches, and array controllers...
Page 35: Multiple Redundant Fibre Channel Fabrics
Fibre Channel Fabric. The maximum number of redundant Fibre Channel Fabrics you can install in a PDC/O2000 cluster is restricted by the number of Fibre Host Adapters your Compaq servers support. Refer to the Compaq server documentation for this information.
Page 36: Figure 2-2. Two-Node Pdc/O2000 With Two Redundant Fibre Channel Fabrics
2-12 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Figure 2-2 shows a two-node PDC/O2000 that contains two redundant Fibre Channel Fabrics. In this example, each redundant Fibre Channel Fabric has its own pair of Fibre Host Adapters in each node, a pair of Fibre Channel SAN Switches, and two RA4000/RA4100 Arrays.
Page 37: Maximum Distances Between Nodes And Shared Storage Components In A Redundant Fibre Channel Fabric
Cluster Architecture 2-13 Maximum Distances Between Nodes and Shared Storage Components in a Redundant Fibre Channel Fabric By using standard short-wave Fibre Channel cables with Gigabit Interface Converter-Shortwave (GBIC-SW) modules, the following maximum distances apply: Each RA4000/RA4100 Array can be placed up to 500 meters from the Fibre Channel SAN Switches to which it is cabled.
Page 38: I/O Data Paths In A Redundant Fibre Channel Fabric
2-14 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide I/O Data Paths in a Redundant Fibre Channel Fabric A distinct I/O path connection exists between each Fibre Host Adapter in a redundant Fibre Channel Fabric and every array controller port in each RA4000/RA4100 Array in that redundant Fibre Channel Fabric.
Page 39: I/O Path Configuration Guidelines For Redundant Fibre Channel Fabrics
Cluster Architecture 2-15 Fibre Channel SAN Switch-to-Array Controller Paths Figure 2-5 highlights the I/O data paths that run between the Fibre Channel SAN Switches and the two single-port RA4000 Array Controllers in each RA4000/RA4100 Array. Fibre Host Fibre Host Adapters (2) Adapters (2) Node 1 Node 2...
Page 40 2-16 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide In every redundant Fibre Channel Fabric, at least two Fibre Host Adapters must be installed in each node (one adapter for each Fibre Channel SAN Switch in that redundant Fibre Channel Fabric).
Page 41: Table 2-2 Features Of Active/Standby And Active/Active Configurations For Redundant Fibre Channel Fabrics
Cluster Architecture 2-17 Table 2-2 identifies the features of the active/standby and active/active configurations for redundant Fibre Channel Fabrics. Table 2-2 Features of Active/Standby and Active/Active Configurations for Redundant Fibre Channel Fabrics I/O Path Configuration Advantage Disadvantage Active/standby with one Active/standby is the only I/O RA4000/RA4100 Array path configuration you can use...
Page 42 2-18 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Table 2-2 Features of Active/Standby and Active/Active Configurations for Redundant Fibre Channel Fabrics continued I/O Path Configuration Advantage Disadvantage Active/active with two or more Provides a small but measurable...
Page 43: I/O Path Configuration Rules For Redundant Fibre Channel Fabrics
Cluster Architecture 2-19 I/O Path Configuration Rules for Redundant Fibre Channel Fabrics The following rules must be observed in I/O path configurations for PDC/O2000 clusters with redundant Fibre Channel Fabrics: Each redundant Fibre Channel Fabric must use either an active/standby or active/active configuration.
Page 44: Active/Standby Configuration Examples For Redundant Fibre Channel Fabrics
2-20 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Standby Configuration Examples for Redundant Fibre Channel Fabrics This section describes examples of active/standby configurations when one, two, three, four, and five RA4000/RA4100 Arrays are present in one redundant Fibre Channel Fabric of a four-node PDC/O2000.
Page 45 Cluster Architecture 2-21 In each RA4000/RA4100 Array, the bottom (rack model) or left rear (tower model) array controller is always the standby controller. The odd-numbered Fibre Channel SAN Switch (Fibre Channel SAN Switch #1) is connected to the active array controller in each RA4000/RA4100 Array.
Page 46: Figure 2-6. Active/Standby Configuration With One Ra4000/Ra4100 Array
2-22 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Standby Configuration with One RA4000/4100 Array Figure 2-6 shows an active/standby I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with one RA4000/RA4100 Array.
Page 47: Figure 2-7. Active/Standby Configuration With Two Ra4000/Ra4100 Arrays
Cluster Architecture 2-23 Active/Standby Configuration with Two RA4000/4100 Arrays Figure 2-7 shows an active/standby I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with two RA4000/RA4100 Arrays. Fibre Host Fibre Host ProLiant Adapter Adapter Servers (4) (Active) (Standby) Node 1...
Page 48 2-24 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Standby Configuration with Three RA4000/RA4100 Arrays Figure 2-8 shows an active/standby I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with three RA4000/RA4100 Arrays.
Page 49 Cluster Architecture 2-25 Active/Standby Configuration with Four RA4000/RA4100 Arrays Figure 2-9 shows an active/standby I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with four RA4000/RA4100 Arrays. Fibre Host Fibre Host ProLiant Adapter Adapter Servers (4) (Active) (Standby) Node 1...
Page 50 2-26 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Standby Configuration with Five RA4000/RA4100 Arrays Figure 2-10 shows an active/standby I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with five RA4000/RA4100 Arrays.
Page 51: Active/Active Configuration Examples For Redundant Fibre Channel Fabrics
Cluster Architecture 2-27 Active/Active Configuration Examples for Redundant Fibre Channel Fabrics This section describes examples of active/active configurations when two, three, four, and five RA4000/RA4100 Arrays are present in one redundant Fibre Channel Fabric of a four-node PDC/O2000 cluster. These examples represent one method for configuring active/active configurations.
Page 52 2-28 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide The even-numbered Fibre Channel SAN Switch (Fibre Channel SAN Switch #2) is connected to the active array controller in each even-numbered RA4000/RA4100 Array (2 and 4).
Page 53: Figure 2-11. Active/Active Configuration With Two Ra4000/Ra4100 Arrays
Cluster Architecture 2-29 Active/Active Configuration with Two RA4000/RA4100 Arrays Figure 2-11 shows an active/active I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with two RA4000/RA4100 Arrays. Fibre Host ProLiant Adapters (2) Servers (4) Node 1 Node 2 Node 3 Node 4...
Page 54: Figure 2-12. Active/Active Configuration With Three Ra4000/Ra4100 Arrays
2-30 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Active Configuration with Three RA4000/RA4100 Arrays Figure 2-12 shows an active/active I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with three RA4000/RA4100 Arrays.
Page 55: Figure 2-13. Active/Active Configuration With Four Ra4000/Ra4100 Arrays
Cluster Architecture 2-31 Active/Active Configuration with Four RA4000/RA4100 Arrays Figure 2-13 shows an active/active I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with four RA4000/RA4100 Arrays. Fibre Host ProLiant Adapters (2) Servers (4) Node 1 Node 2 Node 3 Node 4...
Page 56: Figure 2-14. Active/Active Configuration With Five Ra4000/Ra4100 Arrays
2-32 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Active Configuration with Five RA4000/RA4100 Arrays Figure 2-14 shows an active/active I/O path configuration for a redundant Fibre Channel Fabric in a four-node cluster with five RA4000/RA4100 Arrays.
Page 57: Summary Of I/O Path Failure And Failover Scenarios For Redundant Fibre Channel Fabrics
Cluster Architecture 2-33 Summary of I/O Path Failure and Failover Scenarios for Redundant Fibre Channel Fabrics Table 2-3 identifies possible I/O path failure events in redundant Fibre Channel Fabrics for active/standby configurations with one RA4000/RA4100 Array and the failover response, if any, implemented by Secure Path for each failure.
Page 58 2-34 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Table 2-3 I/O Path Failure and Failover Scenarios in Redundant Fibre Channel Fabrics for Active/Standby Configurations With One RA4000/RA4100 Array continued Description of Failure Failover Response...
Page 59 Cluster Architecture 2-35 Table 2-4 identifies possible I/O path failure events for active/standby configurations in redundant Fibre Channel Fabrics with two or more RA4000/RA4100 Arrays and the failover response, if any, implemented by Secure Path for each failure. Table 2-4 I/O Path Failure and Failover Scenarios in Redundant Fibre Channel Fabrics for Active/Standby Configurations With Two or More RA4000/RA4100 Arrays Description of Failure...
Page 60 2-36 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Table 2-4 I/O Path Failure and Failover Scenarios in Redundant Fibre Channel Fabrics for Active/Standby Configurations With Two or More RA4000/RA4100 Arrays continued Description of Failure...
Page 61 Cluster Architecture 2-37 Table 2-5 identifies possible I/O path failure events in redundant Fibre Channel Fabrics for active/active configurations with two or more RA4000/RA4100 Arrays and the failover response, if any, implemented by Secure Path for each failure. Table 2-5 I/O Path Failure and Failover Scenarios in Redundant Fibre Channel Fabrics for Active/Active Configurations With Two or More RA4000/RA4100 Arrays Description of Failure...
Page 62 2-38 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Table 2-5 I/O Path Failure and Failover Scenarios in Redundant Fibre Channel Fabrics for Active/Active Configurations With Two or More RA4000/RA4100 Arrays continued Description of Failure...
Page 63: I/O Path Configurations For Redundant Fibre Channel Arbitrated Loops
Cluster Architecture 2-39 I/O Path Configurations for Redundant Fibre Channel Arbitrated Loops Overview of FC-AL SAN Topology Fibre Channel standards define a multi-layered architecture for moving data across the storage area network (SAN). This layered architecture can be implemented using the Fibre Channel Fabric or the Fibre Channel Arbitrated Loop (FC-AL) topology.
Page 64: Redundant Fibre Channel Arbitrated Loops
2-40 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Redundant Fibre Channel Arbitrated Loops A redundant FC-AL refers to the redundant hardware implemented to connect Fibre Host Adapters to a particular set of shared storage devices in a PDC/O2000.
Page 65: Figure 2-15. Two-Node Pdc/O2000 With A Two-Loop Redundant Fibre Channel Arbitrated Loop
Cluster Architecture 2-41 Figure 2-15 shows a two-node PDC/O2000 with a redundant FC-AL that contains two loops, one for each Fibre Host Adapter per node. The components and cable paths for the first loop are shaded to distinguish them from the components for the other loop. Fibre Host Fibre Host Adapters (2)
Page 66: Multiple Redundant Fibre Channel Arbitrated Loops
2-42 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Multiple Redundant Fibre Channel Arbitrated Loops The PDC/O2000 supports the use of multiple redundant FC-ALs within the same cluster. You would install additional redundant FC-ALs in a PDC/O2000 to: Increase the amount of shared storage space available to the cluster’s...
Page 67: Figure 2-2. Two-Node Pdc/O2000 With Two Redundant Fibre Channel
Cluster Architecture 2-43 Figure 2-16 shows a two-node PDC/O2000 that contains two redundant FC-ALs. In this example, each redundant FC-AL has its own pair of Fibre Host Adapters in each node, a pair of Storage Hubs or FC-AL Switches, and two RA4000/RA4100 Arrays.
Page 68: Maximum Distances Between Nodes And Shared Storage Components In A Redundant Fc-Al
2-44 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Maximum Distances Between Nodes and Shared Storage Components in a Redundant FC-AL By using standard short-wave Fibre Channel cables with Gigabit Interface Converter-Shortwave (GBIC-SW) modules, the following maximum distances...
Page 69: I/O Data Paths In A Redundant Fc-Al
Cluster Architecture 2-45 I/O Data Paths in a Redundant FC-AL A distinct I/O path connection exists between each Fibre Host Adapter in a cluster node and every array controller port in each RA4000/RA4100 Array in the redundant FC-AL. Fibre Host Adapter-to-Storage Hub/Switch Paths Figure 2-18 highlights the I/O data paths that run between the Fibre Host Adapters and the Storage Hubs or FC-AL Switches in a redundant FC-AL.
Page 70: Figure 2-19. Fc-Al Switch/Storage Hub-To-Ra4000/Ra4100 Array Data Paths
2-46 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Storage Hub/Switch-to-Array Controller Paths Figure 2-19 highlights the I/O data paths that run between the Storage Hubs or FC-AL Switches and the two single-port RA4000 Array Controllers in each RA4000/RA4100 Array.
Page 71: I/O Path Configuration Guidelines For Redundant Fibre Channel Arbitrated Loops
Cluster Architecture 2-47 I/O Path Configuration Guidelines for Redundant Fibre Channel Arbitrated Loops You can use either of two I/O path configurations for every redundant FC-AL in your PDC/O2000 cluster: Active/standby configuration Active/active configuration In every redundant FC-AL, at least two Fibre Host Adapters must be installed in each node (one Fibre Host Adapter for each Storage Hub or FC-AL Switch in that redundant FC-AL).
Page 72: Table 2-6 Features Of Active/Standby And Active/Active Configurations For Redundant Fibre Channel Arbitrated Loops
2-48 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide See “Active/Standby Configuration Examples for Redundant FC-ALs” in this chapter for a detailed description of active/standby configuration examples for a redundant FC-AL with from one to five RA4000/RA4100 Arrays. See “Active/Active Configuration Examples for Redundant FC-ALs”...
Page 73 Cluster Architecture 2-49 Table 2-6 Features of Active/Standby and Active/Active Configurations for Redundant Fibre Channel Arbitrated Loops continued I/O Path Configuration Advantage Disadvantage Active/active with two or more Provides a small but measurable Does not provide true cabling RA4000/RA4100 Arrays improvement in I/O performance symmetry between Storage over the active/standby...
Page 74: I/O Path Configuration Rules For Redundant Fc-Als
2-50 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide I/O Path Configuration Rules for Redundant FC-ALs The following rules must be observed in I/O path configurations for PDC/O2000 clusters with redundant FC-ALs: Each redundant FC-AL must use either an active/standby or active/active configuration.
Page 75: Active/Standby Configuration Examples For Redundant Fc-Als
Cluster Architecture 2-51 Active/Standby Configuration Examples for Redundant FC-ALs This section describes examples of active/standby configurations when one, two, three, four, and five RA4000/RA4100 Arrays are present in one redundant FC-AL of a four-node PDC/O2000. These examples represent one method for configuring active/standby configurations. They are presented here to provide a relatively simple and consistent method for building active/standby configurations.
Page 76 2-52 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide In each RA4000/RA4100 Array, the bottom (rack model) or left rear (tower model) array controller is always the standby controller. The odd-numbered Storage Hub or FC-AL Switch (Storage Hub/FC-AL Switch #1) is connected to the active array controller in each RA4000/RA4100 Array.
Page 77: Figure 2-20. Active/Standby Configuration With One Ra4000/Ra4100 Array
Cluster Architecture 2-53 Active/Standby Configuration with One RA4000/4100 Array Figure 2-20 shows an active/standby I/O path configuration for a redundant FC-AL in a four-node cluster with one RA4000/RA4100 Array. Fibre Host Fibre Host ProLiant Adapter Adapter Servers (4) (Active) (Standby) Node 1 Node 2 Node 3...
Page 78: Figure 2-21. Active/Standby Configuration With Two Ra4000/Ra4100 Arrays
2-54 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Standby Configuration with Two RA4000/4100 Arrays Figure 2-21 shows an active/standby I/O path configuration for a redundant FC-AL in a four-node cluster with two RA4000/RA4100 Arrays.
Page 79: Figure 2-8. Active/Standby Configuration With Three Ra4000/Ra4100 Arrays
Cluster Architecture 2-55 Active/Standby Configuration with Three RA4000/RA4100 Arrays Figure 2-22 shows an active/standby I/O path configuration for a redundant FC-AL in a four-node cluster with three RA4000/RA4100 Arrays. Fibre Host Fibre Host ProLiant Adapter Adapter Servers (4) (Active) (Standby) Node 1 Node 2 Node 3...
Page 80: Figure 2-9. Active/Standby Configuration With Four Ra4000/Ra4100 Arrays
2-56 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Standby Configuration with Four RA4000/RA4100 Arrays Figure 2-23 shows an active/standby I/O path configuration for a redundant FC-AL in a four-node cluster with four RA4000/RA4100 Arrays.
Page 81: Figure 2-10. Active/Standby Configuration With Five Ra4000/Ra4100 Arrays
Cluster Architecture 2-57 Active/Standby Configuration with Five RA4000/RA4100 Arrays Figure 2-24 shows an active/standby I/O path configuration for a redundant FC-AL in a four-node cluster with five RA4000/RA4100 Arrays. Fibre Host Fibre Host ProLiant Adapter Adapter Servers (4) (Active) (Standby) Node 1 Node 2 Node 3...
Page 82: Active/Active Configuration Examples For Redundant Fc-Als
2-58 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Active Configuration Examples for Redundant FC-ALs This section describes examples of active/active configurations when two, three, four, and five RA4000/RA4100 Arrays are present in one redundant FC-AL of a four-node PDC/O2000 cluster.
Page 83 Cluster Architecture 2-59 The even-numbered Storage Hub or FC-AL Switch (Storage Hub/FC-AL Switch #2) is connected to the active array controller in each even-numbered RA4000/RA4100 Array (2 and 4). The even-numbered Storage Hub or FC-AL Switch is connected to the standby array controller in each odd-numbered RA4000/RA4100 Array (1, 3, and 5).
Page 84: Figure 2-25. Active/Active Configuration With Two Ra4000/Ra4100 Arrays
2-60 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Active Configuration with Two RA4000/RA4100 Arrays Figure 2-25 shows an active/active I/O path configuration for a redundant FC-AL in a four-node cluster with two RA4000/RA4100 Arrays.
Page 85: Figure 2-26. Active/Active Configuration With Three Ra4000/Ra4100 Arrays
Cluster Architecture 2-61 Active/Active Configuration with Three RA4000/RA4100 Arrays Figure 2-26 shows an active/active I/O path configuration for a redundant FC-AL in a four-node cluster with three RA4000/RA4100 Arrays. Fibre Host ProLiant Adapters (2) Servers (4) Node 1 Node 2 Node 3 Node 4 FC-AL...
Page 86: Figure 2-27. Active/Active Configuration With Four Ra4000/Ra4100 Arrays
2-62 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Active/Active Configuration with Four RA4000/RA4100 Arrays Figure 2-27 shows an active/active I/O path configuration for a redundant FC-AL in a four-node cluster with four RA4000/RA4100 Arrays.
Page 87: Figure 2-28. Active/Active Configuration With Five Ra4000/Ra4100 Arrays
Cluster Architecture 2-63 Active/Active Configuration with Five RA4000/RA4100 Arrays Figure 2-28 shows an active/active I/O path configuration for a redundant FC-AL in a four-node cluster with five RA4000/RA4100 Arrays. Fibre Host ProLiant Adapters (2) Servers (4) Node 1 Node 2 Node 3 Node 4 FC-AL...
Page 88: Summary Of I/O Path Failure And Failover Scenarios For Redundant Fc-Als
2-64 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Summary of I/O Path Failure and Failover Scenarios for Redundant FC-ALs Table 2-7 identifies possible I/O path failure events in redundant FC-ALs for active/standby configurations with one RA4000/RA4100 Array and the failover response, if any, implemented by Secure Path for each failure.
Page 89: Cluster Architecture
Cluster Architecture 2-65 Table 2-7 I/O Path Failure and Failover Scenarios in Redundant FC-ALs for Active/Standby Configurations With One RA4000/RA4100 Array continued Description of Failure Failover Response The Fibre Channel cable None connection between a Fibre Host Adapter and the Storage Hub or FC-AL Switch connected to the standby array controller is broken.
Page 90: Table 2-8 I/O Path Failure And Failover Scenarios In Redundant Fc-Als For
2-66 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Table 2-8 identifies possible I/O path failure events for active/standby configurations with two or more RA4000/RA4100 Arrays and the failover response, if any, implemented by Secure Path for each failure.
Page 91 Cluster Architecture 2-67 Table 2-8 I/O Path Failure and Failover Scenarios in Redundant FC-ALs for Active/Standby Configurations With Two or More RA4000/RA4100 Arrays continued Description of Failure Failover Response The Fibre Channel cable Secure Path forces a complete failover to the standby I/O path connection between the Storage components, including the standby Fibre Host Adapter in each Hub or FC-AL Switch that is...
Page 92: Table 2-9 I/O Path Failure And Failover Scenarios In Redundant Fc-Als For
2-68 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Table 2-9 identifies possible I/O path failure events in redundant FC-ALs for active/active configurations with two or more RA4000/RA4100 Arrays and the failover response, if any, implemented by Secure Path for each failure.
Page 93 Cluster Architecture 2-69 Table 2-9 I/O Path Failure and Failover Scenarios in Redundant FC-ALs for Active/Active Configurations With Two or More RA4000/RA4100 Arrays continued Description of Failure Failover Response A Storage Hub or FC-AL Switch Secure Path makes each active array controller to which the failed fails.
Page 94: Cluster Interconnect Options
2-70 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Cluster Interconnect Options The cluster interconnect is the data path over which all of the nodes in a cluster communicate. The nodes use the cluster interconnect data path to: Communicate individual resource and overall cluster status.
Page 95 Cluster Architecture 2-71 Non-Redundant Ethernet Cluster Interconnect Components The following components are used in a non-redundant Ethernet cluster interconnect: One Ethernet adapter in each cluster node Ethernet cables and a switch or hub For two-node PDC/O2000 clusters, you can either use one Ethernet crossover cable or one 100-Mbit/second Ethernet switch or hub and standard Ethernet cables to connect the two servers.
Page 96 2-72 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Ethernet Cluster Interconnect Adapters To implement the Ethernet cluster interconnect, each cluster node must be equipped with Ethernet adapters capable of 100-Mbit/second transfer rates. Some adapters may be capable of operating at both 10-Mbit/second and 100-Mbit/second;...
Page 97: Figure 2-29. Non-Redundant Ethernet Cluster Interconnect Using A Crossover Cable
Cluster Architecture 2-73 Ethernet Cluster Interconnect Diagrams Figure 2-29 shows the non-redundant Ethernet cluster interconnect components used in a two-node PDC/O2000 cluster. These components include a dual-port Ethernet adapter in each node. The top port on each adapter connects by Ethernet crossover cable to the top port on the adapter in the other node.
Page 98: Figure 2-30. Non-Redundant Ethernet Cluster Using An Ethernet Switch Or Hub
2-74 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Figure 2-30 shows another option for a non-redundant Ethernet cluster interconnect in a two-node PDC/O2000 cluster. These include an Ethernet adapter in each node connected by Ethernet cables to an Ethernet switch or hub.
Page 99: Figure 2-31. Redundant Ethernet Cluster Interconnect For A Two-Node Pdc/O2000 Cluster
Cluster Architecture 2-75 Figure 2-31 shows the redundant Ethernet cluster interconnect components used in a two-node PDC/O2000 cluster. Ethernet Switch/Hub #1 for Cluster Interconnect Ethernet Switch/Hub #2 Crossover for Cluster Interconnect Cable Dual-port Dual-port Ethernet Ethernet Adapters (2) Adapters (2) Node 1 Node 2 Crossover...
Page 100: Local Area Network
2-76 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Local Area Network NOTE: For the PDC/O2000, the client LAN and the cluster interconnect must be treated as separate networks. Do not use either network to handle the other network’s traffic.
Page 101: Cluster Software Components
Chapter Cluster Software Components Overview of the Cluster Software The Compaq Parallel Database Cluster Model PDC/O2000 (referred to here as the PDC/O2000) combines software from several leading computer vendors. The integration of these components creates a stable cluster management environment in which the Oracle database can operate.
Page 102: Compaq Software
3-2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Compaq Software Compaq offers an extensive set of features and optional tools to support effective configuration and management of the PDC/O2000: Compaq SmartStart™ and Support Software...
Page 103: Compaq System Configuration Utility
Support Software CD, is used to configure the hardware aspects of any disk drives attached to an array controller, including the non-shared drives in the servers and the shared drives in the Compaq StorageWorks RAID Array 4000s (RA4000 Arrays) or Compaq StorageWorks RAID Array 4100s (RA4100 Arrays).
Page 104: Compaq Insight Manager
3-4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Compaq Insight Manager Compaq Insight Manager, loaded from the Compaq Management CD, is a software utility used to collect information about the servers in the cluster.
Page 105: Compaq Storageworks Secure Path For Windows 2000
Guide. The OSD software is found on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD. This CD is supplied in the cluster kit for the Compaq Parallel Database Cluster Model PDC/O2000 for Windows 2000.
Page 106: Oracle Software
3-6 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Oracle Software The PDC/O2000 supports Oracle8i software. If you are using a release other than Oracle8i Release 8.1.7, confirm that the release has been certified for the PDC/O2000 on the Compaq website at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 107: Oracle8I Enterprise Manager
Oracle8i Parallel Server that does not strictly conform to the configurations certified with this process. For a complete list of certified Compaq servers, see the Compaq Parallel Database Cluster Model PDC/02000 Certification Matrix for Windows 2000 at...
Page 108: Application Failover And Reconnection Software
3-8 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Application Failover and Reconnection Software When a network client computer operates in a clustered environment, it must be more resilient than when operating with a stand-alone server. Because a...
Page 109: Cluster Planning
Chapter Cluster Planning Before connecting any cables or powering on any hardware on your Compaq Parallel Database Cluster Model PDC/O2000 (referred to here as the PDC/O2000), it is important that you understand how all the various cluster components fit together to meet your operational requirements. The major...
Page 110: Site Planning
4-2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Site Planning You must carefully select and prepare the site to ensure a smooth installation and a safe and efficient work environment. To select and prepare a location for...
Page 111: Capacity Planning For Cluster Hardware
NOTE: Certain restrictions apply to the server models and server configurations that are supported by the Compaq Parallel Database Cluster. For a current list of PDC/O2000-certified servers and details on supported configurations, refer to the Compaq Parallel Database Cluster Model PDC/O2000 Certification Matrix for Windows 2000. This document is available on the Compaq website at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 112 4-4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide One or more Compaq StorageWorks RAID Array 4000s (RA4000 Arrays) or Compaq StorageWorks RAID Array 4100s (RA4100 Arrays) Two single-port Compaq StorageWorks RAID Array 4000 Controllers (RA4000 Array Controllers) installed in each RA4000/RA4100 Array.
Page 113: Planning Shared Storage Components For Redundant Fibre Channel Arbitrated Loops
PDC/O2000. Each redundant Fibre Channel Arbitrated Loop (FC-AL) in a PDC/O2000 uses these hardware components: Two or more Fibre Host Adapters in each node Two or more Compaq StorageWorks Storage Hubs (Storage Hubs) or Compaq StorageWorks FC-AL Switches (FC-AL Switches) One or more RA4000 Arrays or RA4100 Arrays Two single-port RA4000 Array Controllers installed in each RA4000/RA4100 Array.
Page 114: Planning Cluster Interconnect And Client Lan Components
4-6 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Planning Cluster Interconnect and Client LAN Components PDC/O2000 clusters running Oracle8i Parallel Server can use a redundant or non-redundant Ethernet cluster interconnect. Redundant cluster interconnects are recommended because they provide fault tolerance along the entire cluster interconnect path.
Page 115: Planning Cluster Configurations For Redundant Fibre Channel Fabrics
Cluster Planning 4-7 Planning Cluster Configurations for Redundant Fibre Channel Fabrics Once you have investigated your requirements with respect to particular parts of the cluster (ProLiant servers, shared storage components, cluster interconnect components, client LAN components), you need to plan the configuration of the entire PDC/O2000.
Page 116 4-8 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide The small redundant Fibre Channel Fabric cluster configuration shown in Figure 4-1 contains these key cluster components: Two ProLiant servers (nodes) Two Fibre Host Adapters in each server Two Fibre Channel SAN Switches One RA4000/RA4100 Array.
Page 117: Sample Large Configuration In A Redundant Fibre Channel Fabric
Cluster Planning 4-9 Sample Large Configuration in a Redundant Fibre Channel Fabric Figure 4-2 shows an example of a larger PDC/O2000 cluster: a six-node cluster with five RA4000/RA4100 Arrays in one redundant Fibre Channel Fabric. Fibre Host Adapters (2) Node 5 Node 6 Node 1 Node 2...
Page 118 4-10 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide The larger redundant Fibre Channel Fabric configuration shown in Figure 4-2 contains these key cluster components: Six ProLiant servers (nodes) Two Fibre Host Adapters in each server node Two Fibre Channel SAN Switches Five RA4000/RA4100 Arrays.
Page 119: Cluster Planning
Cluster Planning 4-11 Planning Cluster Configurations for Redundant Fibre Channel Arbitrated Loops Once you have investigated your requirements with respect to particular parts of the cluster (ProLiant servers, shared storage components, cluster interconnect components, client LAN components), you need to plan the configuration of the entire PDC/O2000.
Page 120 4-12 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide The small redundant FC-AL configuration shown in Figure 4-3 contains these key cluster components: Two ProLiant servers (nodes) Two Fibre Host Adapters in each server Two Storage Hubs or FC-AL Switches One RA4000/RA4100 Array.
Page 121: Sample Large Configuration In A Redundant Fc-Al
Cluster Planning 4-13 Sample Large Configuration in a Redundant FC-AL Figure 4-4 shows an example of a larger PDC/O2000 cluster: a six-node cluster with five RA4000/RA4100 Arrays in one redundant FC-AL. Fibre Host Adapters (2) Node 5 Node 6 Node 1 Node 2 Node 3 Node 4...
Page 122: Raid Planning
4-14 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide The larger redundant FC-AL configuration shown in Figure 4-4 contains these key cluster components: Six ProLiant servers (nodes) Two Fibre Host Adapters in each server node Two Storage Hub 12s or 11-port FC-AL Switches Five RA4000/RA4100 Arrays.
Page 123 RAID. When you implement RAID on shared storage arrays, you use the hardware RAID to perform such functions as making copies of the data or calculating checksums. Use the Compaq Array Configuration Utility to implement RAID on your logical disks.
Page 124: Supported Raid Levels
4-16 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Supported RAID Levels RAID provides several fault-tolerant options to protect your cluster’s shared data. However, each RAID level offers a different mix of performance, reliability, and cost.
Page 125: Raw Data Storage And Database Size
Cluster Planning 4-17 Raw Data Storage and Database Size Raw data storage is the amount of storage available before any RAID levels have been configured. It is called raw data storage because RAID volumes require some overhead. The maximum size of a database stored in a RAID system will always be less than the amount of raw data storage available.
Page 126: Planning The Grouping Of Physical Disk Storage Space
4-18 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide The following general guidelines apply to RAID selection for a cluster with RA4000/RA4100 Arrays using Oracle8i Parallel Server: Oracle recommends that some form of disk fault tolerance be implemented in the cluster.
Page 127: Disk Drive Planning
“logical drives” in the Oracle documentation.) IMPORTANT: To performing partitioning, all required drivers must already be installed for each server. For information about drivers, refer to the Compaq Parallel Database Cluster Model PDC/O2000 Certification Matrix for Windows 2000.
Page 128: Shared Disk Drives
4-20 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Shared Disk Drives Shared disk drives are contained in the RA4000/RA4100 Arrays and are accessible to each node in the PDC/O2000. If a logical drive is configured with a RAID level that does not support fault...
Page 129: Client Lan
Cluster Planning 4-21 Client LAN Physically, the structure of the client network is no different than that used for a nonclustered configuration. To ensure continued access to the database when a cluster node is evicted from the cluster, each network client should have physical network access to all of the cluster nodes.
Page 130: Installation And Configuration
Chapter Installation and Configuration This chapter provides instructions for installing and configuring the Compaq Parallel Database Cluster Model PDC/O2000 (referred to here as the PDC/O2000) for use with Oracle8i software. A PDC/O2000 is a combination of several individually available products. As you set up your cluster, have the following materials available during installation.
Page 131: Installation Overview
5-2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Compaq StorageWorks Fibre Channel SAN Switch Management Guide Compaq SmartStart Installation poster Compaq SmartStart and Support Software CD Microsoft Windows 2000 Advanced Server Administrator’s Guide Microsoft Windows 2000 Advanced Server CD with Service Pack 1 or...
Page 132 Installing the Secure Path Client software Verifying Shared Storage Using Secure Path Manager Defining Active Array controllers Installing and configuring the Compaq operating system dependent modules (OSDs), including: Using Oracle Universal Installer to install OSDs for an Ethernet cluster interconnect...
Page 133: Installing The Hardware
5-4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Installing the Hardware Setting Up the Nodes Physically preparing the nodes (servers) for a cluster is not very different than preparing them for individual use. You will install all necessary adapters and insert all internal hard disks.
Page 134: Installing Gbic-Sw Modules For The Fibre Host Adapters
Installation and Configuration 5-5 Installing GBIC-SW Modules for the Fibre Host Adapters Each Fibre Host Adapter ships with two GBIC-SW modules. Verify that one module is installed in the Fibre Host Adapter and the other in its Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch. Each end of the Fibre Channel cable connecting a Fibre Host Adapter to a Storage Hub, FC-AL Switch, or Fibre Channel SAN Switch plugs into a GBIC-SW module.
Page 135: Installing The Cluster Interconnect Adapters
5-6 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-1 shows the Fibre Host Adapters in two servers connected to two Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches. Fibre Host Fibre Host...
Page 136: Installing The Client Lan Adapters
ProLiant server you are using. Setting Up the RA4000/RA4100 Arrays Unless otherwise indicated in this guide, follow the instructions in the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide o set up shared storage subsystem components.
Page 137: Figure 5-2. Ra4000/Ra4100 Arrays Connected To Clustered Servers Through One Redundant Fibre Channel Fabric Or Redundant Fc-Al
5-8 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-2 shows two RA4000/RA4100 Arrays connected to two clustered servers through one redundant Fibre Channel Fabric or redundant FC-AL. The Fibre Channel Fabric or FC-AL is redundant because there are two paths from each node to each RA4000/RA4100 Array.
Page 138: Installing Gbic-Sw Modules For The Ra4000 Array Controllers
Installation and Configuration 5-9 For detailed information about configuring the drives using the Array Configuration Utility, see the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide. For information about configuring your shared storage subsystem with RAID, see Chapter 4, “Cluster Planning.”...
Page 139 5-10 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide In an active/active configuration, both Fibre Host Adapters in each node are connected to an active array controller. The number of array controllers each Fibre Host Adapter is connected to depends on the number of RA4000/RA4100 Arrays in the redundant Fibre Channel Fabric or redundant FC-AL.
Page 140: Switches To Ra4000 Array Controllers In An Active/Standby Configuration
Installation and Configuration 5-11 Figure 5-3 shows two Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches connected to the RA4000 Array Controllers in two RA4000/RA4100 Arrays. Storage Storage Hub/Switch #1 Hub/Switch #2 Active Standby RA4000/4100 RA4000/4100 Array Controller Array Controller Array #1 Array #2 Figure 5-3.
Page 141: Table 5-1 Active/Active Cabling Methods
5-12 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Table 5-1 summarizes the active/active cabling methods. Select the method that is best for your site. Table 5-1 Active/Active Cabling Methods Active Array Method Controller Advantage...
Page 142: Figure 5-4. Method 1: Cabling An Active/Active Configuration With Two Ra4000/Ra4100 Arrays
Installation and Configuration 5-13 Figure 5-4 shows an example of using method 1 cabling to connect two Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches to RA4000 Array Controllers in two RA4000/RA4100 Arrays. Storage Storage Hub/Switch #1 Hub/Switch #2 Active Standby RA4000/4100...
Page 143: Figure 5-5. Method 2: Cabling An Active/Active Configuration With Two Ra4000/Ra4100 Arrays
5-14 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Figure 5-5 shows an example of using method 2 cabling to connect two Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches to RA4000 Array Controllers in two RA4000/RA4100 Arrays.
Page 144: Installing Additional Redundant Fibre Channel Fabrics
Installation and Configuration 5-15 Installing Additional Redundant Fibre Channel Fabrics At this point, you have installed the hardware for one redundant Fibre Channel Fabric or one redundant FC-AL. To add a second redundant Fibre Channel Fabric to the PDC/O2000, install another set of the hardware required for one redundant Fibre Channel Fabric, including: Two or more Fibre Host Adapters in each server Two or more Fibre Channel SAN Switches...
Page 145: Cabling The Ethernet Cluster Interconnect
5-16 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Cabling the Ethernet Cluster Interconnect A PDC/O2000 running Oracle8i software uses an Ethernet cluster interconnect. The following components are used in a non-redundant Ethernet cluster interconnect:...
Page 146: Figure 5-6. Non-Redundant Ethernet Cluster Interconnect Using A Crossover Cable
Installation and Configuration 5-17 Figure 5-6 shows the non-redundant Ethernet cluster interconnect components used in a two-node PDC/O2000 cluster. These components include a dual-port Ethernet adapter in each node. The top port on each adapter connects by Ethernet crossover cable to the top port on the adapter in the other node. The bottom port on each adapter connects by Ethernet cable to the client LAN switch or hub.
Page 147: Figure 5-7. Non-Redundant Ethernet Cluster Interconnect Using An Ethernet Switch Or Hub
5-18 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide To connect two or more nodes in a non-redundant Ethernet cluster interconnect: 1. Install a standard Ethernet cable between an Ethernet adapter in one node and one Ethernet hub or switch.
Page 148: Figure 5-8. Redundant Ethernet Cluster Interconnect For A Two-Node Pdc/O2000 Cluster
Ethernet cables to connect the nodes. For more information on configuring Ethernet connections in a redundant or non-redundant cluster interconnect, including enabling failover from one Ethernet path to another, see Supported Ethernet Interconnects for Compaq Parallel Database Clusters Using Oracle Parallel Server at www.compaq.com/support/techpubs/whitepapers...
Page 149: Cabling The Client Lan
5-20 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Cabling the Client LAN You can use any TCP/IP network to connect to a client LAN. The following procedure contains instructions for cabling an Ethernet client LAN.
Page 150: Installing The Operating System Software And Configuring The Ra4000/Ra4100 Arrays
Installation and Configuration 5-21 Installing the Operating System Software and Configuring the RA4000/RA4100 Arrays You will follow an automated procedure using Compaq SmartStart to install the operating system software and configure the shared storage disks on the RA4000/RA4100 Arrays. Guidelines for Clusters Installing clustering software requires several specific steps and guidelines that might not be necessary when installing software on a single server.
Page 151: Automated Installation Using Smartstart
5-22 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide When you set up an Ethernet cluster interconnect, be sure to select TCP/IP as the network protocol. The Ethernet cluster interconnect should be on its own subnet.
Page 152 Installation and Configuration 5-23 The one difference is that through the Array Configuration Utility, SmartStart gives you the opportunity to configure the shared drives on all servers. For cluster configuration, you should configure the drives on the first server, then accept the same settings for the shared drives when given the option on the other servers.
Page 153 5-24 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 9. Enter unique IP addresses and node names for each node in the hosts and lmhosts files located at %SystemRoot%\system32\drivers\etc. Record this information: For an Ethernet cluster interconnect, one IP address and node name is for the redundant cluster interconnect, and the other IP address and node name is for the client LAN.
Page 154 IMPORTANT: Record the cluster interconnect node name, the client LAN node name, and the IP addresses assigned to them. You will need this information later when installing Compaq OSDs. 12. If you are installing node1, open Windows 2000 Advanced Server Disk Management to create extended disk partitions and logical partitions within the extended partitions on all RA4000/RA4100 Arrays.
Page 155: Installing Secure Path Software For Windows 2000
5-26 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Installing Secure Path Software for Windows 2000 The Secure Path software for Windows 2000 is installed on both the server and the client. The Secure Path server software is installed on the Windows host system to which the RAID storage system is connected.
Page 156: Installing The Secure Path Client Software
Secure Path Server software or any Windows workstation. The workstation must be TCP/IP-capable. To install the Secure Path Client software: 1. Insert the Compaq SANWorks Secure Path Software CD into your CD-ROM drive. 2. If you have AutoRun enabled on your server, the Secure Path setup program will start automatically.
Page 157: Defining Active Array Controllers
5-28 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 3. If you double-click “Storage System X”, you should get two more branches that represent the two RA4000 Array Controllers in that array. These controllers are named “ControllerA” and “ControllerB,” which correspond to the Active controller and the Standby controller, respectively.
Page 158: Table 5-2 Active Array Controller Locations
Installation and Configuration 5-29 Table 5-2 Active Array Controller Locations Configuration Active Array Controller Location Active/Standby Top array controller slot Active/Active: Cabling Method 1 Top array controller slot Active/Active: Cabling Method 2 Top array controller slot in odd-numbered RA4000/RA4100 Arrays Bottom array controller slot in even-numbered RA4000/RA4100 Arrays NOTE: If Secure Path Manager indicates that the active array controllers are already...
Page 159: Installing Compaq Osds
5-30 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 1. On any cluster node, click Start, Programs. Under Secure Path, click SPM. The Secure Path Manager (Fibre Channel) screen is displayed. 2. After clicking on the logical drives (LUNS) you would like to move to the other array controller, right-click the first controller and a pop-up message appears that states “Move to Other Controller.”...
Page 160: Verifying Cluster Communications
If the OSDs do not install successfully, the OUI will not offer the option to install Oracle8i Parallel Server. The Compaq operating system dependent modules (OSDs) are installed from one node in the cluster (the installing node). The node on which you run the OUI program must be able to communicate with every other node in the cluster in order to copy information to all other cluster nodes.
Page 161: Mounting Remote Drives And Verifying Administrator Privileges
5-32 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide For example, the command ping node1 should return the following type of information. Pinging node1.loc1.yoursite.com [127.0.0.1] with 32 bytes of data: Reply from 127.0.0.1: bytes=32 time<10ms TTL=128 Reply from 127.0.0.1: bytes=32 time<10ms TTL=128...
Page 162: Installing The Ethernet Osds
Then make that local user a member of the local Administrators group on each node before installing the OSDs. 2. Insert the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD into the CD-ROM drive.
Page 163 5-34 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 3. If autorun is enabled, the Oracle Universal Installer Welcome screen appears. If autorun is disabled, enter d:\setup at a C: prompt. Substitute the letter assigned to the CD drive for d. The Oracle Universal Installer Welcome screen appears.
Page 164 Installation and Configuration 5-35 4. Click Next to continue. The License screen appears. 5. Click Accept to accept the terms of the license agreement. The File Locations screen appears.
Page 165 5-36 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 6. Click Next to continue. The Installation Types screen appears.
Page 166 Installation and Configuration 5-37 7. Select Typical to install the Ethernet OSDs into the default location, C:\Compaq\OPS. Skip to step 12. 8. Select Custom to install the Ethernet OSDs into a location of your choice. The Component Locations screen appears.
Page 167 5-38 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 11. If you want to specify an alternate location but do not know the exact directory path, click Change Location. The Choose Directory screen appears.
Page 168 Installation and Configuration 5-39 12. Browse to the location where you want to install the OSDs. Click OK. The Component Locations screen appears indicating the alternate location you specified.
Page 169 5-40 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 13. Click Next to continue. The Cluster Members screen appears. 14. The Cluster Members screen indicates the host name of the installing machine. Define the node names for the client LAN by adding the node names you specified in the hosts and lmhosts files for the client LAN for all other cluster nodes.
Page 170 Installation and Configuration 5-41 15. Click Next to continue. The Cluster Members screen reappears. 16. The Cluster Members screen now shows the node names you specified on the first Cluster Members screen with “_san” appended to each name, indicating it as a cluster interconnect node name. If necessary, retype the cluster interconnect node names to match the names you entered into the hosts and lmhosts files for the Ethernet cluster interconnect.
Page 171 5-42 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 17. Click Next to continue. The OSD Summary screen appears.
Page 172 Installation and Configuration 5-43 18. If the information on the OSD Summary screen is not correct, click Previous to modify the previous screens. When the information is correct, click Next to continue. The Summary screen appears. 19. Click Install to install the components listed in the summary. At this point, the OUI verifies that the installing node can communicate with other cluster nodes through both the client LAN and cluster interconnect node names.
Page 173: Installing Oracle Software
5-44 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 21. When the installation completes, the End of Installation screen appears. 22. Click Exit to quit the OUI or Next Install to install another product. 23. Start the cluster manager, OracleCMService, on all nodes in the cluster.
Page 174: Configuring Oracle Software
Installation and Configuration 5-45 Oracle8i Parallel Server Manager Oracle8i Enterprise Manager NOTE: If the Oracle Universal Installer does not offer the option to install Oracle8i Parallel Server, it could mean that the OSDs did not install successfully or that the OracleCMService is not running.
Page 175: Additional Notes On Configuring Oracle Software
5-46 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Object Link Manager simplifies the creation and maintenance of symbolic links between disk partitions and Oracle data files by placing the symbolic link names directly into the disk partitions, thereby tracking the symbolic links dynamically.
Page 176: Verifying The Hardware And Software Installation
If the Oracle service did not start successfully, make sure that the OracleCMService is started and the SID is in the Oracle registry. There are two logs that can provide information about why the Oracle service did not start. These logs are: \Compaq\OPS\cmsrvr.log \Compaq\OPS\nm.log...
Page 177: Other Verification Tasks
5-48 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Other Verification Tasks Perform these other verification tasks: Verify that Object Link Manager has been installed and is running. Verify that the OPS_CMDISK symbolic link has actually been set.
Page 178: Server Power Distribution
Installation and Configuration 5-49 Server Power Distribution Figure 5-9 shows an example of server power distribution for a three-node cluster. Power supply #1 of each server is connected to PDU #1. Power supply #2 of each server is connected to PDU #2. PDU #1 is connected to UPS #1, and PDU #2 is connected to UPS #2.
Page 179: Power Sequencing
5-50 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Power Sequencing Be sure to power up the cluster components in the following order: 1. RA4000/RA4100 Arrays 2. Storage Hubs, FC-AL Switches, or Fibre Channel SAN Switches (Power is applied to Storage Hub when the AC power cord is plugged in.)
Page 180: Cluster Management
This chapter describes these management activities for the Compaq Parallel Database Cluster Model PDC/O2000 (referred to here as the PDC/O2000). The topics addressed in this chapter include:...
Page 181: Cluster Management Concepts
6-2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Cluster Management Concepts Powering Off a Node Without Interrupting Cluster Services At some time during the life of your cluster you will need to perform an operation on a cluster node that will require it to be powered off.
Page 182: Managing Network Clients Connected To A Cluster
Cluster Management 6-3 4. Determine how critical it is to repair the problem. If the problem is not considered to be critical, wait until a non-peak time to service the problem. If the problem is critical but does not affect all cluster nodes, shut down the Oracle instances on the affected nodes and wait until a non-peak time to service the problem.
Page 183: Management Applications
6-4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide The following log files are used: The Node Manager (NM), provided by Compaq, logs messages to C:\compaq\ops\nm.log By default the OracleCMService sends messages to C:\compaq\ops\cmsrvr.log Management Applications...
Page 184: Managing Shared Drives
The ACU is available on the SmartStart and Support Software CD, which is found in the Compaq Server Setup and Management pack. For additional information about the ACU, see the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide.
Page 185: Monitoring Redundant Fibre Channel Arbitrated Loops
6-6 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Compaq Secure Path software can detect failures in redundant Fibre Channel Fabric components. When a failure occurs on an active path, Secure Path reroutes I/O through the standby path, allowing applications to continue processing.
Page 186: Monitoring The Database
Oracle8i Enterprise Edition. Remotely Managing a Cluster Oracle8i Enterprise Manager, Compaq Insight Manager, and Compaq Insight Manager XE can be run from any network client machine that has network access to the cluster nodes. They can operate on Windows NT Server,...
Page 187: Software Maintenance For Oracle8I
6-8 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Before installing Compaq Insight Manager, Compaq Insight Manager XE, or Oracle8i Enterprise Manager, it is recommended that you read through the corresponding documentation to determine how to set up and configure each of these programs to run remotely.
Page 188 Cluster Management 6-9 4. If autorun is disabled, enter at a prompt. Substitute the letter d:\setup assigned to the CD-ROM drive for The Oracle Universal Installer Welcome screen appears.
Page 189 6-10 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 5. Click Deinstall Products. Next, a window containing an inventory of the installed components appears. The following shows the Inventory screen that is displayed when Ethernet is used for the cluster interconnect.
Page 190: Upgrading Oracle8I Server
Significant changes in Oracle8i Server will likely affect the operation of the cluster. Before upgrading to any new release of Oracle8i Server, consult your Compaq and Oracle service representatives. Make sure that the new release is certified and supported on the PDC/O2000, and get assistance to determine which procedure you should follow to perform the upgrade properly.
Page 191 6-12 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Upgrading the Ethernet OSDs To upgrade the Ethernet OSDs: 1. Insert the CD containing the new revision of the OSDs into the CD-ROM drive on the primary node. (This is the node that was used for the previous installation of the OSDs.)
Page 192: Deinstalling A Partial Osd Installation
Deinstalling a Partial OSD Installation While not a common occurrence, the Oracle Universal Installer might not be able to complete the installation of the Compaq Ethernet OSDs. An incomplete OSD installation could result for various reasons, for example: A client LAN cable is no longer seated tightly into its connector.
Page 193: Upgrading Oracle8I Server
6-14 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Upgrading Oracle8i Server The design of the PDC/O2000 is tightly integrated with Oracle8i Server. Significant changes in Oracle8i Server will likely affect the operation of the cluster.
Page 194: Adding Disk Drives To Increase Storage Capacity
Cluster Management 6-15 Adding Disk Drives to Increase Storage Capacity During the life of your PDC/ O 2000 cluster, you might need to expand the capacity of an RA4000 Array or RA4100 Array. The following steps describe how to add drives to an RA4000/RA4100 Array and how to allocate the added disk capacity to Oracle.
Page 195: Replacing A Failed Fibre Host Adapter
6-16 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 5. Insert drives into the added RA4000/RA4100 Array. 6. Using the Fibre Channel cables, physically connect the RA4000 Array Controllers in the added RA4000/RA4100 Array to the Fibre Channel SAN Switches, FC-AL Switches, or Storage Hubs.
Page 196: Cluster Management
Replacing a Cluster Node Before replacing a node, verify that the new configuration is supported by the PDC/O2000. To determine if your new server is supported, see the Compaq Parallel Database Cluster Model PDC/O2000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 197: Adding The Replacement Node
6-18 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 4. Stop all Oracle services on the node to be removed. 5. Shut down the server on that node and power it off. 6. Disconnect the Fibre Channel cables between the evicted node and the Fibre Channel SAN Switches, FC-AL Switches, or Storage Hubs.
Page 198 5. Install and configure Windows 2000 Advanced Server with Service Pack 1 or later (whichever is installed on the other cluster nodes). 6. Install necessary Compaq drivers and utilities from the Compaq SmartStart and Support Software CD. 7. Install Fibre Host Adapters in the replacement node in the same slot locations used in the other cluster nodes.
Page 199 6-20 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide 1. To add a node to the cluster, you must disassemble the existing cluster and then reform it to include the replacement node. Do the following on all original cluster nodes (not the replacement node): a.
Page 200: Adding A Cluster Node
Cluster Management 6-21 Adding a Cluster Node Before adding a node, verify the new configuration is supported by the PDC/O2000. To determine if your new server is supported, see the Compaq Parallel Database Cluster Model PDC/O2000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 201: Preparing The New Node
6-22 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Preparing the New Node To minimize downtime of the cluster, several preparation steps should be performed on the new node before it is integrated into the cluster.
Page 202: Preparing The Existing Cluster Nodes
Cluster Management 6-23 10. Restart the new node. 11. Verify TCP/IP connectivity between the replacement node and the existing cluster nodes. Run the ping utility from the replacement node, pinging the client LAN adapters on all other nodes. If Ethernet is used as the cluster interconnect, also ping the cluster interconnect adapters of all other nodes.
Page 203 6-24 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide To install the cluster software, perform the following steps: 1. To add a node to the cluster, you must disassemble the existing cluster and then reform it to include the new node. Do the following on all original cluster nodes (not the new node): a.
Page 204: Monitoring Cluster Operation
Compaq Insight Manager and Compaq Insight Manager XE can be used to gather low-level, in-depth hardware statistics for the server and networking hardware components used in the cluster.
Page 205 6-26 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Secure Path Manager is the cluster management component of Secure Path. Secure Path Manager provides a Graphical User Interface (GUI) utility that: Reports the status of the I/O paths in each redundant FC-AL or redundant Fibre Channel Fabric.
Page 206 Repair an I/O path. Use this action when you want Secure Path Manager to restore access to a failed path after the problem has been corrected. Refer to the Compaq documentation provided with the Secure Path software for details on performing these operations.
Page 207: Uninstalling Secure Path
6-28 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Note that Secure Path Manager client is not required to be running in order for Secure Path to protect path availability. The RaiDisk device driver running on cluster nodes handles Secure Path’s protection capability.
Page 208: Troubleshooting
Chapter Troubleshooting This chapter addresses potential problems you might encounter as you install, configure, test, and operate your Compaq Parallel Database Cluster Model PDC/O2000 (referred to here as the PDC/O2000). This chapter is organized as follows: Basic troubleshooting tips Troubleshooting Oracle8i and operating system dependent module...
Page 209: Basic Troubleshooting Tips
7-2 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Basic Troubleshooting Tips Power Verify that the following are all powered on: Cluster nodes StorageWorks RAID Array 4000s (RA4000 Arrays) or StorageWorks RAID Array 4100s (RA4100 Arrays)
Page 210: Access To Cluster Components
Oracle8i Server Release 8.1.7 with Oracle8i Parallel Server Option Release 8.1.7 RA4000 Array Controller drivers Compaq Insight Manager Compaq Array Configuration Utility Compaq Secure Path for Windows 2000 Compaq Ethernet OSDs Cluster interconnect drivers Microsoft Windows 2000 Advanced Server, including Service Pack 1 or later...
Page 211: Firmware Revisions
7-4 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide For information about currently supported software revisions for the PDC/O2000, refer to the Compaq Parallel Database Cluster Model PDC/O2000 Certification Matrix for Windows 2000 at www.compaq.com/solutions/enterprise/ha-pdc.html...
Page 212: Troubleshooting Oracle8I And Osd Installation Problems And Error Messages
Troubleshooting 7-5 Troubleshooting Oracle8i and OSD Installation Problems and Error Messages Potential Difficulties Installing the OSDs with the Oracle Universal Installer The Oracle Universal Installer (OUI) uses an inventory file to track which components of the OSDs have been installed on each of the cluster nodes.
Page 213: Unable To Start Oraclecmservice
7-6 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Determine whether or not you need to deinstall a partial OSD installation. An incomplete installation can result in a partial installation of the OSDs on the cluster. If this occurs, you must clean up after the partial installation (by removing it) before you can successfully run the OUI again.
Page 214: Unable To Start Oraclenmservice
Verify that the OracleNMService has properly started by checking the error log. The default location for the error log is C:\compaq\ops\nm.log. If not found in the default location, look in the same directory in which the OSD modules were installed.
Page 215: Initialization Of The Dynamic Link Library Nm.dll Failed
7-8 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Initialization of the Dynamic Link Library NM.DLL Failed If this failure occurs, stop the currently running OracleCMService and OracleNMService. Restart the OracleNMService, then the OracleCMService. From Administrative Tools, open the Services screen. Locate each service in the displayed list, highlight it, then stop it.
Page 216: Unable To Ping The Cluster Interconnect Or The Client Lan
Troubleshooting 7-9 Unable to Ping the Cluster Interconnect or the Client LAN If you are using the recommended method of using static IP addresses, ensure that the hosts file and lmhosts files are properly set up. When Using the Ethernet Cluster Interconnect Make sure that the Ethernet cables are properly connected and that the Ethernet switches or hubs are powered on.
Page 217: Troubleshooting Client-To-Cluster Connectivity Problems
7-10 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Make sure the OracleCMService and all other Oracle services have started without error. From the Services screen, scan the list of services and check to see if the Oracle services have started. From the Application log, check for error messages about the Oracle services.
Page 218: Troubleshooting Shared Storage Problems
This section does not address RA4000/RA4100 Array problems that are specific to the product itself or to the use of the RA4000/RA4100 Array in a stand-alone server configuration. For those issues, refer to the Compaq StorageWorks RAID Array 4000 User Guide or the Compaq StorageWorks RAID Array 4100 User Guide.
Page 219: Verifying Connectivity To A Redundant Fibre Channel Arbitrated Loop
7-12 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Verifying Connectivity to a Redundant Fibre Channel Arbitrated Loop To ensure that all I/O path connections in the redundant FC-AL are intact and functioning, verify the following: For each redundant FC-AL, at least two Fibre Host Adapters have been installed in the same PCI slots across all servers in the cluster.
Page 220 Troubleshooting 7-13 Determine whether or not the Compaq servers support the physical SCSI disks installed in the RA4000/RA4100 Arrays. Check the event log for disk I/O error messages or indications of problems with the communications transport. If the drives are still not recognized, it is recommended that you start investigating the problem from the lowest level of drive configuration.
Page 221: A Cluster Node Cannot Connect To The Shared Drives
7-14 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide A Cluster Node Cannot Connect to the Shared Drives Verify that a physical connection exists from each node to each Fibre Channel SAN Switch, FC-AL Switch, or Storage Hub.
Page 222: Secure Path Manager Cannot Start With Hosts That Use Hyphenated Host Names
Troubleshooting 7-15 The same Secure Path Manager password must be used for all of the hosts in a profile. Before you change the Secure Path Agent’s list of authorized clients or passwords, you must first manually stop the Secure Path Agent. Be sure to restart the Secure Path Agent after the changes have been completed.
Page 223: Secure Path Manager Is Delayed In Reporting Path Failure Information
7-16 Compaq Parallel Database Cluster Model PDC/02000 for Oracle8i and Windows 2000 Administrator Guide Secure Path Manager Is Delayed In Reporting Path Failure Information Due to extended error checking features present in the Secure Path software, there may be some delay with Secure Path Manager in reporting failed path information.
Page 224: Diagnosing And Resolving Shared Disk Problems
Diagnosing and Resolving Shared Disk Problems Introduction When one or more nodes in a Compaq Parallel Database Cluster Model PDC/O2000 (referred to here as the PDC/O2000) cannot properly join the cluster, you need to consider two possible causes: A problem related to the shared disk storage in the cluster’s Compaq...
Page 225: Figure A-1. Tasks For Diagnosing And Resolving Shared Storage Problems
A-2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Figure A-1 identifies, in a flowchart format, the sequence of tasks you should perform to diagnose, isolate, and resolve shared disk problems in a PDC/O2000. A detailed description of each task is provided in the following sections of this appendix.
Page 226: Run Object Link Manager On All Nodes
“Restart All Affected Nodes in the Cluster” in this appendix for further instructions. For instructions on installing and using Object Link Manager, see the readme file: \Object Link Manager\readme.txt This file is located on the Compaq Parallel Database Cluster Clustering Software for Oracle8i on Microsoft Windows 2000 CD.
Page 227: Restart All Affected Nodes In The Cluster
A-4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Restart All Affected Nodes in the Cluster If Object Link Manager does not run properly on one or more nodes or does not provide consistent information about Oracle partitions and symbolic links from all nodes, perform these steps: 1.
Page 228: Run And Validate Secure Path Manager On All Nodes
Secure Path Manager screens. 2. Verify that the version number for Secure Path Manager is correct. Refer to the Compaq Parallel Database Cluster Model PDC/O2000 Certification Matrix for Windows 2000 for version information. 3. Verify that Secure Path Manager has been properly installed on the node.
Page 229: Run And Validate The Array Configuration Utility On All Nodes
A-6 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Run and Validate the Array Configuration Utility On All Nodes The Array Configuration Utility is used to configure the hardware aspects of disk drives connected to the RA4000 Array Controllers. The Array Configuration Utility allows you to configure RAID levels and to add disk drives or RA4000/RA4100 Arrays to an existing configuration.
Page 230: Perform Cluster Hardware Checks
I/O error messages or indications of problems with the communications transport. 3. Verify that the indicator lights on the Fibre Host Adapters, Compaq StorageWorks Storage Hubs, Compaq StorageWorks FA-AL Switches, or Compaq StorageWorks Fibre Channel SAN Switches, and RA4000 Array Controllers are all in the correct state.
Page 231: Contact Your Compaq Support Representative
A-8 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Contact Your Compaq Support Representative If you are unable to resolve the problem, contact your Compaq support representative for assistance. In North America, call the Compaq Technical Support Phone Center at 1-800-OK-COMPAQ.
Page 232 Glossary Availability A measure of how well a computer system can deliver services to its clients continuously. Availability is typically expressed as a percentage, with 100 percent being the best possible rating. Client LAN A local area network dedicated to servicing requests between network client machines and between the cluster nodes and the client machines.
Page 233: Fabrics
2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Fibre Channel An IEEE standard for providing high-speed data transfer among workstations, mainframes, supercomputers, desktop computers, storage devices, and display devices. Fibre Channel A data path from the Fibre Host Adapters in the cluster nodes to the Arbitrated Loop array controllers in the RA4000 Arrays or RA4100 Arrays.
Page 234 Glossary 3 Load balancing To balance the amount of work (processing) undertaken by each node in a cluster. Network Interface Controller. A board that allows a computer to be connected with a network, and works with the network operating system to control the flow of information over the network. Node An individual server in a cluster.
Page 235 4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Scalability The ability of a system (server, storage, network, or cluster) to grow or expand in performance, throughput, or capacity. Secure Path The Compaq software that provides cluster nodes with continuous data access to RA4000/RA4100 Array disks drives.
Page 236 Index example with four RA4000/RA4100 Arrays 2-31 active/active configurations example with three cabling for Oracle8i 5-11 RA4000/RA4100 active/active configurations for Arrays 2-30 redundant FC-ALs example with two definition 2-47 RA4000/RA4100 example with five Arrays 2-29 RA4000/RA4100 examples overview 2-27 Arrays 2-63 active/standby configurations example with four cabling for Oracle8i 5-10...
Page 237 2 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide active/standby configurations for redundant Fibre Channel Fabrics cabling definition 2-16 active/active configuration for example with five Oracle8i 5-11 RA4000/RA4100 active/standby configuration Arrays 2-26 for Oracle8i 5-10...
Page 238 4-20 management 2-70 site planning 4-2 Ethernet cluster supported RAID levels 4-16 interconnect 2-70 cluster software for Oracle8i options 2-70 Compaq software troubleshooting 7-8 overview 3-2 cluster management Oracle software 3-6 overview 6-1 overview 3-1 remote 6-7...
Page 239 4 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide node cannot connect to shared Compaq Insight Manager drives 7-14 monitoring cluster operation nodes cannot with 6-25 communicate 7-8 monitoring server and network nodes unable to rejoin...
Page 240 Index 5 Compaq software overview Compaq StorageWorks RAID Compaq Array Configuration Array 4000/4100 Array Utility 3-3 Controllers Compaq Insight Manager 3-4 defining active Compaq Insight Manager controllers 5-28 XE 3-4 Compaq StorageWorks RAID Compaq operating system Array 4000/4100s power distribution for...
Page 241 6 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide FC-AL Switches See Compaq StorageWorks FC-AL Switches error logs description 2-7 for Oracle8i 6-4 FFIU See Fibre Channel Fault Ethernet adapters Isolation Utility planning 4-6 Fibre Channel Arbitrated Loops...
Page 242 2-47 Oracle8i 5-47 rules 2-50 help summary of features 2-48 additional sources xix I/O path configurations for Compaq authorized resellers, redundant Fibre Channel Fabrics telephone numbers xx active/active examples 2-27 Compaq website xx active/standby examples 2-20 technical support telephone...
Page 243 8 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide installation Microsoft Windows NT Server additional Fibre Channel installing for Oracle8i 5-22 Arbitrated Loops for multiple Fibre Channel Arbitrated Oracle8i 5-15 Loops 2-42 automated using SmartStart for...
Page 244 Edition 3-6 overview 3-6 size 4-17 OracleCMService selecting RAID levels 4-17 starting 5-44 shared disk drives 4-20 troubleshooting 7-6 shared storage components for redundant FC-ALs 4-5 OracleNMService, troubleshooting 7-7 OSDs See Compaq operating system dependent modules OUI See Oracle Universal Installer...
Page 245 10 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide shared storage components for Redundant Ethernet client LAN for redundant Fibre Channel Oracle8i 5-20 Fabrics 4-3 redundant Ethernet cluster site planning 4-2 interconnect supported RAID levels 4-16...
Page 246 Oracle8i 5-47 resources 5-27 installing for Oracle8i 5-21 revisions 7-3 Service Packs for Windows 2000 Advanced Server Storage Hubs See Compaq versions supported 3-1 StorageWorks Fibre Channel Service Packs for Windows NT Storage Hubs Server storagesets supported for Oracle8i 5-22...
Page 247 12 Compaq Parallel Database Cluster Model PDC/O2000 for Oracle8i and Windows 2000 Administrator Guide Fibre Channel Fabric connectivity uninterruptible power supplies guidelines 7-11 for Oracle8i 5-48 NM.DLL 7-8 upgrading node cannot connect to shared Oracle8i Server 6-11 drives 7-14 nodes cannot...

This manual is also suitable for:

Proliant 6500 Proliant 8500 6400r - proliant - 1 gb ram Pdc/o2000

Compaq 112726-001 - ProLiant - 6500 Administrator's Manual

About this Guide

Chapter 1 Clustering Overview

Chapter 2 Cluster Architecture

Cluster Architecture

Chapter 3 Cluster Software Components

Chapter 4 Cluster Planning

Cluster Planning

Chapter 5 Installation and Configuration

Chapter 6 Cluster Management

Cluster Management

Chapter 7 Troubleshooting

Appendix A Diagnosing and Resolving Shared Disk Problems

Quick Links

Windows 2000

Troubleshooting

Related Manuals for Compaq 112726-001 - ProLiant - 6500

Summary of Contents for Compaq 112726-001 - ProLiant - 6500