Introduction 1. Document Conventions ....................vi 1.1. Typographic Conventions ..................vi 1.2. Pull-quote Conventions ..................vii 1.3. Notes and Warnings ................... viii 2. Feedback ........................viii 1. Red Hat Cluster Suite Overview 1.1. Cluster Basics ......................1 1.2. Red Hat Cluster Suite Introduction ................2 1.3.
• Red Hat Cluster Suite Release Notes — Provides information about the current release of Red Hat Cluster Suite. Red Hat Cluster Suite documentation and other Red Hat documents are available in HTML, http:// PDF, and RPM versions on the Red Hat Enterprise Linux Documentation CD and online at www.redhat.com/docs/.
Introduction 1. Document Conventions This manual uses several conventions to highlight certain words and phrases and draw attention to specific pieces of information. Liberation Fonts In PDF and paper editions, this manual uses typefaces drawn from the set. The Liberation Fonts set is also used in HTML editions if the set is installed on your system. If not, alternative but equivalent typefaces are displayed.
Pull-quote Conventions To insert a special character into a gedit file, choose Applications Accessories Character Map from the main menu bar. Next, choose Search Find… from the Character Map menu bar, type the name of the character in the Search field and click Next.
2. Feedback If you spot a typo, or if you have thought of a way to make this document better, we would love to hear from you. Please submit a report in Bugzilla (http://bugzilla.redhat.com/bugzilla/) against the component Documentation-cluster. Be sure to mention the document's identifier: Cluster_Suite_Overview(EN)-5 (2010-02-05T09:51) By mentioning this document's identifier, we know exactly which version of the guide you have.
Chapter 1. Red Hat Cluster Suite Overview Clustered systems provide reliability, scalability, and availability to critical production services. Using Red Hat Cluster Suite, you can create a cluster to suit your needs for performance, high availability, load balancing, scalability, file sharing, and economy. This chapter provides an overview of Red Hat Cluster Suite components and functions, and consists of the following sections: Section 1.1, “Cluster Basics”...
Chapter 1. Red Hat Cluster Suite Overview Load-balancing clusters dispatch network service requests to multiple cluster nodes to balance the request load among the cluster nodes. Load balancing provides cost-effective scalability because you can match the number of nodes according to load requirements. If a node in a load-balancing cluster becomes inoperative, the load-balancing software detects the failure and redirects requests to other cluster nodes.
Cluster Infrastructure Note When you create or modify a CLVM volume for a clustered environment, you must ensure that you are running the clvmd daemon. For further information, refer to Section 1.6, “Cluster Logical Volume Manager”. • Global Network Block Device (GNBD) — An ancillary component of GFS/GFS2 that exports block- level storage to Ethernet.
Chapter 1. Red Hat Cluster Suite Overview infrastructure, you can use other Red Hat Cluster Suite components to suit your clustering needs (for example, setting up a cluster for sharing files on a GFS file system or setting up service failover). The cluster infrastructure performs the following functions: •...
Lock Management Figure 1.2. CMAN/DLM Overview 1.3.2. Lock Management Lock management is a common cluster-infrastructure service that provides a mechanism for other cluster infrastructure components to synchronize their access to shared resources. In a Red Hat cluster, DLM (Distributed Lock Manager) is the lock manager. As implied in its name, DLM is a distributed lock manager and runs in each cluster node;...
Chapter 1. Red Hat Cluster Suite Overview • Power fencing — A fencing method that uses a power controller to power off an inoperable node. • Fibre Channel switch fencing — A fencing method that disables the Fibre Channel port that connects storage to an inoperable node.
Fencing Figure 1.4. Fibre Channel Switch Fencing Example Specifying a fencing method consists of editing a cluster configuration file to assign a fencing-method name, the fencing agent, and the fencing device for each node in the cluster. The way in which a fencing method is specified depends on if a node has either dual power supplies or multiple paths to storage.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.5. Fencing a Node with Dual Power Supplies...
Cluster Configuration System Figure 1.6. Fencing a Node with Dual Fibre Channel Connections You can configure a node with one fencing method or multiple fencing methods. When you configure a node for one fencing method, that is the only fencing method available for fencing that node. When you configure a node for multiple fencing methods, the fencing methods are cascaded from one fencing method to another according to the order of the fencing methods specified in the cluster configuration file.
Chapter 1. Red Hat Cluster Suite Overview cluster system administrator updates the configuration file in Node A, CCS propagates the update Figure 1.7, “CCS Overview”). from Node A to the other nodes in the cluster (refer to Figure 1.7. CCS Overview Other cluster components (for example, CMAN) access configuration information from the Figure 1.7, “CCS Overview”).
High-availability Service Management Figure 1.8. Accessing Configuration Information The cluster configuration file (/etc/cluster/cluster.conf) is an XML file that describes the following cluster characteristics: • Cluster name — Displays the cluster name, cluster configuration file revision level, and basic fence timing properties used when a node joins a cluster or is fenced from the cluster. •...
Chapter 1. Red Hat Cluster Suite Overview apparent interruption to cluster clients. Cluster-service failover can occur if a cluster node fails or if a cluster system administrator moves the service from one cluster node to another (for example, for a planned outage of a cluster node).
High-availability Service Management Figure 1.9. Failover Domains Figure 1.10, “Web Server Cluster Service Example” shows an example of a high-availability cluster service that is a web server named "content-webserver". It is running in cluster node B and is in a failover domain that consists of nodes A, B, and D.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.10. Web Server Cluster Service Example Clients access the cluster service through the IP address 10.10.10.201, enabling interaction with the web server application, httpd-content. The httpd-content application uses the gfs-content-webserver file system. If node B were to fail, the content-webserver cluster service would fail over to node D. If node D were not available or also failed, the service would fail over to node A.
Cluster Logical Volume Manager GFS/GFS2 is based on a 64-bit architecture, which can theoretically accommodate an 8 EB file system. However, the current supported maximum size of a GFS/GFS2 file system is 25 TB. If your system requires GFS/GFS2 file systems larger than 25 TB, contact your Red Hat service representative.
Chapter 1. Red Hat Cluster Suite Overview Note Shared storage for use in Red Hat Cluster Suite requires that you be running the cluster logical volume manager daemon (clvmd) or the High Availability Logical Volume Management agents (HA-LVM). If you are not able to use either the clvmd daemon or HA-LVM for operational reasons or because you do not have the correct entitlements, you must not use single-instance LVM on the shared disk as this may result in data corruption.
Global Network Block Device Figure 1.14. Creating Logical Volumes 1.7. Global Network Block Device Global Network Block Device (GNBD) provides block-device access to Red Hat GFS over TCP/IP. GNBD is similar in concept to NBD; however, GNBD is GFS-specific and tuned solely for use with GFS.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.15. GNBD Overview 1.8. Linux Virtual Server Linux Virtual Server (LVS) is a set of integrated software components for balancing the IP load across a set of real servers. LVS runs on a pair of equally configured computers: one that is an active LVS router and one that is a backup LVS router.
Linux Virtual Server Figure 1.16. Components of a Running LVS Cluster The pulse daemon runs on both the active and passive LVS routers. On the backup LVS router, pulse sends a heartbeat to the public interface of the active router to make sure the active LVS router is properly functioning.
Chapter 1. Red Hat Cluster Suite Overview • Synchronize the data across the real servers. • Add a third layer to the topology for shared data access. The first option is preferred for servers that do not allow large numbers of users to upload or change data on the real servers.
Two-Tier LVS Topology Service requests arriving at an LVS router are addressed to a virtual IP address or VIP. This is a publicly-routable address that the administrator of the site associates with a fully-qualified domain name, such as www.example.com, and which is assigned to one or more virtual servers .
Chapter 1. Red Hat Cluster Suite Overview • Source Hash Scheduling — Distributes requests to the pool of real servers by looking up the source IP in a static hash table. This algorithm is for LVS routers with multiple firewalls. Also, the active LVS router dynamically monitors the overall health of the specific services on the real servers through simple send/expect scripts.
Routing Methods Figure 1.18. Three-Tier LVS Topology This topology is suited well for busy FTP servers, where accessible data is stored on a central, highly available server and accessed by each real server via an exported NFS directory or Samba share. This topology is also recommended for websites that access a central, high-availability database for transactions.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.19. LVS Implemented with NAT Routing In the example, there are two NICs in the active LVS router. The NIC for the Internet has a real IP address on eth0 and has a floating IP address aliased to eth0:1. The NIC for the private network interface has a real IP address on eth1 and has a floating IP address aliased to eth1:1.
Routing Methods outgoing packets through the LVS router. Direct routing reduces the possibility of network performance issues by relegating the job of the LVS router to processing incoming packets only. Figure 1.20. LVS Implemented with Direct Routing In a typical direct-routing LVS configuration, an LVS router receives incoming server requests through a virtual IP (VIP) and uses a scheduling algorithm to route the request to real servers.
Chapter 1. Red Hat Cluster Suite Overview address of the LVS router must also be associated to a MAC. However, because both the LVS router and the real servers have the same VIP, the ARP request is broadcast to all the nodes associated with the VIP.
Cluster Administration Tools 1.9. Cluster Administration Tools Red Hat Cluster Suite provides a variety of tools to configure and manage your Red Hat Cluster. This section provides an overview of the administration tools available with Red Hat Cluster Suite: Section 1.9.1, “Conga” •...
Chapter 1. Red Hat Cluster Suite Overview Each luci instance has one user at initial installation — admin. Only the admin user may add systems to a luci server. Also, the admin user can create additional user accounts and determine which users are allowed to access clusters and computers registered in the luci database.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.23. luci storage Tab 1.9.2. Cluster Administration GUI This section provides an overview of the system-config-cluster cluster administration graphical user interface (GUI) available with Red Hat Cluster Suite. The GUI is for use with the cluster Section 1.3, infrastructure and the high-availability service management components (refer to “Cluster Infrastructure”...
Cluster Administration GUI Figure 1.24. Cluster Configuration Tool The Cluster Configuration Tool represents cluster configuration components in the configuration file (/etc/cluster/cluster.conf) with a hierarchical graphical display in the left panel. A triangle icon to the left of a component name indicates that the component has one or more subordinate components assigned to it.
Chapter 1. Red Hat Cluster Suite Overview Properties), you can add fence devices, delete fence devices, and edit fence-device properties. Fence devices must be defined before you can configure fencing (with the Manage Fencing For This Node button) for each node. •...
Command Line Administration Tools Figure 1.25. Cluster Status Tool The nodes and services displayed in the Cluster Status Tool are determined by the cluster configuration file (/etc/cluster/cluster.conf). You can use the Cluster Status Tool to enable, disable, restart, or relocate a high-availability service. 1.9.3.
Chapter 1. Red Hat Cluster Suite Overview Command Line Used With Purpose Tool Cluster ccs_tool is a program for making online updates to the ccs_tool — Cluster Infrastructure cluster configuration file. It provides the capability to create Configuration and modify cluster infrastructure components (for example, System Tool creating a cluster, adding and removing a node).
CONTROL/MONITORING Figure 1.26. The Welcome Panel The following sections provide a brief description of the Piranha Configuration Tool configuration pages. 1.10.1. CONTROL/MONITORING The CONTROL/MONITORING Panel displays runtime status. It displays the status of the pulse daemon, the LVS routing table, and the LVS-spawned nanny processes.
Chapter 1. Red Hat Cluster Suite Overview Figure 1.27. The CONTROL/MONITORING Panel Auto update Enables the status display to be updated automatically at a user-configurable interval set in the Update frequency in seconds text box (the default value is 10 seconds). It is not recommended that you set the automatic update to an interval less than 10 seconds.
GLOBAL SETTINGS Figure 1.28. The GLOBAL SETTINGS Panel The top half of this panel sets up the primary LVS router's public and private network interfaces. Primary server public IP The publicly routable real IP address for the primary LVS node. Primary server private IP The real IP address for an alternative network interface on the primary LVS node.
Chapter 1. Red Hat Cluster Suite Overview NAT Router device Defines the device name of the network interface for the floating IP address, such as eth1:1. 1.10.3. REDUNDANCY The REDUNDANCY panel allows you to configure of the backup LVS router node and set various heartbeat monitoring options.
VIRTUAL SERVERS Assume dead after (seconds) If the primary LVS node does not respond after this number of seconds, then the backup LVS router node will initiate failover. Heartbeat runs on port Sets the port at which the heartbeat communicates with the primary LVS node. The default is set to 539 if this field is left blank.
Chapter 1. Red Hat Cluster Suite Overview To enable or disable a virtual server in the table click its radio button and click the (DE)ACTIVATE button. After adding a virtual server, you can configure it by clicking the radio button to its left and clicking the EDIT button to display the VIRTUAL SERVER subsection.
VIRTUAL SERVERS Protocol Provides a choice of UDP or TCP, in a drop-down menu. Virtual IP Address The virtual server's floating IP address. Virtual IP Network Mask The netmask for this virtual server, in the drop-down menu. Firewall Mark For entering a firewall mark integer value when bundling multi-port protocols or creating a multi- port virtual server for separate, but related protocols.
Chapter 1. Red Hat Cluster Suite Overview 22.214.171.124. REAL SERVER Subsection Clicking on the REAL SERVER subsection link at the top of the panel displays the EDIT REAL SERVER subsection. It displays the status of the physical server hosts for a particular virtual service. Figure 1.32.
VIRTUAL SERVERS Figure 1.33. The REAL SERVER Configuration Panel This panel consists of three entry fields: Name A descriptive name for the real server. Note This name is not the hostname for the machine, so make it descriptive and easily identifiable.
Chapter 1. Red Hat Cluster Suite Overview 126.96.36.199. EDIT MONITORING SCRIPTS Subsection Click on the MONITORING SCRIPTS link at the top of the page. The EDIT MONITORING SCRIPTS subsection allows the administrator to specify a send/expect string sequence to verify that the service for the virtual server is functional on each real server.
VIRTUAL SERVERS Send A string for the nanny daemon to send to each real server in this field. By default the send field is completed for HTTP. You can alter this value depending on your needs. If you leave this field blank, the nanny daemon attempts to open the port and assume the service is running if it succeeds.
Chapter 2. Red Hat Cluster Suite Component Summary This chapter provides a summary of Red Hat Cluster Suite components and consists of the following sections: Section 2.1, “Cluster Components” • Section 2.2, “Man Pages” • Section 2.3, “Compatible Hardware” • 2.1.
Chapter 2. Red Hat Cluster Suite Component Summary Function Components Description Diagnostic and testing command that ccs_test is used to retrieve information from configuration files through ccsd. CCS daemon that runs on all cluster ccsd nodes and provides configuration file data to cluster software.
Cluster Components Function Components Description Fence agent for Bull Novascale Platform fence_bullpap Administration Processor (PAP) Interface. Fencing agent for Dell Remote Access fence_drac Card. Fence agent for machines controlled by fence_ipmilan IPMI (Intelligent Platform Management Interface) over LAN. Fence agent for WTI power switch. fence_wti Fence agent for Brocade Fibre Channel fence_brocade...
Chapter 2. Red Hat Cluster Suite Component Summary Function Components Description Command that repairs an unmounted gfs_fsck GFS file system. Command that grows a mounted GFS gfs_grow file system. Command that adds journals to a gfs_jadd mounted GFS file system. Command that creates a GFS file gfs_mkfs system on a storage device.
Man Pages Function Components Description The lvs daemon runs on the active LVS lvsd router once called by pulse. It reads the configuration file /etc/sysconfig/ ha/lvs.cf, calls the ipvsadm utility to build and maintain the IPVS routing table, and assigns a nanny process for each configured LVS service.
Chapter 2. Red Hat Cluster Suite Component Summary • ccs_tool (8) - The tool used to make online updates of CCS config files • ccs_test (8) - The diagnostic tool for a running Cluster Configuration System • ccsd (8) - The daemon used to access CCS cluster configuration files •...
Man Pages • fence_wti (8) - I/O Fencing agent for WTI Network Power Switch • fence_xvm (8) - I/O Fencing agent for Xen virtual machines • fence_xvmd (8) - I/O Fencing agent host for Xen virtual machines • fenced (8) - the I/O Fencing daemon •...
• send_arp (8) - tool to notify network of a new IP address / MAC address mapping 2.3. Compatible Hardware For information about hardware that is compatible with Red Hat Cluster Suite components (for example, supported fence devices, storage devices, and Fibre Channel switches), refer to the hardware configuration guidelines at http://www.redhat.com/cluster_suite/hardware/.
Appendix A. Revision History Revision 5.4-1 Fri Feb 05 2010 Paul Kennedy firstname.lastname@example.org Resolves #548603 Changed supported node count to 16 for cluster and GFS/GFS2. Revision 5.3-1 Tue Jan 20 2008 Paul Kennedy email@example.com Consolidation of point releases...