Introduction 1. Document Conventions ....................vi 1.1. Typographic Conventions ..................vi 1.2. Pull-quote Conventions ..................vii 1.3. Notes and Warnings ................... viii 2. Feedback ........................ix 1. Linux Virtual Server Overview 1.1. A Basic LVS Configuration .................... 1 1.1.1. Data Replication and Data Sharing Between Real Servers ........3 1.2.
Virtual Server Administration 4.3. CONTROL/MONITORING ................... 32 4.4. GLOBAL SETTINGS ....................34 4.5. REDUNDANCY ......................35 4.6. VIRTUAL SERVERS ....................37 4.6.1. The VIRTUAL SERVER Subsection ..............38 4.6.2. REAL SERVER Subsection ................41 4.6.3. EDIT MONITORING SCRIPTS Subsection ............44 4.7.
Introduction This document provides information about installing, configuring, and managing Red Hat Virtual Linux Server (LVS) components. LVS provides load balancing through specialized routing techniques that dispatch traffic to a pool of servers. This document does not include information about installing, configuring, and managing Red Hat Cluster software.
Red Hat Cluster Suite documentation and other Red Hat documents are available in HTML, http:// PDF, and RPM versions on the Red Hat Enterprise Linux Documentation CD and online at www.redhat.com/docs/. 1. Document Conventions This manual uses several conventions to highlight certain words and phrases and draw attention to specific pieces of information.
Pull-quote Conventions Choose System > Preferences > Mouse from the main menu bar to launch Mouse Preferences. In the Buttons tab, click the Left-handed mouse check box and click Close to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand).
Introduction Output sent to a terminal is set in Mono-spaced Roman and presented thus: books Desktop documentation drafts photos stuff books_tests Desktop1 downloads images notes scripts svgs Source-code listings are also set in Mono-spaced Roman but are presented and highlighted as follows: package org.jboss.book.jca.ex1;...
2. Feedback If you spot a typo, or if you have thought of a way to make this manual better, we would love to hear from you. Please submit a report in Bugzilla (http://bugzilla.redhat.com/bugzilla/) against the component Documentation-cluster. Be sure to mention the manual's identifier: Virtual_Server_Administration(EN)-5 (2009-08-18T15:53) By mentioning this manual's identifier, we know exactly which version of the guide you have.
Chapter 1. Linux Virtual Server Overview Linux Virtual Server (LVS) is a set of integrated software components for balancing the IP load across a set of real servers. LVS runs on a pair of equally configured computers: one that is an active LVS router and one that is a backup LVS router.
Chapter 1. Linux Virtual Server Overview Figure 1.1. A Basic LVS Configuration Service requests arriving at the LVS routers are addressed to a virtual IP address, or VIP. This is a publicly-routable address the administrator of the site associates with a fully-qualified domain name, such as www.example.com, and is assigned to one or more virtual servers.
Data Replication and Data Sharing Between Real Servers spoofing — where the backup LVS router announces itself as the destination for IP packets addressed to the failed node. When the failed node returns to active service, the backup node assumes its hot- backup role again.
Chapter 1. Linux Virtual Server Overview Figure 1.2. A Three-Tier LVS Configuration This configuration is ideal for busy FTP servers, where accessible data is stored on a central, highly available server and accessed by each real server via an exported NFS directory or Samba share. This topology is also recommended for websites that access a central, highly available database for transactions.
Scheduling Algorithms over application-level request forwarding because balancing loads at the network packet level causes minimal computational overhead and allows for greater scalability. Using scheduling, the active router can take into account the real servers' activity and, optionally, an administrator-assigned weight factor when routing service requests. Using assigned weights gives arbitrary priorities to individual machines.
Chapter 1. Linux Virtual Server Overview Weighted Least-Connections (default) Distributes more requests to servers with fewer active connections relative to their capacities. Capacity is indicated by a user-assigned weight, which is then adjusted upward or downward by dynamic load information. The addition of weighting makes this algorithm ideal when the real Section 1.3.2, “Server Weight and server pool contains hardware of varying capacity.
Routing Methods 1.4. Routing Methods Red Hat Enterprise Linux uses Network Address Translation or NAT routing for LVS, which allows the administrator tremendous flexibility when utilizing available hardware and integrating the LVS into an existing network. 1.4.1. NAT Routing Figure 1.3, “LVS Implemented with NAT Routing”, illustrates LVS utilizing NAT routing to move requests between the Internet and a private network.
Chapter 1. Linux Virtual Server Overview Using this NAT routing, the real servers may be any kind of machine running various operating systems. The main disadvantage is that the LVS router may become a bottleneck in large cluster deployments because it must process outgoing as well as incoming requests. 1.4.2.
Persistence and Firewall Marks 18.104.22.168. Direct Routing and the ARP Limitation While there are many advantages to using direct routing in LVS, there are limitations as well. The most common issue with LVS via direct routing is with Address Resolution Protocol (ARP). In typical situations, a client on the Internet sends a request to an IP address.
Chapter 1. Linux Virtual Server Overview the problem of grouping together connections destined for different ports. For these situations, it is best to use firewall marks. 1.5.2. Firewall Marks Firewall marks are an easy and efficient way to a group ports used for a protocol or group of related protocols.
LVS Components The pulse daemon runs on both the active and passive LVS routers. On the backup router, pulse sends a heartbeat to the public interface of the active router to make sure the active router is still properly functioning. On the active router, pulse starts the lvs daemon and responds to heartbeat queries from the backup LVS router.
Chapter 1. Linux Virtual Server Overview 22.214.171.124. /etc/sysconfig/ha/lvs.cf This is the LVS configuration file. Directly or indirectly, all daemons get their configuration information from this file. 126.96.36.199. Piranha Configuration Tool This is the Web-based tool for monitoring, configuring, and administering LVS. This is the default tool to maintain the /etc/sysconfig/ha/lvs.cf LVS configuration file.
Chapter 2. Initial LVS Configuration After installing Red Hat Enterprise Linux, you must take some basic steps to set up both the LVS routers and the real servers. This chapter covers these initial steps in detail. Note The LVS router node that becomes the active node once LVS is started is also referred to as the primary node.
Chapter 2. Initial LVS Configuration Warning Turning any of the above services on using chkconfig does not actually start the Section 2.3, “Starting the daemon. To do this use the /sbin/service command. See Piranha Configuration Tool Service” for an example of how to use the /sbin/service command.
Configuring the Piranha Configuration Tool Web Server Port Warning If the command /sbin/service httpd stop or /sbin/service httpd restart is issued on an LVS router, you must start the piranha-gui service by issuing the following command: /sbin/service piranha-gui start The piranha-gui service is all that is necessary to begin configuring LVS. However, if you are configuring LVS remotely, the sshd service is also required.
Chapter 2. Initial LVS Configuration Order deny,allow Deny from all Allow from 127.0.0.1 You can also allow specific hosts or subnets as seen in this example: Order deny,allow Deny from all Allow from 192.168.1.100 Allow from 172.16.57 In this example, only Web browsers from the machine with the IP address of 192.168.1.100 and machines on the 172.16.57/24 network can access the Piranha Configuration Tool.
Configuring Services on the Real Servers It may also be useful to access the real servers remotely, so the sshd daemon should also be installed and running.
Chapter 3. Setting Up LVS LVS consists of two basic groups: the LVS routers and the real servers. To prevent a single point of failure, each groups should contain at least two member systems. The LVS router group should consist of two identical or very similar systems running Red Hat Enterprise Linux.
Chapter 3. Setting Up LVS BOOTPROTO=static ONBOOT=yes IPADDR=192.168.26.9 NETMASK=255.255.255.0 GATEWAY=192.168.26.254 The /etc/sysconfig/network-scripts/ifcfg-eth1 for the private NAT interface on the LVS router could look something like this: DEVICE=eth1 BOOTPROTO=static ONBOOT=yes IPADDR=10.11.12.9 NETMASK=255.255.255.0 In this example, the VIP for the LVS router's public interface will be 192.168.26.10 and the VIP for the NAT or private interface will be 10.11.12.10.
Enabling NAT Routing on the LVS Routers So the real server's /etc/sysconfig/network-scripts/ifcfg-eth0 file could look similar to this: DEVICE=eth0 ONBOOT=yes BOOTPROTO=static IPADDR=10.11.12.1 NETMASK=255.255.255.0 GATEWAY=10.11.12.10 Warning If a real server has more than one network interface configured with a GATEWAY= line, the first one to come up will get the gateway.
Chapter 3. Setting Up LVS Direct routing requires that the real servers be physically connected to a network segment with the LVS router and be able to process and direct outgoing packets as well. Network Layout In a direct routing LVS setup, the LVS router needs to receive incoming requests and route them to the proper real server for processing.
Direct Routing and iptables However, using the arptables_jf method, VIPs can not be configured to start on boot using standard Red Hat Enterprise Linux system configuration tools. To configure each real server to ignore ARP requests for each virtual IP addresses, perform the following steps: 1.
Chapter 3. Setting Up LVS transparent proxy so that a real server will service packets sent to the VIP address, even though the VIP address does not exist on the system. The iptables method is simpler to configure than the arptables_jf method. This method also circumvents the LVS ARP issue entirely, because the virtual IP address(es) only exist on the active LVS director.
General LVS Networking Tips system-config-network, see the chapter titled Network Configuration in the Red Hat Enterprise Linux Deployment Guide. For the remainder of the chapter, example alterations to network interfaces are made either manually or through the Piranha Configuration Tool. 3.3.1.
Chapter 3. Setting Up LVS firewall marks must be performed by the network packet filter, iptables, outside of Piranha Configuration Tool. 3.4.1. Assigning Firewall Marks To assign firewall marks to a packet destined for a particular port, the administrator must use iptables.
Configuring FTP 3.5. Configuring FTP File Transport Protocol (FTP) is an old and complex multi-port protocol that presents a distinct set of challenges to an LVS environment. To understand the nature of these challenges, you must first understand some key things about how FTP works. 3.5.1.
Chapter 3. Setting Up LVS 3.5.3. Creating Network Packet Filter Rules Section 3.4.1, Before assigning any iptables rules for FTP service, review the information in “Assigning Firewall Marks” concerning multi-port services and techniques for checking the existing network packet filtering rules. Below are rules which assign the same firewall mark, 21, to FTP traffic.
Saving Network Packet Filter Settings The following iptables commands have the net effect of assigning any traffic addressed to the floating IP on the appropriate ports a firewall mark of 21, which is in turn recognized by IPVS and forwarded appropriately: /sbin/iptables -t mangle -A PREROUTING -p tcp -d n.n.n.n/32 --dport 21 -j MARK --set-mark 21 /sbin/iptables -t mangle -A PREROUTING -p tcp -d n.n.n.n/32 --dport...
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool The Piranha Configuration Tool provides a structured approach to creating the necessary configuration file for LVS — /etc/sysconfig/ha/lvs.cf. This chapter describes the basic operation of the Piranha Configuration Tool and how to activate LVS once configuration is complete. Important The configuration file for LVS follows strict formatting rules.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Figure 4.1. The Welcome Panel Click on the Login button and enter piranha for the Username and the administrative password you created in the Password field. The Piranha Configuration Tool is made of four main screens or panels. In addition, the Virtual Servers panel contains four subsections.
CONTROL/MONITORING Figure 4.2. The CONTROL/MONITORING Panel Auto update The status display on this page can be updated automatically at a user configurable interval. To enable this feature, click on the Auto update checkbox and set the desired update frequency in the Update frequency in seconds text box (the default value is 10 seconds).
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool 4.4. GLOBAL SETTINGS The GLOBAL SETTINGS panel is where the you define the networking details for the primary LVS router's public and private network interfaces. Figure 4.3. The GLOBAL SETTINGS Panel The top half of this panel sets up the primary LVS router's public and private network interfaces.
REDUNDANCY Note The private IP address is not needed for Direct Routing configurations, as all real servers as well as the LVS directors share the same virtual IP addresses and should have the same IP route configuration. Note The primary LVS router's private IP can be configured on any interface that accepts TCP/IP, whether it be an Ethernet adapter or a serial port.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Note The first time you visit this screen, it displays an "inactive" Backup status and an ENABLE button. To configure the backup LVS router, click on the ENABLE button so that Figure 4.4, “The REDUNDANCY Panel”.
VIRTUAL SERVERS Heartbeat Interval (seconds) This field sets the number of seconds between heartbeats — the interval that the backup node will check the functional status of the primary LVS node. Assume dead after (seconds) If the primary LVS node does not respond after this number of seconds, then the backup LVS router node will initiate failover.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Figure 4.5. The VIRTUAL SERVERS Panel Each server displayed in the VIRTUAL SERVERS panel can be configured on subsequent screens or subsections. To add a service, click the ADD button. To remove a service, select it by clicking the radio button next to the virtual server and click the DELETE button.
The VIRTUAL SERVER Subsection Figure 4.6. The VIRTUAL SERVERS Subsection Name Enter a descriptive name to identify the virtual server. This name is not the hostname for the machine, so make it descriptive and easily identifiable. You can even reference the protocol used by the virtual server, such as HTTP.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Firewall Mark Do not enter a firewall mark integer value in this field unless you are bundling multi-port protocols or creating a multi-port virtual server for separate, but related protocols. In this example, the above virtual server has a Firewall Mark of 80 because we are bundling connections to HTTP on port 80 and to HTTPS on port 443 using the firewall mark value of 80.
REAL SERVER Subsection Scheduling Select your preferred scheduling algorithm from the drop-down menu. The default is Weighted Section 1.3.1, least-connection. For more information on scheduling algorithms, see “Scheduling Algorithms”. Persistence If an administrator needs persistent connections to the virtual server during client transactions, enter the number of seconds of inactivity allowed to lapse before a connection times out in this text field.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Figure 4.7. The REAL SERVER Subsection Click the ADD button to add a new server. To delete an existing server, select the radio button beside it and click the DELETE button. Click the EDIT button to load the EDIT REAL SERVER panel, as seen Figure 4.8, “The REAL SERVER Configuration Panel”.
REAL SERVER Subsection Figure 4.8. The REAL SERVER Configuration Panel This panel consists of three entry fields: Name A descriptive name for the real server. Note This name is not the hostname for the machine, so make it descriptive and easily identifiable.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Warning Remember to click the ACCEPT button after making any changes in this panel. To make sure you do not lose any changes when selecting a new panel. 4.6.3. EDIT MONITORING SCRIPTS Subsection Click on the MONITORING SCRIPTS link at the top of the page.
EDIT MONITORING SCRIPTS Subsection Note To ensure that each server in the real server pool is checked, use the special token %h after the path to the script in the Sending Program field. This token is replaced with each real server's IP address as the script is called by the nanny daemon. The following is a sample script to use as a guide when composing an external service-checking script: #!/bin/sh...
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Note To determine what to send for a given service, you can open a telnet connection to the port on a real server and see what is returned. For instance, FTP reports 220 upon connecting, so could enter quit in the Send field and 220 in the Expect field.
Synchronizing sysctl Important Section 2.1, “Configuring To use scp the sshd must be running on the backup router, see Services on the LVS Routers” for details on how to properly configure the necessary services on the LVS routers. Issue the following command as the root user from the primary LVS router to sync the lvs.cf files between the router nodes: scp /etc/sysconfig/ha/lvs.cf n.n.n.n:/etc/sysconfig/ha/lvs.cf In the command, replace n.n.n.n with the real IP address of the backup LVS router.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool tail -f /var/log/messages Then start LVS by typing the following command into the other terminal: /sbin/service pulse start Follow the progress of the pulse service's startup in the terminal with the kernel log messages. When you see the following output, the pulse daemon has started properly: gratuitous lvs arps finished To stop watching /var/log/messages, type Ctrl-c.
Appendix A. Using LVS with Red Hat Cluster You can use LVS routers with a Red Hat Cluster to deploy a high-availability e-commerce site that provides load balancing, data integrity, and application availability. Figure A.1, “LVS with a Red Hat Cluster” The configuration in represents an e-commerce site used for online merchandise ordering through a URL.
Appendix A. Using LVS with Red Hat Cluster A three-tier configuration is required to provide dynamic content. While a two-tier LVS configuration is suitable if the Web servers serve only static Web content (consisting of small amounts of infrequently changing data), a two-tier configuration is not suitable if the Web servers serve dynamic content. Dynamic content could include product inventory, purchase orders, or customer databases, which must be consistent on all the Web servers to ensure that customers have access to up-to-date and accurate information.