Summary of Contents for Red Hat LINUX VIRTUAL SERVER - FOR ENTERPRISE LINUX 5.2 REV 05-2008
Linux Virtual Server Administration Linux Virtual Server (LVS) for Red Hat Enterprise Linux Virtual_Server_Administration ISBN: N/A Publication date: May 2008...
Linux Virtual Server Administration Building a Linux Virtual Server (LVS) system offers highly-available and scalable solution for production services using specialized routing and load-balancing techniques configured through the PIRANHA. This book discusses the configuration of high-performance systems and services with Red Hat Enterprise Linux and LVS for Red Hat Enterprise Linux 5.2.
All other trademarks referenced herein are the property of their respective owners. The GPG fingerprint of the firstname.lastname@example.org key is: CA 20 86 86 2B D6 9D FC 65 F6 EC C4 21 91 80 CD DB 42 A6 0E...
Introduction ......................vii 1. Document Conventions ................viii 2. Feedback ...................... ix 1. Linux Virtual Server Overview ................. 1 1. A Basic LVS Configuration ................1 1.1. Data Replication and Data Sharing Between Real Servers ..... 3 2. A Three-Tier LVS Configuration ..............3 3.
Linux Virtual Server Administration 3. CONTROL/MONITORING ................38 4. GLOBAL SETTINGS ..................40 5. REDUNDANCY ....................42 6. VIRTUAL SERVERS ..................44 6.1. The VIRTUAL SERVER Subsection ............45 6.2. REAL SERVER Subsection ..............49 6.3. EDIT MONITORING SCRIPTS Subsection ..........51 7. Synchronizing Configuration Files ..............53 7.1.
Introduction This document provides information about installing, configuring, and managing Red Hat Virtual Linux Server (LVS) components. LVS provides load balancing through specialized routing techniques that dispatch traffic to a pool of servers. This document does not include information about installing, configuring, and managing Red Hat Cluster software. Information about that is in a separate document.
Red Hat Cluster Suite documentation and other Red Hat documents are available in HTML, PDF, and RPM versions on the Red Hat Enterprise Linux Documentation CD and online at http://www.redhat.com/docs/. 1. Document Conventions Certain words in this manual are represented in different fonts, styles, and weights. This highlighting indicates that the word is part of a specific category.
2. Feedback If you spot a typo, or if you have thought of a way to make this manual better, we would love to hear from you. Please submit a report in Bugzilla (http://bugzilla.redhat.com/bugzilla/) against the component Documentation-cluster. Be sure to mention the manual's identifier:...
Introduction Virtual_Server_Administration(EN)-5.2 (2008-05-15T16:07) By mentioning this manual's identifier, we know exactly which version of the guide you have. If you have a suggestion for improving the documentation, try to be as specific as possible. If you have found an error, please include the section number and some of the surrounding text so we can find it easily.
Chapter 1. Linux Virtual Server Overview Linux Virtual Server (LVS) is a set of integrated software components for balancing the IP load across a set of real servers. LVS runs on a pair of equally configured computers: one that is an active LVS router and one that is a backup LVS router.
Chapter 1. Linux Virtual Server Overview Figure 1.1. A Basic LVS Configuration Service requests arriving at the LVS routers are addressed to a virtual IP address, or VIP. This is a publicly-routable address the administrator of the site associates with a fully-qualified domain name, such as www.example.com, and is assigned to one or more virtual servers.
Data Replication and Data Sharing Between The active router also dynamically monitors the overall health of the specific services on the real servers through simple send/expect scripts. To aid in detecting the health of services that require dynamic data, such as HTTPS or SSL, the administrator can also call external executables.
Chapter 1. Linux Virtual Server Overview Each of the real servers then accesses a shared data source over the network. Figure 1.2. A Three-Tier LVS Configuration This configuration is ideal for busy FTP servers, where accessible data is stored on a central, highly available server and accessed by each real server via an exported NFS directory or Samba share.
Real Servers available database for transactions. Additionally, using an active-active configuration with Red Hat Cluster Manager, administrators can configure one high-availability cluster to serve both of these roles simultaneously. The third tier in the above example does not have to use Red Hat Cluster Manager, but failing to use a highly available solution would introduce a critical single point of failure.
Chapter 1. Linux Virtual Server Overview Weighted Round-Robin Scheduling Distributes each request sequentially around the pool of real servers but gives more jobs to servers with greater capacity. Capacity is indicated by a user-assigned weight factor, which is then adjusted upward or downward by dynamic load information. Refer to Section 3.2, “Server Weight and Scheduling”...
Server Weight and Scheduling Distributes requests to the pool of real servers by looking up the source IP in a static hash table. This algorithm is designed for LVS routers with multiple firewalls. 3.2. Server Weight and Scheduling The administrator of LVS can assign a weight to each node in the real server pool. This weight is an integer value which is factored into any weight-aware scheduling algorithms (such as weighted least-connections) and helps the LVS router more evenly load hardware with different capabilities.
Chapter 1. Linux Virtual Server Overview Figure 1.3. LVS Implemented with NAT Routing In the example, there are two NICs in the active LVS router. The NIC for the Internet has a real IP address on eth0 and has a floating IP address aliased to eth0:1. The NIC for the private network interface has a real IP address on eth1 and has a floating IP address aliased to eth1:1.
Direct Routing which uses network address translation to replace the address of the real server in the packets with the LVS routers public VIP address. This process is called IP masquerading because the actual IP addresses of the real servers is hidden from the requesting clients. Using this NAT routing, the real servers may be any kind of machine running various operating systems.
Chapter 1. Linux Virtual Server Overview Figure 1.4. LVS Implemented with Direct Routing In the typical direct routing LVS setup, the LVS router receives incoming server requests through the virtual IP (VIP) and uses a scheduling algorithm to route the request to the real servers.
Persistence and Firewall Marks While there are many advantages to using direct routing in LVS, there are limitations as well. The most common issue with LVS via direct routing is with Address Resolution Protocol (ARP). In typical situations, a client on the Internet sends a request to an IP address. Network routers typically send requests to their destination by relating IP addresses to a machine's MAC address with ARP.
Chapter 1. Linux Virtual Server Overview it is handled according to the scheduling rules in place. Persistence also allows the administrator to specify a subnet mask to apply to the client IP address test as a tool for controlling what addresses have a higher level of persistence, thereby grouping connections to that subnet.
LVS Components Figure 1.5. LVS Components daemon runs on both the active and passive LVS routers. On the backup router, pulse sends a heartbeat to the public interface of the active router to make sure the active pulse router is still properly functioning. On the active router, starts the daemon and pulse...
Chapter 1. Linux Virtual Server Overview 6.1. LVS Components Section 6.1.1, “ ” shows a detailed list of each software component in an LVS router. pulse 6.1.1. pulse This is the controlling process which starts all other daemons related to LVS routers. At boot time, the daemon is started by the script.
LVS Components Chapter 2, Initial LVS Configuration reviews important post-installation configuration steps you should take before configuring Red Hat Enterprise Linux to be an LVS router.
Chapter 2. Initial LVS Configuration After installing Red Hat Enterprise Linux, you must take some basic steps to set up both the LVS routers and the real servers. This chapter covers these initial steps in detail. Note The LVS router node that becomes the active node once LVS is started is also referred to as the primary node.
Chapter 2. Initial LVS Configuration /sbin/chkconfig --level 35 daemon on In the above command, replace with the name of the service you are activating. To get daemon a list of services on the system as well as what runlevel they are set to activate on, issue the following command: /sbin/chkconfig --list Warning...
Configuring the Piranha Configuration Tool following command as root: /sbin/service piranha-gui start /sbin/service piranha-gui restart Issuing this command starts a private session of the Apache HTTP Server by calling the symbolic link . For security reasons, the /usr/sbin/piranha_gui -> /usr/sbin/httpd version of runs as the piranha user in a separate process.
Chapter 2. Initial LVS Configuration Now that the Piranha Configuration Tool is running, you may wish to consider limiting who has access to the tool over the network. The next section reviews ways to accomplish this task. 4. Limiting Access To the Piranha Configuration Tool The Piranha Configuration Tool prompts for a valid username and password combination.
Web Server Port configuration pages in the directory but not /etc/sysconfig/ha/web/secure/ to the login and the help pages in . To limit access to /etc/sysconfig/ha/web/ this directory, create a file in the directory .htaccess /etc/sysconfig/ha/web/ with , and lines identical to order allow deny...
Chapter 3. Setting Up LVS LVS consists of two basic groups: the LVS routers and the real servers. To prevent a single point of failure, each groups should contain at least two member systems. The LVS router group should consist of two identical or very similar systems running Red Hat Enterprise Linux.
Chapter 3. Setting Up LVS which link to the real servers ( ) will be on the 10.11.12/24 network. eth1 So on the active or primary LVS router node, the public interface's network script, , could look something like this: /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 BOOTPROTO=static...
Enabling NAT Routing on the LVS Routers 1.2. Routing on the Real Servers The most important thing to remember when configuring the real servers network interfaces in a NAT topology is to set the gateway for the NAT floating IP address of the LVS router. In this example, that address is 10.11.12.10.
Chapter 3. Setting Up LVS firewall marks, see Section 4, “Multi-port Services and LVS”. Once forwarding is enabled on the LVS routers and the real servers are set up and have the clustered services running, use the Piranha Configuration Tool to configure LVS as shown in Chapter 4, Configuring the LVS Routers with Piranha Configuration Tool.
Direct Routing and arptables_jf the incoming requests and perform load-balancing for the real servers, the real servers do not need to be Linux machines to function correctly. The LVS routers need one or two NICs each (depending on if there is a back-up router). You can use two NICs for ease of configuration and to distinctly separate traffic —...
Chapter 3. Setting Up LVS requests for any of the VIPs is the current active LVS node. 2. Once this has been completed on each real server, save the ARP table entries by typing the following commands on each real server: service arptables_jf save chkconfig --level 2345 arptables_jf on command will cause the system to reload the arptables configuration on...
Putting the Configuration Together instead of the virtual IP addresses. INADDR_ANY To configure direct routing using the method, perform the following steps: iptables 1. On each real server, run the following command for every VIP, port, and protocol (TCP or UDP) combination intended to be serviced for the real server: iptables -t nat -A PREROUTING -p <tcp|udp>...
Chapter 3. Setting Up LVS 3.1. General LVS Networking Tips Configure the real IP addresses for both the public and private networks on the LVS routers before attempting to configure LVS using the Piranha Configuration Tool. The sections on each topology give example network addresses, but the actual network addresses are needed. Below are some useful commands for bringing up network interfaces or checking their status.
Assigning Firewall Marks persistence to a virtual server, see Section 6.1, “The VIRTUAL SERVER Subsection”. Unfortunately, the mechanism used to balance the loads on the real servers — IPVS — can recognize the firewall marks assigned to a packet, but cannot itself assign firewall marks. The job of assigning firewall marks must be performed by the network packet filter, iptables outside of Piranha Configuration Tool.
Chapter 3. Setting Up LVS addressed to the VIP on the appropriate ports a firewall mark of 80, which in turn is recognized by IPVS and forwarded appropriately. Warning The commands above will take effect immediately, but do not persist through a reboot of the system.
How This Affects LVS Routing and passive connections. The FTP client/server relationship can potentially open a large number of ports that the Piranha Configuration Tool and IPVS do not know about. 5.2. How This Affects LVS Routing IPVS packet forwarding only allows connections in and out of the cluster based on it recognizing its port number or its firewall mark.
Chapter 3. Setting Up LVS Warning If you are limiting the port range for passive connections, you must also configure the VSFTP server to use a matching port range. This can be accomplished by adding the following lines to /etc/vsftpd.conf pasv_min_port=10000 pasv_max_port=20000 You must also control the address that the server displays to the client for...
Saving Network Packet Filter Settings runlevels. For more on this, refer to Section 1, “Configuring Services on the LVS Routers”. 6. Saving Network Packet Filter Settings After configuring the appropriate network packet filters for your situation, save the settings so they get restored after a reboot.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool The Piranha Configuration Tool provides a structured approach to creating the necessary configuration file for LVS — . This chapter describes the basic /etc/sysconfig/ha/lvs.cf operation of the Piranha Configuration Tool and how to activate LVS once configuration is complete.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Figure 4.1. The Welcome Panel Click on the Login button and enter for the Username and the administrative piranha password you created in the Password field. The Piranha Configuration Tool is made of four main screens or panels. In addition, the Virtual Servers panel contains four subsections.
CONTROL/MONITORING Figure 4.2. The CONTROL/MONITORING Panel Auto update The status display on this page can be updated automatically at a user configurable interval. To enable this feature, click on the Auto update checkbox and set the desired update frequency in the Update frequency in seconds text box (the default value is 10 seconds).
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Clicking this button takes you to a help screen with information on how to change the administrative password for the Piranha Configuration Tool. 4. GLOBAL SETTINGS The GLOBAL SETTINGS panel is where the you define the networking details for the primary LVS router's public and private network interfaces.
GLOBAL SETTINGS Enter the real IP address for an alternative network interface on the primary LVS node. This address is used solely as an alternative heartbeat channel for the backup router and does not have to correlate to the real private IP address assigned in Section 1.1, “Configuring Network Interfaces for LVS with NAT”.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool to the private network. In this example, the private network is on the eth1 interface, so is the floating IP address. eth1:1 Warning After completing this page, click the ACCEPT button to make sure you do not lose any changes when selecting a new panel.
REDUNDANCY Figure 4.4. The REDUNDANCY Panel Redundant server public IP Enter the public real IP address for the backup LVS router node. Redundant server private IP Enter the backup node's private real IP address in this text field. If you do not see the field called Redundant server private IP, go back to the GLOBAL SETTINGS panel and enter a Primary server private IP address and click ACCEPT.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Assume dead after (seconds) If the primary LVS node does not respond after this number of seconds, then the backup LVS router node will initiate failover. Heartbeat runs on port This field sets the port at which the heartbeat communicates with the primary LVS node.
The VIRTUAL SERVER Subsection Figure 4.5. The VIRTUAL SERVERS Panel Each server displayed in the VIRTUAL SERVERS panel can be configured on subsequent screens or subsections. To add a service, click the ADD button. To remove a service, select it by clicking the radio button next to the virtual server and click the DELETE button.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool any of the subsections related to this virtual server, complete this page and click on the ACCEPT button. Figure 4.6. The VIRTUAL SERVERS Subsection Name Enter a descriptive name to identify the virtual server. This name is not the hostname for the machine, so make it descriptive and easily identifiable.
The VIRTUAL SERVER Subsection Enter the virtual server's floating IP address in this text field. Virtual IP Network Mask Set the netmask for this virtual server with the drop-down menu. Firewall Mark Do not enter a firewall mark integer value in this field unless you are bundling multi-port protocols or creating a multi-port virtual server for separate, but related protocols.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Load monitoring tool The LVS router can monitor the load on the various real servers by using either . If you select from the drop-down menu, each real server must run the ruptime rstatd service.
REAL SERVER Subsection Warning Remember to click the ACCEPT button after making any changes in this panel. To make sure you do not lose changes when selecting a new panel. 6.2. REAL SERVER Subsection Clicking on the REAL SERVER subsection link at the top of the panel displays the EDIT REAL SERVER subsection.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Figure 4.8. The REAL SERVER Configuration Panel This panel consists of three entry fields: Name A descriptive name for the real server. This name is not the hostname for the machine, so make it descriptive and easily identifiable.
EDIT MONITORING SCRIPTS Subsection Weight An integer value indicating this host's capacity relative to that of other hosts in the pool. The value can be arbitrary, but treat it as a ratio in relation to other real servers in the pool. For more on server weight, see Section 3.2, “Server Weight and Scheduling”.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool Figure 4.9. The EDIT MONITORING SCRIPTS Subsection Sending Program For more advanced service verification, you can use this field to specify the path to a service-checking script. This functionality is especially helpful for services that require dynamically changing data, such as HTTPS or SSL.
Synchronizing Configuration Files Only one send sequence is allowed in this field, and it can only contain printable, ASCII characters as well as the following escape characters: • \n for new line. • \r for carriage return. • \t for tab. •...
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool • — If you are using firewall marks, you should synchronize one of /etc/sysconfig/iptables these files based on which network packet filter you are using. Important files do not change /etc/sysctl.conf /etc/sysconfig/iptables when you configure LVS using the Piranha Configuration Tool.
Synchronizing Network Packet Filtering Important If you are not sure whether or not packet forwarding is enabled in the kernel, see Section 5, “Turning on Packet Forwarding” for instructions on how to check and, if necessary, enable this key functionality. 7.3.
Chapter 4. Configuring the LVS Routers with Piranha Configuration Tool requests to LVS at this point, you should start the backup LVS router before putting LVS into service. To do this, simply repeat the process described above on the backup LVS router node. After completing this final step, LVS will be up and running.
Appendix A. Using LVS with Red Hat Cluster You can use LVS routers with a Red Hat Cluster to deploy a high-availability e-commerce site that provides load balancing, data integrity, and application availability. The configuration in Figure A.1, “LVS with a Red Hat Cluster” represents an e-commerce site used for online merchandise ordering through a URL.
Appendix A. Using LVS with Red Hat Cluster Figure A.1. LVS with a Red Hat Cluster Serving dynamic Web content with LVS requires a three-tier configuration (as shown in Figure A.1, “LVS with a Red Hat Cluster”). This combination of LVS and Red Hat Cluster allows for the configuration of a high-integrity, no-single-point-of-failure e-commerce site.
configuration is suitable if the Web servers serve only static Web content (consisting of small amounts of infrequently changing data), a two-tier configuration is not suitable if the Web servers serve dynamic content. Dynamic content could include product inventory, purchase orders, or customer databases, which must be consistent on all the Web servers to ensure that customers have access to up-to-date and accurate information.