The Technical Support group also creates content for our online Knowledge Base. The Technical Support group works collaboratively with the other functional areas within Symantec to answer your questions in a timely fashion. For example, the Technical Support group works with Product Engineering and Symantec Security Response to provide alerting services and virus definition updates.
Troubleshooting that was performed before contacting Symantec Recent software configuration changes and network changes Licensing and registration If your Symantec product requires registration or a license key, access our technical support Web page at the following URL: www.symantec.com/techsupp/ Customer service Customer service information is available at the following URL: www.symantec.com/techsupp/...
Page 6
North America and Latin America supportsolutions@symantec.com Additional enterprise services Symantec offers a comprehensive set of services that allow you to maximize your investment in Symantec products and to develop your knowledge, expertise, and global insight, which enable you to manage your business risks proactively.
About global clusters .............. 17 About I/O fencing ..............18 About VCS optional components ............ 18 About Symantec Product Authentication Service (AT) ....19 About Cluster Manager (Java Console) ........20 About Veritas Cluster Server Management Console ...... 20 About VCS Simulator .............. 20...
Page 8
Contents Creating authentication broker accounts on root broker system ................34 Creating encrypted files for the security infrastructure ....35 Preparing the installation system for the security infrastructure ..............37 Performing preinstallation tasks ............. 38 Obtaining VCS license keys ............39 Setting up the private network ..........
Page 9
Contents Adding VCS users ..............71 Configuring SMTP email notification ......... 71 Configuring SNMP trap notification .......... 73 Configuring global clusters ............74 Installing VCS RPMs ............... 75 Creating VCS configuration files ..........76 Starting VCS ................. 77 Completing the installation ............77 About enabling LDAP authentication for clusters that run in secure mode ..............
Page 10
Contents Verifying LLT, GAB, and cluster operation ........113 Verifying LLT ..............114 Verifying GAB ..............116 Verifying the cluster ............. 117 Verifying the cluster nodes ............ 118 Chapter 7 Adding and removing cluster nodes ......121 About adding and removing nodes ..........121 Adding a node to a cluster ............
Page 11
Contents Bringing up the existing node ..........145 Installing the VCS software manually when adding a node to a single node cluster ............146 Configuring LLT ..............146 Configuring GAB when adding a node to a single node cluster ................149 Starting LLT and GAB ............
About VCS features About VCS optional components About Veritas Cluster Server Veritas™ Cluster Server by Symantec is a high-availability solution for cluster configurations. Veritas Cluster Server (VCS) monitors systems and application services, and restarts services when hardware or software fails.
Introducing Veritas Cluster Server About VCS basics Figure 1-1 illustrates a typical VCS configuration of four nodes that are connected to shared storage. Example of a four-node VCS cluster Figure 1-1 Client workstation Client workstation Public network VCS private network VCS nodes Shared storage Client workstations receive service over the public network from applications...
Introducing Veritas Cluster Server About VCS basics Two examples of shared storage configurations Figure 1-2 Fully shared storage Distributed shared storage About LLT and GAB VCS uses two components, LLT and GAB, to share data over private networks among systems. These components provide the performance and reliability that VCS requires.
Introducing Veritas Cluster Server About VCS basics Figure 1-3 illustrates a two-node VCS cluster where the nodes galaxy and nebula have two private network connections. Two Ethernet connections connecting two nodes Figure 1-3 VCS private network: two ethernet connections galaxy nebula Shared disks Public network...
About VCS notifications You can configure both SNMP and SMTP notifications for VCS. Symantec recommends you to configure one of these notifications. You have the following options: Configure SNMP trap notification of VCS events using the VCS Notifier...
I/O fencing after you install and configure VCS. I/O fencing technology uses coordination points for arbitration in the event of a network partition. Note: Symantec recommends that you use I/O fencing to protect your cluster against split-brain situations. About VCS optional components...
The root broker is only used during initial creation of an authentication broker. A root broker can serve multiple clusters. Symantec recommends that you install a single root broker on a utility system. The utility system, such as an email server or domain controller, can be highly available.
Refer to the Veritas Cluster Server Management Console Implementation Guide for installation, upgrade, and configuration instructions. For information on updates and patches for VCS Management Console, see http://seer.entsupport.symantec.com/docs/308405.htm. To download the most current version of VCS Management Console, go to www.symantec.com/business/cluster-server and click Utilities.
Page 21
Introducing Veritas Cluster Server About VCS optional components configurations for Windows, Linux, and Solaris clusters. VCS Simulator also enables creating and testing global clusters. You can administer VCS Simulator from the Java Console or from the command line.
Page 22
Introducing Veritas Cluster Server About VCS optional components...
Every node where you want to install VCS must meet the hardware and software requirements. For the latest information on updates, patches, and software issues, read the following Veritas Technical Support TechNote: http://entsupport.symantec.com/docs/285834 To find information on supported hardware, see the hardware compatibility list (HCL) in the following TechNote: http://entsupport.symantec.com/docs/286819...
Network Interface In addition to the built-in public NIC, VCS requires at least one more Cards (NICs) NIC per system. Symantec recommends two additional NICs. You can also configure aggregated interfaces. Fibre Channel or Typical VCS configuration requires at least one SCSI or Fibre Channel SCSI host bus Host Bus Adapter per system for shared data disks.
VCS software. Refer to the Red Hat or SUSE documentation for more information on upgrading your system. Symantec supports only Red Hat and SUSE distributed kernel binaries. Symantec products operate on subsequent kernel and patch releases provided the operating systems maintain kernel ABI (application binary interface) compatibility.
Planning to install VCS Supported operating systems Required Linux RPMs for VCS Make sure you installed the following operating system-specific RPMs on the systems where you want to install or upgrade VCS. VCS will support any updates made to the following RPMs, provided the RPMs maintain the ABI compatibility. Table 2-4 lists the RPMs that VCS requires for a given Linux operating system.
Planning to install VCS Supported software Supported software Veritas Cluster Server supports the previous and next versions of Storage Foundation to facilitate product upgrades, when available. VCS supports the following volume managers and files systems: ext2, ext3, reiserfs, NFS, NFSv4, and bind on LVM2, Veritas Volume Manager (VxVM) 5.0, and raw disks.
Page 28
Planning to install VCS Supported software...
VCS setup. Preparing to configure the clusters in secure mode You can set up Symantec Product Authentication Service (AT) for the cluster during the VCS installation or after the installation.
Page 30
Preparing to install VCS Preparing to configure the clusters in secure mode The system clocks of the root broker and authentication brokers must be in sync. The installvcs program provides the following configuration modes: Automatic mode The root broker system must allow rsh or ssh passwordless login to use this mode.
Page 31
Preparing to install VCS Preparing to configure the clusters in secure mode Workflow to configure VCS cluster in secure mode Figure 3-1 Review AT concepts and gather required information Install root broker on a stable system On the root broker system, create authentication broker identities for each node Select a mode to configure the...
Page 32
Preparing to install VCS Preparing to configure the clusters in secure mode Preparatory tasks to configure a cluster in secure mode Table 3-1 Tasks Who performs this task Decide one of the following configuration modes to set up a cluster in VCS administrator secure mode: Automatic mode...
# ./installer From the opening Selection Menu, choose: I for "Install/Upgrade a Product." From the displayed list of products to install, choose: Symantec Product Authentication Service. To install the root broker, select the mode of AT installation as root mode...
Preparing to install VCS Preparing to configure the clusters in secure mode Enter when the installer prompts you to configure the Symantec Product Authentication Service. Press the Enter key to start the Authentication Server processes. Do you want to start Symantec Product Authentication Service processes now? [y,n,q] y Enter an encryption key.
Preparing to install VCS Preparing to configure the clusters in secure mode If the output displays the following error, then the account for the given authentication broker is not created on this root broker: "Failed To Get Attributes For Principal" Proceed to step 3.
Page 36
Preparing to install VCS Preparing to configure the clusters in secure mode identity The value for the authentication broker identity, which you provided to create authentication broker principal on the root broker system. This is the value for the --prplname option of the addprpl command.
Preparing to install VCS Preparing to configure the clusters in secure mode Note that for security purposes, the command to create the output file for the encrypted file deletes the input file. For each node in the cluster, create the output file for the encrypted file from the root broker system using the following command.
Preparing to install VCS Performing preinstallation tasks Manual mode Do the following: Copy the root_hash file that you fetched to the system from where you plan to install VCS. Note the path of the root hash file that you copied to the installation system.
However, you are legally limited to the number of product licenses purchased. The product installation procedure describes how to activate the key. To register and receive a software license key, go to the Symantec Licensing Portal at the following location: https://licensing.symantec.com...
Preparing to install VCS Performing preinstallation tasks You can only install the Symantec software products for which you have purchased a license. The enclosed software discs might include other products for which you have not purchased a license. Setting up the private network VCS requires you to set up a private network between the systems that form a cluster.
Page 41
Preparing to install VCS Performing preinstallation tasks Private network setup with crossed links Figure 3-3 Public network Private networks Crossed link To set up the private network Install the required network interface cards (NICs). Create aggregated interfaces if you want to use these to set up private network. Connect the VCS private NICs on each system.
VCS must be able to find the same network interface names across reboots. VCS must have network interfaces up before LLT starts to run. Symantec suggests the following steps for configuring network interfaces on SUSE. Note: You must not reboot the system between configuring the persistent interface names and configuring the interfaces to be up before starting LLT.
Page 43
-a Make sure that the interface name to MAC address mapping remains same across the reboots. Symantec recommends adding the PERSISTENT_NAME entries to the configuration files for all the network interfaces (including the network interfaces that are not used).
Preparing to install VCS Performing preinstallation tasks collisions:0 txqueuelen:1000 RX bytes:35401016 (33.7 Mb) TX bytes:999899 (976.4 Kb) Base address:0xdce0 Memory:fcf20000-fcf40000 If a file named etc/sysconfig/network/ifcfg-eth-id-00:02:B3:DB:38:FE does not exist, do the following task: Create the file. If the file /etc/sysconfig/network/ifcfg-eth0 exists, then copy the contents of this file into etc/sysconfig/network/ifcfg-eth-id-00:02:B3:DB:38:FE.
Page 45
Preparing to install VCS Performing preinstallation tasks where you run installvcs program. This privilege facilitates to issue ssh or rsh commands on all systems in the cluster. If ssh is used to communicate between systems, it must be configured in a way such that it operates without requests for passwords or passphrases.
Preparing to install VCS Performing preinstallation tasks When the command prompts, enter a passphrase and confirm it. Change the permissions of the .ssh directory by typing: # chmod 755 ~/.ssh The file ~/.ssh/id_dsa.pub contains a line that begins with ssh_dss and ends with the name of the system on which it was created.
Preparing to install VCS Performing preinstallation tasks See also the Veritas Cluster Server User's Guide for a description of I/O fencing. Setting the PATH variable Installation commands as well as other commands reside in the /sbin, /usr/sbin, /opt/VRTS/bin, and /opt/VRTSvcs/bin directories. Add these directories to your PATH environment variable.
Review the following guidelines for setting the media speed of the LLT interconnects: Symantec recommends that you manually set the same media speed setting on each Ethernet card on each node. If you have hubs or switches for LLT interconnects, then set the hub or switch port to the same setting as used on the cards on each node.
Preparing to install VCS Performing preinstallation tasks Mounting the product disc You must have superuser (root) privileges to load the VCS software. To mount the product disc Log in as superuser on a system where you want to install VCS. The system from which you install VCS need not be part of the cluster.
Page 50
Preparing to install VCS Performing preinstallation tasks To check the systems Navigate to the folder that contains the installvcs program. “Mounting the product disc” on page 49. Start the pre-installation check: # ./installvcs -precheck galaxy nebula The program proceeds in a noninteractive mode to examine the systems for licenses, RPMs, disk space, and system-to-system communications.
Chapter Installing and configuring This chapter includes the following topics: About installing and configuring VCS Getting your VCS installation and configuration information ready About the VCS installation program Installing and configuring VCS 5.0 RU3 Verifying and updating licenses on the system Accessing the VCS documentation About installing and configuring VCS You can install Veritas Cluster Server on clusters of up to 32 systems.
Installing and configuring VCS Getting your VCS installation and configuration information ready Getting your VCS installation and configuration information ready The VCS installation and configuration program prompts you for information about certain VCS components. When you perform the installation, prepare the following information: To install VCS RPMs you need: The system names where you Example: galaxy, nebula...
Page 53
To configure VCS clusters in secure mode (optional), you need: For automatic mode (default) The name of the Root Broker system Example: east “About Symantec Product Authentication Service (AT)” on page 19. Access to the Root Broker system without use of a password.
Installing and configuring VCS Getting your VCS installation and configuration information ready The domain-based address of The SMTP server sends notification emails about the the SMTP server events within the cluster. Example: smtp.symantecexample.com The email address of each Example: john@symantecexample.com SMTP recipient to be notified To decide the minimum Events have four levels of severity: I=Information,...
Starting VCS processes You can choose to configure different optional features, such as the following: SNMP and SMTP notification The Symantec Product Authentication Services feature The wide area Global Cluster feature Review the highlights of the information for which installvcs program prompts you as you proceed to configure.
Installing and configuring VCS About the VCS installation program installvcs optional features (continued) Table 4-1 Optional action Reference Perform secure installations using the values “Installing VCS with a response file that are stored in a configuration file. where ssh or rsh are disabled” on page 171.
Page 57
Installing and configuring VCS About the VCS installation program installvcs [ system1 system2... ] [ options ] Table 4-2 lists the installvcs command options. installvcs options Table 4-2 Option and Syntax Description Configure VCS after using -installonly option to install -configure VCS.
Page 58
NFS mounted. Verify that systems meet the installation requirements before -precheck proceeding with VCS installation. Symantec recommends doing a precheck before installing VCS. “Performing automated pre-installation check” on page 49. Displays all required VCS packages in correct installation -requiredpkgs order.
Enable or disable Symantec Product Authentication Service -security in a VCS cluster that is running. Install and configure Root Broker for Symantec Product Authentication Service.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 cluster configuration. The installvcs program prompts for cluster information, and creates VCS configuration files without performing installation. “Configuring the basic cluster” on page 67. option can be used to reconfigure a VCS cluster. VCS must not -configure be running on systems when this reconfiguration is performed.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Installation and configuration tasks Table 4-3 Task Reference License and install VCS “Starting the software installation” on page 61. “Specifying systems for installation” on page 62. “Licensing VCS” on page 63. “Choosing VCS RPMs for installation”...
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Note: The system from where you install VCS must run the same Linux distribution as the target systems. To install VCS using the product installer Confirm that you are logged in as the superuser and mounted the product disc.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 To specify system names for installation Enter the names of the systems where you want to install VCS. Enter the system names separated by spaces on which to install VCS: galaxy nebula For a single node installation, enter one name for the system.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Enter keys for additional product features. Do you want to enter another license key for galaxy? [y,n,q,?] (n) y Enter a VCS license key for galaxy: [?] XXXX-XXXX-XXXX-XXXX-XXX XXXX-XXXX-XXXX-XXXX-XXX successfully registered on galaxy Do you want to enter another license key for galaxy? [y,n,q,?] Review the output as the installer registers the license key on the other nodes.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 To install VCS RPMs Review the output as the installer checks the RPMs that are already installed. Choose the VCS RPMs that you want to install. Select the RPMs to be installed on all systems? [1-3,q,?] (3) 2 Based on what RPMs you want to install, enter one of the following: Installs only the required VCS RPMs.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 To install VCS packages now and configure VCS later If you do not want to configure VCS now, enter n at the prompt. Are you ready to configure VCS? [y,n,q] (y) n The utility checks for the required file system space and makes sure that any processes that are running do not conflict with the installation.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 To configure VCS using the installvcs program Confirm that you are logged in as the superuser and mounted the product disc. Navigate to the folder that contains the installvcs program. # cd /dvdrom/cluster_server Start the installvcs program.
Page 68
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 To configure the cluster Review the configuration instructions that the installer presents. Enter the unique cluster name and cluster ID. Enter the unique cluster name: [?] clus1 Enter the unique Cluster ID number between 0-65535: [b,?] 7 Review the NICs available on the first system as the installer discovers and reports them.
“Preparing to configure the clusters in secure mode” on page 29. To configure the cluster in secure mode Choose whether to configure VCS to use Symantec Product Authentication Service. Would you like to configure VCS to use Symantec Security Services? [y,n,q] (n) y...
Page 70
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Option 1. Automatic Enter the name of the Root Broker system when prompted. configuration Requires a remote access to the Root Broker. Review the output as the installer verifies communication with the Root Broker system, checks vxatd process and version, and checks security domain.
See Veritas Cluster Server User's Guide for more information. Adding VCS users If you have enabled Symantec Product Authentication Service, you do not need to add VCS users now. Otherwise, on systems operating under an English locale, you can add VCS users at this time.
Page 72
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Refer to the Veritas Cluster Server User s Guide for more information. To configure SMTP email notification Review the required information to configure the SMTP email notification. Specify whether you want to configure the SMTP notification. Do you want to configure SMTP notification? [y,n,q] (y) y If you do not want to configure the SMTP notification, you can skip to the next configuration option.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 If you do not want to add, answer n. Would you like to add another SMTP recipient? [y,n,q,b] (n) Verify and confirm the SMTP notification information. SMTP Address: smtp.example.com Recipient: ozzie@example.com receives email for Warning or higher events Recipient: harriet@example.com receives email for Error or higher events...
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Enter the SNMP console system name: [b,?] saturn Enter the minimum security level of messages to be sent to each console. Enter the minimum severity of events for which SNMP traps should be sent to saturn [I=Information, W=Warning, E=Error, S=SevereError]: [b,?] E Add more SNMP consoles, if necessary.
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 information to the VCS configuration file. You must perform additional configuration tasks to set up a global cluster. See Veritas Cluster Server User's Guide for instructions to set up VCS global clusters.
Cluster Server configured successfully. If you chose to configure the cluster in secure mode, the installer also configures the Symantec Product Authentication Service. Depending on the mode you chose to set up Authentication Service, the installer does one of the following:...
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 CPI WARNING V-9-122-1021 No PERSISTENT_NAME set for NIC with MAC address 00:11:43:33:17:28 (present name eth0), though config file exists! CPI WARNING V-9-122-1022 No config file for NIC with MAC address 00:11:43:33:17:29 (present name eth1) found! CPI WARNING V-9-122-1022 No config file for NIC with MAC address 00:04:23:ac:25:1f...
LDAP authentication. The AT component packaged with VCS requires you to manually edit the VRTSatlocal.conf file to enable LDAP authentication. Refer to the Symantec Product Authentication Service Administrator s Guide for instructions. If you have not already added VCS users during installation, you can add the users later.
Page 79
The LDAP schema and syntax for LDAP commands (such as, ldapadd, ldapmodify, and ldapsearch) vary based on your LDAP implementation. Before adding the LDAP domain in Symantec Product Authentication Service, note the following information about your LDAP environment: The type of LDAP schema used (the default is RFC 2307)
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Distinguished name for the user container (for example, UserBaseDN=ou=people,dc=comp,dc=com) Distinguished name for the group container (for example, GroupBaseDN=ou=group,dc=comp,dc=com) Installing the Java Console You can administer VCS using the VCS Java-based graphical user interface, Java Console.
Page 81
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Note: Symantec recommends using Pentium III, 400MHz, 256MB RAM, and 800x600 display resolution. The version of the Java™ 2 Runtime Environment (JRE) requires 32 megabytes of RAM. This version is supported on the Intel Pentium platforms that run the Linux kernel v 2.2.12 and glibc v2.1.2-11 (or later).
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 Go to \windows\VCSWindowsInstallers\ClusterManager. Open the language folder of your choice, for example EN. Double-click setup.exe. The Veritas Cluster Manager Install Wizard guides you through the installation process. Installing VCS Simulator You can administer VCS Simulator from the Java Console or from the command line.
Page 83
Installing and configuring VCS Installing and configuring VCS 5.0 RU3 To install VCS Simulator on Windows systems Insert the VCS installation disc into a drive. Navigate to the path of the Simulator installer file: \your_platform_architecture\cluster_server\windows\ VCSWindowsInstallers\Simulator Double-click the installer file. Read the information in the Welcome screen and click Next.
Installing and configuring VCS Verifying and updating licenses on the system Verifying the cluster after installation When you have used installvcs program and chosen to configure and start VCS, VCS and all components are properly configured and can start correctly. You must verify that your cluster operates properly after the installation.
Installing and configuring VCS Verifying and updating licenses on the system Reserved Mode = VCS Updating product licenses using vxlicinst You can use the command to add the VCS license key on each node. If vxlicinst you have VCS already installed and configured and you use a demo license, you can replace the demo license.
Accessing the VCS documentation The software disc contains the documentation for VCS in Portable Document Format (PDF) in the cluster_server/docs directory. After you install VCS, Symantec recommends that you copy the PDF version of the documents to the /opt/VRTS/docs directory on each node to make it available for reference.
Chapter Configuring VCS clusters for data integrity This chapter includes the following topics: About configuring VCS clusters for data integrity About I/O fencing components About setting up disk-based I/O fencing Preparing to configure disk-based I/O fencing Setting up disk-based I/O fencing manually About configuring VCS clusters for data integrity When a node fails, VCS takes corrective action and configures its components to reflect the altered membership.
Configuring VCS clusters for data integrity About I/O fencing components If a system is so busy that it appears to stop responding, the other nodes could declare it as dead. This declaration may also occur for the nodes that use the hardware that supports a "break"...
I/O fencing uses SCSI-3 disk policy that is either raw or dmp based on the disk device that you use. The disk policy is raw by default. Symantec recommends using the DMP disk policy. See the Veritas Volume Manager Administrator s Guide.
Page 90
Configuring VCS clusters for data integrity About setting up disk-based I/O fencing Workflow to configure disk-based I/O fencing Figure 5-1 Preparing to set up I/O fencing Initialize disks as VxVM disks Identify disks to use as coordinator disks Check shared disks for I/O fencing compliance Setting up I/O fencing Set up coordinator disk group Create I/O fencing configuration files...
Page 91
Each of the coordinator disks must use a physically separate disk or LUN. Symantec recommends using the smallest possible LUNs for coordinator disks. Each of the coordinator disks should exist on a different disk array, if possible. The coordinator disks must support SCSI-3 persistent reservations.
Configuring VCS clusters for data integrity Preparing to configure disk-based I/O fencing /etc/vxfentab When you run the vxfen startup file to start I/O fencing, the script creates this /etc/vxfentab file on each node with a list of all paths to each coordinator disk. The startup script uses the contents of the /etc/vxfendg and /etc/vxfenmode files.
If the Array Support Library (ASL) for the array that you add is not installed, obtain and install it on each node before proceeding. The ASL for the supported storage device that you add is available from the disk array vendor or Symantec technical support.
Page 94
Configuring VCS clusters for data integrity Preparing to configure disk-based I/O fencing Verify that the ASL for the disk array is installed on each of the nodes. Run the following command on each node and examine the output to verify the installation of ASL.
Configuring VCS clusters for data integrity Preparing to configure disk-based I/O fencing Scan all disk drives and their attributes, update the VxVM device list, and reconfigure DMP with the new devices. Type: # vxdisk scandisks See the Veritas Volume Manager documentation for details on how to add and configure disks.
Page 96
Configuring VCS clusters for data integrity Preparing to configure disk-based I/O fencing command option verifies that the same serial number for the LUN is returned on all paths to the LUN. Make sure to test the disks that serve as coordinator disks. The vxfentsthdw utility has additional options suitable for testing many disks.
Page 97
Configuring VCS clusters for data integrity Preparing to configure disk-based I/O fencing The same serial number information should appear when you enter the equivalent command on node B using the /dev/sdy path. On a disk from another manufacturer, Hitachi Data Systems, the output is different and may resemble: # /sbin/vxfenadm -i /dev/sdz SCSI ID=>Host: 2 Channel: 0 Id: 0 Lun: E...
Page 98
Configuring VCS clusters for data integrity Preparing to configure disk-based I/O fencing If you use for communication: # /opt/VRTSvcs/vxfen/bin/vxfentsthdw -n The script warns that the tests overwrite data on the disks. After you review the overview and the warning, confirm to continue the process and enter the node names.
Configuring VCS clusters for data integrity Setting up disk-based I/O fencing manually Setting up disk-based I/O fencing manually Make sure you completed the preparatory tasks before you set up I/O fencing. Tasks that are involved in setting up I/O fencing include: Tasks to set up I/O fencing manually Table 5-1 Action...
Configuring VCS clusters for data integrity Setting up disk-based I/O fencing manually Deport the coordinator disk group: # vxdg deport vxfencoorddg Import the disk group with the option to avoid automatically importing it when the nodes restart: # vxdg -t import vxfencoorddg Deport the disk group.
Configuring VCS clusters for data integrity Setting up disk-based I/O fencing manually # cp /etc/vxfen.d/vxfenmode_scsi3_raw /etc/vxfenmode To check the updated /etc/vxfenmode configuration, enter the following command on one of the nodes. For example: # more /etc/vxfenmode Modifying VCS configuration to use I/O fencing After you add coordinator disks and configure I/O fencing, add the UseFence = SCSI3 cluster attribute to the VCS configuration file /etc/VRTSvcs/conf/config/main.cf.
Configuring VCS clusters for data integrity Setting up disk-based I/O fencing manually Save and close the file. Verify the syntax of the file /etc/VRTSvcs/conf/config/main.cf: # hacf -verify /etc/VRTSvcs/conf/config Using rcp or another utility, copy the VCS configuration file from a node (for example, galaxy) to the remaining cluster nodes.
Chapter Verifying the VCS installation This chapter includes the following topics: About verifying the VCS installation About the LLT and GAB configuration files About the VCS configuration file main.cf Verifying the LLT, GAB, and VCS configuration files Verifying LLT, GAB, and cluster operation About verifying the VCS installation After you install and configure VCS, you can inspect the contents of the key VCS configuration files that you have installed and modified during the process.
Page 106
Verifying the VCS installation About the LLT and GAB configuration files The file is a database that contains one entry per system. This file llthosts links the LLT system ID (in the first column) with the LLT host name. This file is identical on each node in the cluster.
Verifying the VCS installation About the VCS configuration file main.cf Note: The use of the option for is not recommended. -c -x /sbin/gabconfig About the VCS configuration file main.cf The VCS configuration file /etc/VRTSvcs/conf/config/main.cf is created during the installation process. “Sample main.cf file for VCS clusters”...
Verifying the VCS installation About the VCS configuration file main.cf Refer to the Veritas Cluster Server User's Guide to review the configuration concepts, and descriptions of main.cf and types.cf files for Linux for IBM Power systems. Sample main.cf file for VCS clusters The following sample main.cf file is for a secure cluster that is managed locally by the Cluster Management Console.
Verifying the VCS installation About the VCS configuration file main.cf SystemList = { galaxy = 0, nebula = 1 } Parallel = 1 OnlineRetryLimit = 3 OnlineRetryInterval = 120 Phantom phantom_vxss ( ProcessOnOnly vxatd ( IgnoreArgs = 1 PathName = "/opt/VRTSat/bin/vxatd" // resource dependency tree // group VxSS // {...
Page 111
Verifying the VCS installation About the VCS configuration file main.cf In the following main.cf file example, bold text highlights global cluster specific entries. include "types.cf" cluster vcs03 ( ClusterAddress = "10.182.13.50" SecureClus = 1 system sysA ( system sysB ( system sysC ( group ClusterService ( SystemList = { sysA = 0, sysB = 1, sysC = 2 }...
Verifying the VCS installation Verifying the LLT, GAB, and VCS configuration files PathName = "/opt/VRTSat/bin/vxatd" // resource dependency tree group VxSS Phantom phantom_vxss ProcessOnOnly vxatd Verifying the LLT, GAB, and VCS configuration files Make sure that the LLT, GAB, and VCS configuration files contain the information you provided during VCS installation and configuration.
Verifying the VCS installation Verifying LLT, GAB, and cluster operation Verify LLT operation. “Verifying LLT” on page 114. Verify GAB operation. “Verifying GAB” on page 116. Verify the cluster operation. “Verifying the cluster” on page 117. Verifying LLT Use the command to verify that links are active for LLT.
Page 115
Verifying the VCS installation Verifying LLT, GAB, and cluster operation To view additional information about LLT, run the command lltstat -nvv on each node. For example, run the following command on the node galaxy in a two-node cluster: lltstat -nvv | more The output on galaxy resembles: Node State...
Verifying the VCS installation Verifying LLT, GAB, and cluster operation However, the output in the example shows different details for the node nebula. The private network connection is possibly broken or the information in the /etc/llttab file may be incorrect. To obtain information about the ports open for LLT, type on any lltstat -p...
Verifying the VCS installation Verifying LLT, GAB, and cluster operation To verify GAB To verify that GAB operates, type the following command on each node: /sbin/gabconfig -a Review the output of the command: If GAB operates, the following GAB port membership information is returned: GAB Port Memberships ===================================...
Verifying the VCS installation Verifying LLT, GAB, and cluster operation To verify the cluster To verify the status of the cluster, type the following command: hastatus -summary The output resembles: -- SYSTEM STATE -- System State Frozen galaxy RUNNING nebula RUNNING -- GROUP STATE -- Group...
Page 119
Verifying the VCS installation Verifying LLT, GAB, and cluster operation The example shows the output when the command is run on the node galaxy. The list continues with similar information for nebula (not shown) and any other nodes in the cluster. #System Attribute Value...
Page 120
Verifying the VCS installation Verifying LLT, GAB, and cluster operation #System Attribute Value galaxy Limits galaxy LinkHbStatus eth1 UP eth2 UP galaxy LoadTimeCounter galaxy LoadTimeThreshold galaxy LoadWarningLevel galaxy NoAutoDisable galaxy NodeId galaxy OnGrpCnt galaxy ShutdownTimeout galaxy SourceFile ./main.cf galaxy SysInfo Linux:galaxy,#1 SMP Mon Dec 12 18:32:25 UTC 2005,2.6.5-7.244-pseries64,ppc64...
Chapter Adding and removing cluster nodes This chapter includes the following topics: About adding and removing nodes Adding a node to a cluster Removing a node from a cluster About adding and removing nodes After you install VCS and create a cluster, you can add and remove nodes from the cluster.
Adding and removing cluster nodes Adding a node to a cluster Tasks that are involved in adding a node to a cluster (continued) Table 7-1 Task Reference Install the software “Preparing for a manual installation when adding a node” manually. on page 123.
Adding and removing cluster nodes Adding a node to a cluster Adding a node to a two-node cluster using two switches Figure 7-1 Public network Private network New node: saturn To set up the hardware Connect the VCS private Ethernet controllers. Perform the following tasks as necessary: When you add nodes to a two-node cluster, use independent switches or hubs for the private network connections.
RPMs are installed in the /opt directory. When you select the optional RPMs, review the following information: Symantec recommends that you install the RPMs for VCS manual pages (VRTSvcsmn). The I/O fencing RPM (VRTSvxfen) can be used only with the shared disks that support SCSI-3 Persistent Reservations (PR).
Adding and removing cluster nodes Adding a node to a cluster Setting up the node to run in secure mode You must follow this procedure only if you are adding a node to a cluster that is running in secure mode. If you are adding a node to a cluster that is not running in a secure mode, proceed with configuring LLT and GAB.
Page 127
Adding and removing cluster nodes Adding a node to a cluster # vssat deletecred --domain type:domainname \ --prplname prplname For example: # vssat deletecred --domain vx:root@RB2.brokers.example.com \ --prplname saturn.nodes.example.com Configuring the authentication broker on node saturn Configure a new authentication broker (AB) on node saturn. This AB belongs to root broker RB1.
Adding and removing cluster nodes Adding a node to a cluster Configure AB on node saturn to talk to RB1. # vxatd -o -a -n prplname -p password -x vx -y domainname -q \ rootbroker -z 2821 -h roothash_file_path For example: # vxatd -o -a -n saturn.nodes.example.com -p flurbdicate \ -x vx -y root@RB1.brokers.example.com -q RB1 \ -z 2821 -h roothash_file_path...
Page 129
Create the file /etc/gabtab on the new system. If the /etc/gabtab file on the existing nodes resembles: /sbin/gabconfig -c The file on the new node should be the same. Symantec recommends that you use the option, where N is the number of cluster nodes.
Adding and removing cluster nodes Adding a node to a cluster flag indicates to VCS the number of nodes that must be ready to form a cluster before VCS starts. On the new node, run the command, to configure GAB: # /sbin/gabconfig -c To verify GAB On the new node, run the command:...
Adding and removing cluster nodes Removing a node from a cluster Stop VCS on the new node: # hastop -sys saturn Copy the main.cf file from an existing node to your new node: # rcp /etc/VRTSvcs/conf/config/main.cf \ saturn:/etc/VRTSvcs/conf/config/ Start VCS on the new node: # hastart If necessary, modify any new system attributes.
Adding and removing cluster nodes Removing a node from a cluster Tasks that are involved in removing a node Table 7-3 Task Reference Back up the configuration file. “Verifying the status of nodes and service groups” on page 132. Check the status of the nodes and the service groups.
Adding and removing cluster nodes Removing a node from a cluster To verify the status of the nodes and the service groups Make a backup copy of the current configuration file, main.cf. # cp -p /etc/VRTSvcs/conf/config/main.cf\ /etc/VRTSvcs/conf/config/main.cf.goodcopy Check the status of the systems and the service groups. # hastatus -summary -- SYSTEM STATE -- System...
Page 134
Adding and removing cluster nodes Removing a node from a cluster To remove or switch service groups from the departing node Switch failover service groups from the departing node. You can switch grp3 from node saturn to node nebula. # hagrp -switch grp3 -to nebula Check for any dependencies involving any service groups that run on the departing node;...
Page 135
Adding and removing cluster nodes Removing a node from a cluster Delete the departing node from the SystemList of service groups grp3 and grp4. # hagrp -modify grp3 SystemList -delete saturn # hagrp -modify grp4 SystemList -delete saturn For the service groups that run only on the departing node, delete the resources from the group before you delete the group.
N is not greater than the actual number of nodes in the cluster. When N is greater than the number of nodes, GAB does not automatically seed. Note: Symantec does not recommend the use of the option for -c -x...
Adding and removing cluster nodes Removing a node from a cluster Unloading LLT and GAB and removing VCS on the departing node Perform the tasks on the node that is departing the cluster. If you have configured VCS as part of the Storage Foundation and High Availability products, you may have to delete other dependent RPMs before you can delete all of the following ones.
Page 138
Adding and removing cluster nodes Removing a node from a cluster # rpm -e VRTSvlic # rpm -e VRTSperl # rpm -e VRTSpbx # rpm -e VRTSicsco # rpm -e VRTSatServer # rpm -e VRTSatClient Remove the LLT and GAB configuration files. # rm /etc/llttab # rm /etc/gabtab # rm /etc/llthosts...
Chapter Installing VCS on a single node This chapter includes the following topics: About installing VCS on a single node Creating a single-node cluster using the installer program Creating a single-node cluster manually Adding a node to a single-node cluster About installing VCS on a single node You can install VCS 5.0 RU3 on a single node.
Installing VCS on a single node Creating a single-node cluster using the installer program Tasks to create a single-node cluster using the installer Table 8-1 Task Reference Prepare for installation. “Preparing for a single node installation” on page 140. Install the VCS software on “Starting the installer for the single node cluster”...
Installing VCS on a single node Creating a single-node cluster manually Answer if you plan to incorporate the single node cluster into a multi-node cluster in the future. Continue with the installation. “Licensing VCS” on page 63. Creating a single-node cluster manually Table 8-2 specifies the tasks that you need to perform to install VCS on a single node.
Installing VCS on a single node Creating a single-node cluster manually “Preparing for a manual installation when adding a node” on page 123. “Installing VCS RPMs for a manual installation” on page 124. “Adding a license key” on page 125. Renaming the LLT and GAB startup files You may need the LLT and GAB startup files to upgrade the single-node cluster to a multiple-node cluster at a later time.
Installing VCS on a single node Adding a node to a single-node cluster Adding a node to a single-node cluster All nodes in the new cluster must run the same version of VCS. The example procedure refers to the existing single-node VCS node as Node A. The node that is to join Node A to form a multiple-node cluster is Node B.
Installing VCS on a single node Adding a node to a single-node cluster Setting up a node to join the single-node cluster The new node to join the existing single node running VCS must run the same version of operating system and patch level. To set up a node to join the single-node cluster Do one of the following tasks: If VCS is not currently running on Node B, proceed to step 2.
Installing VCS on a single node Adding a node to a single-node cluster To install and configure Ethernet cards for private network Shut down VCS on Node A. # hastop -local Shut down the node to get to the prompt: # shutdown -r now Install the Ethernet card on Node A.
Installing VCS on a single node Adding a node to a single-node cluster Freeze the service groups. # hagrp -freeze group -persistent Repeat this command for each service group in step 4. Make the configuration read-only. # haconf -dump -makero Stop VCS on Node A.
Page 147
Installing VCS on a single node Adding a node to a single-node cluster It handles the following tasks: Traffic distribution Heartbeat traffic Configured as described in the following sections. Setting up /etc/llthosts The file llthosts(4M) is a database. This file contains one entry per system that links the LLT system ID (in the first column) with the LLT host name.
Page 148
Installing VCS on a single node Adding a node to a single-node cluster LLT directives Table 8-4 Directive Description Assigns the system ID or symbolic name. The system ID number set-node must be unique for each system in the cluster, and must be in the range 0-31.
N systems are ready to form the cluster. By default, N is the number of systems in the cluster. Note: Symantec does not recommend the use of the option for -c -x .
Installing VCS on a single node Adding a node to a single-node cluster To reconfigure VCS on existing nodes On Node A, create the files /etc/llttab, /etc/llthosts, and /etc/gabtab. Use the files that are created on Node B as a guide, customizing the /etc/llttab for Node A.
Page 151
Installing VCS on a single node Adding a node to a single-node cluster To verify the nodes' configuration On Node B, check the cluster membership. # gabconfig -a Start the VCS on Node B. # hastart Verify that VCS is up on both nodes. # hastatus List the service groups.
Page 152
Installing VCS on a single node Adding a node to a single-node cluster...
Chapter Uninstalling VCS This chapter includes the following topics: About the uninstallvcs program Preparing to uninstall VCS Uninstalling VCS 5.0 RU3 About the uninstallvcs program You can uninstall VCS from all nodes in the cluster or from specific nodes in the cluster using the uninstallvcs program.
Uninstalling VCS Uninstalling VCS 5.0 RU3 If you have manually edited any of the VCS configuration files, you need to reformat them. Uninstalling VCS 5.0 RU3 You must meet the following conditions to use the uninstallvcs program to uninstall VCS on all nodes in the cluster at one time: Make sure that the communication exists between systems.
Uninstalling VCS Uninstalling VCS 5.0 RU3 Enter the names of the systems from which you want to uninstall VCS. The program performs system verification checks and asks to stop all running VCS processes. Enter to stop all the VCS processes. The program proceeds with uninstalling the software.
Page 156
Uninstalling VCS Uninstalling VCS 5.0 RU3 The uninstallvcs program is not available in /opt/VRTS/install.
Appendix Advanced VCS installation topics This appendix includes the following topics: Using the UDP layer for LLT Performing automated VCS installations Installing VCS with a response file where ssh or rsh are disabled Using the UDP layer for LLT VCS 5.0 RU3 provides the option of using LLT over the UDP (User Datagram Protocol) layer for clusters using wide-area networks and routers.
Page 158
Advanced VCS installation topics Using the UDP layer for LLT Make sure that the LLT private links are on different physical networks. If the LLT private links are not on different physical networks, then make sure that the links are on separate subnets. Set the broadcast address in /etc/llttab explicitly depending on the subnet for each link.
Page 159
Advanced VCS installation topics Using the UDP layer for LLT “Sample configuration: direct-attached links” on page 161. “Sample configuration: links crossing IP routers” on page 163. Table A-1 describes the fields of the link command that are shown in the /etc/llttab file examples.
Page 160
Advanced VCS installation topics Using the UDP layer for LLT Field description for set-addr command in /etc/llttab Table A-2 Field Description The ID of the cluster node; for example, 0. node-id The string that LLT uses to identify the link; for example link1, link tag-name link2,..
Page 161
Advanced VCS installation topics Using the UDP layer for LLT For example, with the following interfaces: For first network interface IP address=192.168.30.1, Broadcast address=192.168.30.255, Netmask=255.255.255.0 For second network interface IP address=192.168.31.1, Broadcast address=192.168.31.255, Netmask=Mask:255.255.255.0 Configuring the broadcast address for LLT For nodes on different subnets, set the broadcast address in /etc/llttab depending on the subnet that the links are on.
Page 162
Advanced VCS installation topics Using the UDP layer for LLT A typical configuration of direct-attached links that use LLT over Figure A-1 Node0 Node1 UDP Endpoint eth1; UDP Port = 50001; IP = 192.1.3.1; Link Tag = link2 eth1; 192.1.3.2; Link Tag = link2 Switch UDP Endpoint eth2;...
Page 163
Advanced VCS installation topics Using the UDP layer for LLT link link1 udp - udp 50000 - 192.1.2.2 192.1.2.255 link link2 udp - udp 50001 - 192.1.3.2 192.1.3.255 Sample configuration: links crossing IP routers Figure A-2 depicts a typical configuration of links crossing an IP router employing LLT over UDP.
Advanced VCS installation topics Performing automated VCS installations To perform automated installation Navigate to the folder containing the installvcs program. # cd /mnt/cdrom/cluster_server Start the installation from one of the cluster systems where you have copied the response file. # ./installvcs -responsefile /tmp/response_file Where /tmp/response_file is the response file s full path name.
Page 167
Advanced VCS installation topics Performing automated VCS installations Response file variables (continued) Table A-3 Variable Description $CPI::CFG{SYSTEMS} List of systems on which the product is to be installed, uninstalled, or configured. List or scalar: list Optional or required: required $CPI::CFG{SYSTEMSCFG} List of systems to be recognized in configuration if secure environment prevents all systems from being installed at once.
Page 168
Advanced VCS installation topics Performing automated VCS installations Response file variables (continued) Table A-3 Variable Description $CPI::CFG{OPT}{PKGPATH} Defines a location, typically an NFS mount, from which all remote systems can install product depots. The location must be accessible from all target systems. List or scalar: scalar Optional or required: optional $CPI::CFG{OPT}{TMPPATH}...
Page 169
Advanced VCS installation topics Performing automated VCS installations Response file variables (continued) Table A-3 Variable Description $CPI::CFG{KEYS} List of keys to be registered on the system. {SYSTEM} List or scalar: list Optional or required: optional $CPI::CFG{OPT_LOGPATH} Mentions the location where the log files are to be copied.
Page 170
Advanced VCS installation topics Performing automated VCS installations Response file variables (continued) Table A-3 Variable Description $CPI::CFG{VCS_SMTPRSEV} Defines the minimum severity level of messages (Information, Warning, Error, SevereError) that listed SMTP recipients are to receive. Note that the ordering of severity levels must match that of the addresses of SMTP recipients.
Advanced VCS installation topics Installing VCS with a response file where ssh or rsh are disabled Response file variables (continued) Table A-3 Variable Description $CPI::CFG{OPT}{UNINSTALL} List of systems where VCS must be uninstalled. List or scalar: scalar Optional or required: optional Installing VCS with a response file where ssh or rsh are disabled In secure enterprise environments, ssh or rsh communication is not allowed...
Page 172
Advanced VCS installation topics Installing VCS with a response file where ssh or rsh are disabled After the installation is complete, review the installer report. The installer stores the installvcs-universaluniqueidentifier response file in the /opt/VRTS/install/logs/installvcs-universaluniqueidentifier/.response directory where universaluniqueidentifier is a variable to uniquely identify the file.
Need help?
Do you have a question about the Veritas Cluster Server and is the answer not in the manual?
Questions and answers