Related Manuals for Red Hat GLOBAL FILE SYSTEM 5.2
Summary of Contents for Red Hat GLOBAL FILE SYSTEM 5.2
- Page 1 Global File System Red Hat Global File System Global_File_System ISBN: N/A Publication date: May 2008...
- Page 2 Global File System This book provides information about installing, configuring, and maintaining Red Hat GFS (Red Hat Global File System) for Red Hat Enterprise Linux 5.2.
- Page 3 All other trademarks referenced herein are the property of their respective owners. The GPG fingerprint of the security@redhat.com key is: CA 20 86 86 2B D6 9D FC 65 F6 EC C4 21 91 80 CD DB 42 A6 0E...
- Page 4 Global File System...
-
Page 5: Table Of Contents
Introduction ......................vii 1. Audience ...................... vii 2. Related Documentation ................. vii 3. Document Conventions ................viii 4. Feedback ...................... ix 1. GFS Overview ....................... 1 1. New and Changed Features ................2 2. Performance, Scalability, and Economy ............2 2.1. -
Page 7: Introduction
Configuring and Managing a Red Hat Cluster. HTML and PDF versions of all the official Red Hat Enterprise Linux manuals and release notes are available online at http://www.redhat.com/docs/. 1. Audience This book is intended primarily for Linux system administrators who are familiar with the following activities: •... -
Page 8: Document Conventions
Red Hat Cluster Suite documentation and other Red Hat documents are available in HTML, PDF, and RPM versions on the Red Hat Enterprise Linux Documentation CD and online at http://www.redhat.com/docs/. 3. Document Conventions Certain words in this manual are represented in different fonts, styles, and weights. This highlighting indicates that the word is part of a specific category. -
Page 9: Feedback
4. Feedback If you spot a typo, or if you have thought of a way to make this manual better, we would love to hear from you. Please submit a report in Bugzilla (http://bugzilla.redhat.com/bugzilla/) against the component rh-cs... - Page 10 Introduction Bugzilla component: Documentation-cluster Book identifier: Global_File_System(EN)-5.2 (2008-05-21T15:10) By mentioning this manual's identifier, we know exactly which version of the guide you have. If you have a suggestion for improving the documentation, try to be as specific as possible. If you have found an error, please include the section number and some of the surrounding text so we can find it easily.
-
Page 11: Gfs Overview
Chapter 1. GFS Overview The Red Hat GFS file system is a native file system that interfaces directly with the Linux kernel file system interface (VFS layer). A GFS file system can be implemented in a standalone system or as part of a cluster configuration. When implemented as a cluster file system, GFS employs distributed metadata and multiple journals. -
Page 12: New And Changed Features
Chapter 1. GFS Overview 1. New and Changed Features This section lists new and changed features included with the initial release of Red Hat Enterprise Linux 5. • GULM (Grand Unified Lock Manager) is not supported in Red Hat Enterprise Linux 5. If your GFS file systems use the GULM lock manager, you must convert the file systems to use the DLM lock manager. -
Page 13: Economy And Performance
Economy and Performance The GFS SAN configuration in Figure 1.1, “GFS with a SAN” provides superior file performance for shared files and file systems. Linux applications run directly on GFS nodes. Without file protocols or storage servers to slow data access, performance is similar to individual Linux servers with directly connected storage;... - Page 14 Chapter 1. GFS Overview Figure 1.2. GFS and GNBD with a SAN Figure 1.3, “GFS and GNBD with Directly Connected Storage” shows how Linux client applications can take advantage of an existing Ethernet topology to gain shared access to all block storage devices.
-
Page 15: Gfs Software Components
GFS Software Components Figure 1.3. GFS and GNBD with Directly Connected Storage 3. GFS Software Components Table 1.1, “GFS Software Subsystem Components” summarizes the GFS software components. Software Component Description Kernel module that implements the GFS file system and is gfs.ko loaded on GFS cluster nodes. -
Page 16: Before Setting Up Gfs
Chapter 1. GFS Overview 4. Before Setting Up GFS Before you install and set up GFS, note the following key characteristics of your GFS file systems: GFS nodes Determine which nodes in the Red Hat Cluster Suite will mount the GFS file systems. Number of file systems Determine how many GFS file systems to create initially. -
Page 17: Getting Started
Chapter 2. Getting Started This chapter describes procedures for initial setup of GFS and contains the following sections: • Section 1, “Prerequisite Tasks” • Section 2, “Initial Setup Tasks” 1. Prerequisite Tasks Before setting up Red Hat GFS, make sure that you have noted the key characteristics of the GFS nodes (refer to Section 4, “Before Setting Up GFS”). - Page 18 Chapter 2. Getting Started 2. Create GFS file systems on logical volumes created in Step 1. Choose a unique name for each file system. For more information about creating a GFS file system, refer to Section 1, “Creating a File System”.
-
Page 19: Managing Gfs
Chapter 3. Managing GFS This chapter describes the tasks and commands for managing GFS and consists of the following sections: • Section 1, “Creating a File System” • Section 2, “Mounting a File System” • Section 3, “Unmounting a File System” •... - Page 20 Chapter 3. Managing GFS system options. Usage When creating a clustered GFS filesystem, you can use either of the following formats: gfs_mkfs -p LockProtoName -t LockTableName -j NumberJournals BlockDevice mkfs -t gfs -p LockProtoName -t LockTableName -j NumberJournals BlockDevice When creating a local filesystem, you can use either of the following formats: gfs_mkfs -p LockProtoName -j NumberJournals BlockDevice mkfs -t gfs -p LockProtoName -j NumberJournals BlockDevice Warning...
- Page 21 Examples required for each node that mounts the file system. (More journals than are needed can be specified at creation time to allow for future expansion.) BlockDevice Specifies a volume. Examples In these examples, is the locking protocol that the file system uses, since this is a lock_dlm clustered file system.
- Page 22 Chapter 3. Managing GFS gfs_mkfs -p lock_dlm -t alpha:mydata2 -j 8 /dev/vg01/lvol1 mkfs -t gfs -p lock_dlm -t alpha:mydata2 -j 8 /dev/vg01/lvol1 Complete Options Table 3.1, “Command Options: ” describes the command options. gfs_mkfs gfs_mkfs Flag Parameter Description Sets the file system block size to BlockSize BlockSize Default block size is 4096 bytes.
-
Page 23: Mounting A File System
Mounting a File System Flag Parameter Description LockTableName Used in a clustered file system. This parameter has two parts separated by a colon (no spaces) as follows: ClusterName:FSName is the name of the Red Hat cluster for ClusterName which the GFS file system is being created. The cluster name is set in the file via the Cluster /etc/cluster/cluster.conf... -
Page 24: Mount Options
Chapter 3. Managing GFS -o acl GFS-specific option to allow manipulating file ACLs. BlockDevice Specifies the block device where the GFS file system resides. MountPoint Specifies the directory where the GFS file system should be mounted. Example In this example, the GFS file system on is mounted on the /dev/vg01/lvol0 /mydata1... -
Page 25: Unmounting A File System
Unmounting a File System Option Description file system. By default, using lock_nolock ignore_local_fs automatically turns on the localcaching Caution: This option should not be used flags. localflocks when GFS file systems are shared. Tells GFS that it is running as a local file system. localcaching GFS can then turn on selected optimization Caution: This option should not be used... -
Page 26: Unmounting A File System
Chapter 3. Managing GFS 3. Unmounting a File System The GFS file system can be unmounted the same way as any Linux file system — by using the command. umount Note command is a Linux system command. Information about this umount command can be found in the Linux command man pages. - Page 27 Examples gfs_tool gettune MountPoint MountPoint Specifies the directory where the GFS file system is mounted. Examples In this example, all GFS tunable parameters for the file system on the mount point /mnt/gfs displayed. [root@tng3-1]# gfs_tool gettune /mnt/gfs ilimit1 = 100 ilimit1_tries = 3 ilimit1_min = 1 ilimit2 = 500...
-
Page 28: Gfs Quota Management
Chapter 3. Managing GFS statfs_fast = 0 5. GFS Quota Management File-system quotas are used to limit the amount of file system space a user or group can use. A user or group does not have a quota limit until one is set. GFS keeps track of the space used by each user and group even when there are no limits in place. -
Page 29: Displaying Quota Limits And Usage
Displaying Quota Limits and Usage gfs_quota warn -g Group -l Size -f MountPoint User A user ID to limit or warn. It can be either a user name from the password file or the UID number. Group A group ID to limit or warn. It can be either a group name from the group file or the GID number. - Page 30 Chapter 3. Managing GFS gfs_quota get -u User -f MountPoint Displaying Quota Limits for a Group gfs_quota get -g Group -f MountPoint Displaying Entire Quota File gfs_quota list -f MountPoint User A user ID to display information about a specific user. It can be either a user name from the password file or the UID number.
-
Page 31: Synchronizing Quotas
Synchronizing Quotas The hard limit set for the user or group. This value is zero if no limit has been set. Value The actual amount of disk space used by the user or group. Comments When displaying quota information, the command does not resolve UIDs and GIDs gfs_quota into names if the... -
Page 32: Disabling/Enabling Quota Enforcement
Chapter 3. Managing GFS You can use the command to synchronize the quota information from a node gfs_quota sync to the on-disk quota file between the automatic updates performed by GFS. Usage Synchronizing Quota Information gfs_quota sync -f MountPoint MountPoint Specifies the GFS file system to which the actions apply. -
Page 33: Disabling/Enabling Quota Accounting
Disabling/Enabling Quota Accounting enforcement is done by changing a tunable parameter, , with the quota_enforce gfs_tool command. The parameter must be disabled or enabled on each node where quota_enforce quota enforcement should be disabled/enabled. Each time the file system is mounted, enforcement is enabled by default. - Page 34 Chapter 3. Managing GFS tunable parameter to 0. This must be done on each node and after each mount. quota_account (The 0 setting is not persistent across unmounts.) Quota accounting can be enabled by setting tunable parameter to 1. quota_account To see the current values of the GFS tunable parameters, including , you can quota_account...
-
Page 35: Growing A File System
Growing a File System # gfs_tool settune /gfs quota_account 1 # gfs_quota init -f /gfs 6. Growing a File System command is used to expand a GFS file system after the device where the file gfs_grow system resides has been expanded. Running a command on an existing GFS file gfs_grow system fills all spare space between the current end of the file system and the end of the device... - Page 36 Chapter 3. Managing GFS After running the command, you can run a command on the file gfs_grow df MountPoint system to check that the new space is now available in the file system. Examples In this example, the underlying logical volume for the file system file system on the /mnt/gfs directory is extended, and then the file system is expanded.
-
Page 37: Adding Journals To A File System
Usage 7. Adding Journals to a File System command is used to add journals to a GFS file system after the device where the gfs_jadd file system resides has been expanded. Running a command on a GFS file system gfs_jadd uses space between the current end of the file system and the end of the device where the file system resides. - Page 38 Chapter 3. Managing GFS SB multihost format = 1401 Block size = 4096 Journals = 8 Resource Groups = 76 Mounted lock proto = "lock_dlm" Mounted lock table = "tng3-cluster:mydata1" Mounted host data = "jid=0:id=196609:first=1" Journal number = 0 Lock module flags = 0 Local flocks = FALSE Local caching = FALSE Oopses OK = FALSE...
-
Page 39: Direct I/O
Complete Usage Complete Usage gfs_jadd [Options] {MountPoint | Device} [MountPoint | Device] MountPoint Specifies the directory where the GFS file system is mounted. Device Specifies the device node of the file system. Table 3.4, “GFS-specific Options Available When Adding Journals” describes the GFS-specific options that can be used when adding journals to a GFS file system. -
Page 40: O_Direct
Chapter 3. Managing GFS Direct I/O is a feature of the file system whereby file reads and writes go directly from the applications to the storage device, bypassing the operating system read and write caches. Direct I/O is used only by applications (such as databases) that manage their own caches. An application invokes direct I/O by opening a file with the flag. -
Page 41: Gfs Directory Attribute
GFS Directory Attribute File Specifies the file where the flag is assigned. directio Example In this example, the command sets the flag on the file named in directory directio datafile /mnt/gfs gfs_tool setflag directio /mnt/gfs/datafile The following command checks whether the flag is set for . -
Page 42: Data Journaling
Chapter 3. Managing GFS Directory Specifies the directory where the flag is set. inherit_directio Example In this example, the command sets the flag on the directory named inherit_directio /mnt/gfs/data gfs_tool setflag inherit_directio /mnt/gfs/data This command displays the flags that have been set for the directory. -
Page 43: Configuring Updates
Examples gfs_tool setflag inherit_jdata Directory gfs_tool clearflag inherit_jdata Directory Setting and Clearing the Flag jdata gfs_tool setflag jdata File gfs_tool clearflag jdata File Directory Specifies the directory where the flag is set or cleared. File Specifies the zero-length file where the flag is set or cleared. Examples This example shows setting the flag on a directory. -
Page 44: Mount With Noatime
Chapter 3. Managing GFS Each file inode and directory inode has three time stamps associated with it: • — The last time the inode status was changed ctime • — The last time the file (or directory) data was modified mtime •... -
Page 45: Tune Gfs Quantum
Suspending Activity on a File System 10.2. Tune GFS Quantum atime When updates are enabled, GFS (by default) only updates them once an hour. The time atime quantum is a tunable parameter that can be adjusted using the command. gfs_tool Each GFS node updates the access time based on the difference between its system time and the time recorded in the inode. -
Page 46: Displaying Extended Gfs Information And Statistics
Chapter 3. Managing GFS Usage Start Suspension gfs_tool freeze MountPoint End Suspension gfs_tool unfreeze MountPoint MountPoint Specifies the file system. Examples This example suspends writes to file system /gfs gfs_tool freeze /gfs This example ends suspension of writes to file system /gfs gfs_tool unfreeze /gfs 12. -
Page 47: Displaying Gfs Counters
Displaying GFS Counters MountPoint Specifies the file system to which the action applies. Example This example reports extended file system usage about file system /mnt/gfs [root@ask-07 ~]# gfs_tool df /mnt/gfs /gfs: SB lock proto = "lock_dlm" SB lock table = "ask_cluster:mydata1" SB ondisk format = 1309 SB multihost format = 1401 Block size = 4096... - Page 48 Chapter 3. Managing GFS The number of structures that currently exist in gfs. gfs_glock locks held The number of existing structures that are not in the state. gfs_glock UNLOCKED freeze count A freeze count greater than 0 means the file system is frozen. A freeze count of 0 means the file system is not frozen.
- Page 49 Displaying GFS Counters glocks reclaimed The number of glocks which have been reclaimed. glock dq calls The number of glocks released since the file system was mounted. glock prefetch calls The number of glock prefetch calls. lm_lock calls The number of times the lock manager has been contacted to obtain a lock. lm_unlock calls The number of times the lock manager has been contacted to release a lock.
-
Page 50: Displaying Extended Status
Chapter 3. Managing GFS MountPoint Specifies the file system to which the action applies. Example This example reports statistics about the file system mounted at /mnt/gfs [root@tng3-1 gfs]# gfs_tool counters /mnt/gfs locks 165 locks held 133 freeze count 0 incore inodes 34 metadata buffers 5 unlinked inodes 0 quota IDs 0... - Page 51 Displaying Extended Status Note The information that the command displays reflects internal file gfs_tool stat system information. This information is intended for development purposes only. Usage gfs_tool stat File File Specifies the file from which to get information. Example This example reports extended file status about file /gfs/datafile [root@tng3-1 gfs]# gfs_tool stat /gfs/datafile mh_magic = 0x01161970...
-
Page 52: Repairing A File System
Chapter 3. Managing GFS di_entries = 0 no_formal_ino = 0 no_addr = 0 di_eattr = 0 di_reserved = 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 13. - Page 53 Usage other command options. Usage gfs_fsck -y BlockDevice flag causes all questions to be answered with . With the flag specified, the command does not prompt you for an answer before making changes. gfs_fsck BlockDevice Specifies the block device where the GFS file system resides. Example In this example, the GFS file system residing on block device is repaired.
-
Page 54: Context-Dependent Path Names
Chapter 3. Managing GFS Starting pass1c Looking for inodes containing ea blocks... Pass1c complete Starting pass2 Checking directory inodes. Pass2 complete Starting pass3 Marking root inode connected Checking directory linkage. Pass3 complete Starting pass4 Checking inode reference counts. Pass4 complete Starting pass5 Updating Resource Group 92 Pass5 complete... - Page 55 Example Variable Specifies a special reserved name from a list of values (refer to Table 3.5, “CDPN Variable Values”) to represent one of multiple existing files or directories. This string is not the name of an actual file or directory itself. (The real files or directories must be created in a separate step using names that correlate with the type of variable used.) LinkName Specifies a name that will be seen and used by applications and will be followed to get to...
- Page 56 Chapter 3. Managing GFS n01# cd /gfs n01# mkdir n01 n02 n03 n01# ln -s @hostname log n01# ls -l /gfs lrwxrwxrwx 1 root root 9 Apr 25 14:04 log -> @hostname/ drwxr-xr-x 2 root root 3864 Apr 25 14:05 n01/ drwxr-xr-x 2 root root 3864 Apr 25 14:06 n02/ drwxr-xr-x 2 root root 3864 Apr 25 14:06 n03/ n01# touch /gfs/log/fileA...
-
Page 57: Index
O_DIRECT, 30 Index growing, 25 mounting, 13 quota management, 18 disabling/enabling quota accounting, 23 disabling/enabling quota enforcement, adding journals to a file system, 27 atime, configuring updates, 33 displaying quota limits, 19 mounting with noatime, 34 setting quotas, 18 tuning atime quantum, 35 synchronizing quotas, 21 audience, vii repairing, 42... - Page 58 Index initial tasks tables setup, initial, 7 CDPN variable values, 45 introduction, vii GFS software components, 5 audience, vii GFS-specific options for adding journals, 29 GFS-specific options for expanding file systems, 26 gfs_mkfs command options, 12 managing GFS, 9 mount options, 14 maximum size, GFS file system, 1, 6 tunable parameters, GFS, 16 mount table, 14...