| Sun StorageTek QFS File System Configuration and Administration Guide |    
|
| Sun StorageTek QFS File System Configuration and Administration Guide |
|
| C H A P T E R 1 |
|
File System Overview |
This chapter contains the following sections:
The Sun StorageTek QFS file system is a configurable file system that presents a standard UNIX file system (UFS) interface to users. TABLE 1-1 shows how the file system can be used or combined with the storage and archive management (Sun StorageTek SAM) software.
The Sun StorageTek QFS file system does not require changes to user programs or to the UNIX kernel. Some of the features of the Sun StorageTek QFS file system are described in the following sections.
Sun StorageTek QFS file systems support both striped and round-robin disk access. The master configuration file (mcf) and the mount parameters specify the volume management features and enable the file system to recognize the relationships between the devices it controls. This is in contrast to most UNIX file systems, which can address only one device or one portion of a device. Sun StorageTek QFS file systems do not require additional volume management applications. However, if you want to use mirroring for devices in a Sun StorageTek QFS environment, you must obtain an additional package, such as a logical volume manager.
The Sun StorageTek QFS integrated volume management features use the standard Solaris OS device driver interface to pass I/O requests to and from the underlying devices. The Sun StorageTek QFS software groups storage devices into family sets upon which each file system resides.
The Sun StorageTek QFS file system supports two different types of I/O: paged (also called cached or buffered I/O) and direct. These I/O types perform as follows:
The Sun StorageTek QFS software supports files of up to 263 bytes in length. Such very large files can be striped across many disks or RAID devices, even within a single file system. This is true because Sun StorageTek QFS file systems use true 64-bit addressing, in contrast to standard UNIX file systems, which are not true 64-bit file systems.
The number of file systems that you can configure is virtually unlimited. The volume manager enables each file system to include up to 252 device partitions, typically disk. Each partition can include up to 16 terabytes of data. This configuration offers virtually unlimited storage capacity.
There is no predefined limit on the number of files in a Sun StorageTek QFS file system. Because the inode space (which holds information about the files) is dynamically allocated, the maximum number of files is limited only by the amount of disk storage available. The inodes are cataloged in the .inodes file under the mount point. The .inodes file requires 512 bytes of storage per file.
In a Sun StorageTek QFS file system, the inodes are located on the metadata devices and are separated from the file data devices. In practice, the size of your metadata (mm) devices limits the number of files in a Sun StorageTek QFS file system, but you can increase the maximum number of files by adding more metadata devices. The hard limit on the number of files is 232-1 files, and the recommended limit is 107 files.
A key function of a file system is its ability to recover quickly after an unscheduled outage. Standard UNIX file systems require a lengthy file system check (fsck(1M)) to repair inconsistencies after a system failure.
A Sun StorageTek QFS file system often does not require a file system check after a disruption that prevents the file system from being written to disk (using sync(1M)). In addition, Sun StorageTek QFS file systems recover from system failures without using journaling. They accomplish this dynamically by using identification records, serial writes, and error checking for all critical I/O operations. After a system failure, even multiterabyte-sized Sun StorageTek QFS file systems can be remounted immediately.
File systems use metadata to reference file and directory information. Typically, metadata resides on the same device as the file data. However, the Sun StorageTek QFS file system has the option of separating the file system metadata from the file data by storing them on separate devices. The Sun StorageTek QFS file system enables you to define one or more separate metadata devices in order to reduce device head movement and rotational latency, improve RAID cache utilization, or mirror metadata without mirroring file data.
Sun StorageTek QFS file systems store inode metadata information in a separate file. This enables the number of files, and the file system as a whole, to be enlarged dynamically.
The Sun StorageTek QFS file system is implemented through the standard Solaris OS virtual file system (vfs/vnode) interface.
By using the vfs/vnode interface, the file system works with the standard Solaris OS kernel and requires no modifications to the kernel for file management support. Thus, the file system is protected from operating system changes and typically does not require extensive regression testing when the operating system is updated.
The kernel intercepts all requests for files, including those that reside in Sun StorageTek QFS file systems. If a file is identified as a Sun StorageTek QFS file, the kernel passes the request to the appropriate file system for handling. Sun StorageTek QFS file systems are identified as type samfs in the /etc/vfstab file and through the mount(1M) command.
A Sun StorageTek QFS shared file system is a distributed file system that can be mounted on multiple Solaris OS host systems. In a Sun StorageTek QFS shared file system environment, one Solaris OS host acts as the metadata server for the file system, and additional hosts can be configured as clients. You can configure more than one host as a potential metadata server, but only one host can be the metadata server at any one time. There is no limit to the number of Sun StorageTek QFS shared file system mount points.
The advantage of the Sun StorageTek QFS shared file system is that file data passes directly from the Fibre Channel disks to the hosts. Data travels via local path I/O (also known as direct access I/O). This is in contrast to the network file system (NFS), which transfers data over the network.
The shared file system can be implemented either as a Sun StorageTek QFS shared file system or as a SAM-QFS shared file system. It can use either ms or ma file system types.
Sun StorageTek QFS shared file systems do not support the following:
For more information about shared file systems, see Chapter 4, Configuring a Sun StorageTek QFS Shared File System.
Within a shared file system, the Sun StorageTek QFS software can be installed on Linux as well as Solaris clients. The Sun StorageTek QFS Linux Client software supports the following Linux distributions:
|
Note - This is the last version of Sun StorageTek QFS that will support SuSE Enterprise Linux 8. It will not be supported in the next version of the software. |
Unlike shared Sun StorageTek QFS Solaris clients, the Linux client is restricted to client-only behavior. It cannot be configured as a potential metadata server. The Linux client supports interaction with Sun StorageTek SAM software, but does not support Sun StorageTek SAM commands such as stage, archive, release, and samu. It has Sun StorageTek QFS file system functionality only.
For more information about the Sun StorageTek QFS Linux client software, see the Sun StorageTek QFS Linux Client Guide.
The following additional features are also supported by the Sun StorageTek QFS file system:
Sun StorageTek QFS file systems are multithreaded, advanced storage management systems. To take maximum advantage of the software's capabilities, create multiple file systems whenever possible.
Sun StorageTek QFS file systems use a linear search method for directory lookups, searching from the beginning of the directory to the end. As the number of files in a directory increases, the search time through the directory also increases. Search times can become excessive when you have directories with thousands of files. These long search times are also evident when you restore a file system. To increase performance and speed up file system dumps and restores, keep the number of files in a directory under 10,000.
The directory name lookup cache (DNLC) feature improves file system performance. This cache stores the directory lookup information for files whose paths are short (30 characters or less), removing the need for directory lookups to be performed on the fly. The DNLC feature is available in all Solaris OS 9 and later releases.
The following sections cover some additional features that affect file system design:
The types of files to be stored in a file system affect file system design. An inode is a 512-byte block of information that describes the characteristics of a file or directory. This information is allocated dynamically within the file system.
Inodes are stored in the .inodes file located under the file system mount point. A Sun StorageTek QFS .inodes file can reside on a metadata device that is separate from the file data device.
Like a standard Solaris OS inode, a Sun StorageTek QFS file system inode contains the file's POSIX standard inode times: file access, file modification, and inode changed times. A Sun StorageTek QFS file system inode includes other times as well, as shown in TABLE 1-2.
|
Note - If the WORM-FS (write once read many) package is installed, the inode also includes a retention-end date. See Configuring WORM-FS File Systems for more information. |
For more information on viewing inode file information, see Viewing Files and File Attributes.
Disk space is allocated in basic units of online disk storage called disk allocation units (DAUs). Whereas sectors, tracks, and cylinders describe the physical disk geometry, the DAU describes the file system geometry. Choosing the appropriate DAU size and stripe size can improve performance and optimize magnetic disk usage. The DAU setting is the minimum amount of contiguous space that is used when a file is allocated.
The following subsections describe how to configure DAU settings and stripe widths.
Sun StorageTek QFS file systems use an adjustable DAU. You can use this configurable DAU to tune the file system to the physical disk storage device. This feature minimizes the system overhead caused by read-modify-write operations and is therefore particularly useful for applications that manipulate very large files. For information about how to control the read-modify-write operation, see Increasing File Transfer Performance for Large Files.
Each file system can have its own unique DAU setting, even if it is one of several mounted file systems active on a server. The possible DAU settings differ depending on the type of file system you are using. The DAU setting is determined through the sammkfs(1M) command when the file system is created. It cannot be changed dynamically.
DAU settings work in conjunction with the device and file system definitions specified in the master configuration (mcf) file. For details about the mcf file, see Configuring the File System.
Two file allocation schemes are available to you: an ms file system type, and an ma file system type.
For a simple Sun StorageTek QFSSun StorageTek SAM file system, such as one on a single partition, the file system is defined in your mcf file by an Equipment Type value of ms. In the ms file system, the only device type allowed is type md, and both metadata and file data are written to the md devices. By default, the DAU on an md device is 64 kilobytes.
A more complex Sun StorageTek QFS file system installed on multiple partitions is defined as Equipment Type ma in your mcf file. In an ma file system, metadata is written to mm devices, and data can be written to md, mr, or gXXX devices.
Within an ma file system you can mix devices as follows:
For more information on these device types, see Table 2-2.
The md and mm devices use a dual allocation scheme, as follows:
|
Note - When using an ms file system, the stripe width should be set to greater than zero to stripe metadata information across the disk. However, you should read and understand Stripe Widths on Data Disks before setting the stripe width and DAU size. |
Depending on the type of file data stored in the file system, a larger DAU size can improve file system performance significantly. For information about tuning file system performance, see Chapter 7 Advanced Topics.
Only ma Sun StorageTek QFS file systems can include devices that use a single allocation scheme. These file systems consist of separate metadata devices and data devices, as follows:
The mr and gXXX devices follow a single allocation scheme. You can mix mr and gXXX devices in a file system, but you cannot mix md devices with either mr or gXXX devices in a file system.
The DAU size for Sun StorageTek QFS file systems that use mr and gXXX data devices is configurable. The possible DAU sizes that can be used on data devices depend on the Equipment Type value assigned to each data device in the mcf file. TABLE 1-3 shows these DAU sizes.
Data alignment refers to matching the allocation unit of the RAID controller with the allocation unit of the file system. The optimal Sun StorageTek QFS file system alignment formula is as follows:
allocation-unit = RAID-stripe-width x number-of-data-disks
For example, suppose a RAID-5 unit has nine disks, with one of the nine being the parity disk, making the number of data disks eight. If the RAID stripe width is 64 kilobytes, then the optimal allocation unit is 64 multiplied by 8, which is 512 kilobytes.
Data files are allocated as striped or round-robin through each striped group (gXXX) or data disk (mr or md) within the same file system.
A mismatched alignment hurts performance because it can cause a read-modify-write operation.
Stripe width defaults differ between Sun StorageTek QFS ms and ma file systems. The stripe width is specified by the -o stripe=n option in the mount(1M) command. If the stripe width is set to 0, round-robin allocation is used.
The following subsections describe stripe widths on the various file systems.
On ms file systems, the stripe width is set at mount time. TABLE 1-4 shows default stripe widths.
For example, if sammkfs(1M) is run with default settings, the default large DAU is 64 kilobytes. If no stripe width is specified when the mount(1M) command is issued, the default is used, and the stripe width set at mount time is 2.
|
Note - It is important that the stripe width be set to greater than zero in an ms file system so that metadata information is striped across the disk. |
Note that if you multiply the number in the first column of TABLE 1-4 by the number in the second column, the resulting number is 128 kilobytes. Sun StorageTek QFS file systems operate most efficiently if the amount of data being written to disk is at least 128 kilobytes.
On Sun StorageTek QFS ma file systems, the stripe width that is set at mount time depends on whether or not striped groups are configured. A striped group is a collection of devices that are striped as a group. For more information about striped groups, see File Allocation Methods. This section describes stripe widths for Sun StorageTek QFS file systems that are configured without stripe groups.
If striped groups are not configured, the DAU and stripe width relationships on ma file systems are similar to those for ms file systems. The difference is that DAUs larger than 64 kilobytes are possible and that the DAU is configurable in 8-kilobyte blocks. The maximum DAU size is 65,528 kilobytes.
By default, if no stripe width is specified, the amount of data written to disk is at or near 128 kilobytes. Sun StorageTek QFS file systems are most efficient if write operations write at least one whole stripe per I/O request. TABLE 1-5 shows the default stripe widths.
If striped groups are configured for your Sun StorageTek QFS file system, the minimum amount of space allocated is the DAU multiplied by the number of devices in the striped group. The amount of the allocation can be very large with striped groups.
When striped groups are used, data is written to several disk devices at once, as if they were one device. Allocations on striped groups are equal to the DAU size multiplied by the number of elements in the striped group.
The -o stripe=n mount option determines the number of allocations that occur on each stripe group before the allocation moves to a different striped group. If a file system is mounted with -o stripe=0, the allocation is always to one striped group.
By default, the setting is -o stripe=0, which specifies the round-robin allocation method. The setting can be as low as -o stripe=0 (which disables striping) or as high as -o stripe=255. The system sets -o stripe=0 if mismatched striped groups are present, in which case a file can reside on only one striped group.
For more information on allocation methods, see File Allocation Methods.
You can use the -o mm_stripe=n option to the mount_samfs(1M) command to stripe metadata information on the metadata disk. The default stripe width is
-o mm_stripe=1, which specifies that the file system write one 16-kilobyte DAU to a metadata disk before switching to the next metadata disk. The small 4-kilobyte DAU is used for metadata disks.
By default, if you have multiple metadata devices, metadata is allocated as specified in the -o mm_stripe=n option to the mount(1M) command. The setting can be as low as -o mm_stripe=0, which disables striping, or as high as
-o mm_stripe=255.
The Sun StorageTek QFS software enables you to specify both round-robin and striped allocation methods. TABLE 1-6 shows the default file allocation methods used.
The rest of this section describes allocation in more detail.
Metadata allocation varies according to the type of file system you have:
Inodes are 512 bytes in length. Directories are initially 4 kilobytes in length. TABLE 1-7 shows how the system allocates metadata.
The round-robin allocation method writes one data file at a time to each successive device in the family set. Round-robin allocation is useful for multiple data streams, because in this type of environment aggregate performance can exceed striping performance.
Round-robin disk allocation enables a single file to be written to a logical disk. The next file is written to the next logical disk, and so on. When the number of files written equals the number of devices defined in the family set, the file system starts over again with the first device selected. If a file exceeds the size of the physical device, the first portion of the file is written to the first device, and the remainder of the file is written to the next device with available storage. The size of the file being written determines the I/O size.
You can specify round-robin allocation explicitly in the /etc/vfstab file by entering stripe=0.
The following figures depict round-robin allocations in ms and ma file systems. In these figures, file 1 is written to disk 1, file 2 is written to disk 2, file 3 is written to disk 3, and so on. When file 6 is created, it is written to disk 1, restarting the round-robin allocation scheme.
By default, Sun StorageTek QFS file systems use a striped allocation method to spread data over all the devices in the file system family set. Striping is a method of concurrently writing files in an interlaced fashion across multiple devices.
Striping is used when performance for one file requires the additive performance of all the devices. A file system that is using striped devices addresses blocks in an interlaced fashion rather than sequentially. Striping generally increases performance because it enables multiple I/O streams to simultaneously write a file across multiple disks. The DAU and the stripe width determine the size of the I/O transmission.
In a file system using striping, file 1 is written to disk 1, disk 2, disk 3, disk 4, and disk 5. File 2 is written to disks 1 through 5 as well. The DAU multiplied by the stripe width determines the amount of data written to each disk in a block.
When a Sun StorageTek QFS file system writes a file to an md device, it starts by trying to fit the file into a small DAU, which is 4 kilobytes. If the file does not fit into the first eight small DAUs (32 kilobytes) allocated, the file system writes the remainder of the file into one or more large DAUs.
When a Sun StorageTek QFS file system writes a file to an mr device, it writes first to one DAU, then to another, and so on. The mr devices have only one DAU size.
Multiple active files cause significantly more disk head movement with striped than with round-robin allocation. If I/O is to occur to multiple files simultaneously, use round-robin allocation.
The following figures depict ms and ma file systems using striped allocations. In these figures, DAU x stripe-width bytes of the file are written to disk 1. DAU x stripe-width bytes of the file are written to disk 2 and so on. The order of the stripe is first-in-first-out for the files. Striping spreads the I/O load over all the disks.
A striped group is a Sun StorageTek QFS allocation method designed for file systems that have extremely large I/O requirements and terabytes of disk cache. A striped group enables you to designate an Equipment Type value that accounts for multiple physical disks. Multiple striped group Equipment Type entries can make up a single Sun StorageTek QFS file system. Striped groups save bitmap space and system update time for very large RAID configurations.
A striped group is a collection of devices within a Sun StorageTek QFS file system. Defined in the mcf file as gXXX devices, striped groups enable one file to be written to and read from two or more devices. You can specify up to 128 striped groups within a file system.
FIGURE 1-5 depicts a Sun StorageTek QFS ma file system using striped groups and a round-robin allocation. In FIGURE 1-5, files written to the qfs1 file system are allocated round-robin among the defined striped groups g0, g1, and g2. Each group consists of two physical RAID devices.
For the configuration in FIGURE 1-5, the mount point option in /etc/vfstab is set to stripe=0. CODE EXAMPLE 1-1 shows the mcf file that declares these striped groups.
FIGURE 1-6 depicts a Sun StorageTek QFS ma file system using striped groups and striped allocation. Files written to the qfs1 file system are striped through groups g0, g1, and g2. Each group includes four physical RAID devices. The mount point option in /etc/vfstab is set to stripe=1 or greater.
It is possible to build a file system with mismatched striped groups, which are multiple striped groups with different numbers of devices in each group. Sun StorageTek QFS file systems support mismatched striped groups, but they do not support striping on mismatched groups. File systems with mismatched striped groups are mounted as round-robin file systems.
The following example shows how a file system can be set up with mismatched striped groups to store different types of files.
Suppose that you need to create a file system at your site that contains both video and audio data.
Video files are quite large and require greater performance than audio files. You want to store them in a file system with a large striped group, because striped groups maximize performance for very large files.
Audio files are smaller and require lower performance than video files. You want to store them in a small striped group. One file system can support both video and audio files.
FIGURE 1-7 depicts the file system needed. It is an ma file system using mismatched striped groups in a striped allocation.
TABLE 1-8 shows the characteristics of this sample file system.
Add the following line to the /etc/vfstab file so that the environment recognizes the avfs file system:
Note that in the /etc/vfstab file, stripe=0 is used to specify a round-robin file system. This is used because a value greater than 0 is not supported for mismatched striped groups.
CODE EXAMPLE 1-2 shows the mcf file for file system avfs.
When the mcf file for this file system is ready, you can enter the sammkfs(1M) and mount(1M) commands shown in CODE EXAMPLE 1-3 to create and mount the avfs file system.
After the file system is mounted, you can use the commands shown in CODE EXAMPLE 1-4 to create two directories for the two types of files.
# cd /avfs # mkdir video # mkdir audio |
Once the directories are created, you can use the setfa(1) commands shown in CODE EXAMPLE 1-5 to assign the large striped group to video and the small striped group to audio. Files created in these directories are allocated on their respective striped groups because attributes are inherited.
For more information about the sammkfs(1M) command, see the sammkfs(1M) man page. For more information about the mount(1M) commands, see the mount_samfs(1M) man page. For more information about the setfa(1) command, see the setfa(1) man page.
If necessary, you can disable allocation to a specific Sun StorageTek QFS data partition by using a nalloc command, which prohibits any future allocation to that device. The feature is currently only usable on data partitions, not on metadata partitions.
Allocation to a partition can be restarted by either an alloc or on command.
The allocation state of a partition (allocflag) is persistent across boots.
The nalloc and alloc commands are available in the samu interface, and the samu on command also sets allocation to on. The samu screens display the nalloc state for partitions that have been disabled. The samtrace and samfsinfo output also include the allocation state.
For more information about the samu interface, see Using the samu(1M) Operator Utility.
| Sun StorageTek QFS File System Configuration and Administration Guide | 819-7935-10 |
|
Copyright © 2007, Sun Microsystems, Inc. All Rights Reserved.
