Sun StorageTek QFS Installation and Upgrade Guide Table Of ContentsPrevious ChapterNext ChapterBook Index


C H A P T E R  4
Configuring a Shared or Sun Cluster Configuration

This chapter contains instructions for configuring the Sun StorageTek QFS software in a shared or Sun Cluster environment. Before carrying out the configuration procedures in this chapter, you must have installed the software as described in Chapter 3.

This chapter contains the following sections:


Note - Beginning with version 4U6 of the Sun StorageTek QFS software, you can also have shared clients outside of the cluster in a Sun Cluster environment. For complete configuration instructions, see the Sun StorageTek QFS File System Configuration and Administration Guide.


Preparing the Host Systems

Perform this procedure to prepare the host systems for a Sun StorageTek QFS shared file system or for a Sun StorageTek QFS shared file system in a Sun Cluster environment.


procedure icon  To Prepare the Host Systems

1. Verify that all the hosts have the same user and group IDs.

If you are not running the Network Information Name service (NIS), make sure that all /etc/passwd and all /etc/group files are identical. If you are running NIS, the /etc/passwd and /etc/group files should already be identical.

For more information about this, see the nis+(1) man page.

2. If you are configuring a Sun StorageTek QFS shared file system on the Solaris OS, enable the network time daemon command, xntpd(1M), to synchronize the times on all the hosts.

You do not need to perform this step if you are configuring a Sun StorageTek QFS shared file system in a Sun Cluster environment because it has already been done as part of the Sun Cluster installation.

The clocks of all hosts must be synchronized, and must be kept synchronized, during Sun StorageTek QFS shared file system operations. For more information, see the xntpd(1M) man page.

The following steps enable the xntpd(1M) daemon on one host. Follow these steps for each host.

a. Stop the xntpd(1M) daemon.

For example: 


# /etc/init.d/xntpd stop

b. Use vi(1) or another editor to create the file /etc/inet/ntp.conf.

c. Create a line in the file /etc/inet/ntp.conf that specifies the name of the local time server.

This line has the following format: 


server IP-address prefer

In the preceding command, server and prefer are required keywords. Specify the IP address of your local time server for IP-address.

If you have no local time server, see one of the following URLs for information on how to access a public time source:

http://www.eecis.udel.edu/~mills/ntp/servers.html
http://www.boulder.nist.gov/timefreq/general/pdf/1383.pdf

d. Close the file /etc/inet/ntp.conf.

e. Start the xntpd(1M) daemon. 


# /etc/init.d/xntpd start

Editing mcf Files on Other Hosts

Perform the tasks described in this section if you are configuring one of the following types of file systems:

The lines that define a particular file system must be identical in the mcf files on all host systems that support the file system. Only one mcf file can reside on a host. Because you can have other, additional Sun StorageTek QFS file systems defined in an mcf file, the mcf files on different hosts might not be identical.


Note - If you update a metadata server's mcf file at any time after the Sun StorageTek QFS shared file system is mounted, make sure that you update the mcf files as necessary on all hosts that can access that shared file system.



procedure icon  To Edit mcf Files for a Highly Available File System in a Sun Cluster Environment

Perform this procedure for a Sun StorEdge QFS highly available file system in a Sun Cluster environment, on each host that you want to have support the file system you are configuring.

1. Log in to the Sun Cluster node.

2. Become superuser.

3. Use vi(1) or another editor to create an mcf file on that node.

If an mcf file already exists on the host, add the lines for the new file system to this mcf file.

4. Copy the lines that define the file system from the primary node's mcf file to this node's mcf file.


procedure icon  To Edit mcf Files for a Sun StorageTek QFS Shared File System

Perform this procedure for each host that you want to include in a shared file system in a Solaris or Sun Cluster environment.

1. Log in to the host.

2. Become superuser.

3. Use the format(1M) command to verify the presence of client host disks.

4. Use vi(1) or another editor to create an mcf file.

If an mcf file already exists on the host, add the lines for the new file system to this mcf file.

5. Issue the samfsconfig(1M) command.

Examine this command's output to locate the local device names for each additional host to be configured in the Sun StorageTek QFS shared file system.

The samfsconfig(1M) command generates configuration information that can help you to identify the devices included in the Sun StorageTek QFS shared file system. Enter a separate samfsconfig(1M) command on each client host. Note that the controller number might not be the same controller number as on the metadata server because the controller numbers are assigned by each client host.

6. Update the mcf file on other client hosts.

To access or mount a shared file system, a host system must have that file system defined in its mcf file. The content of mcf files varies, depending on whether the Solaris OS or Sun Cluster environment hosts the file system, as follows:

Use vi(1) or another editor to edit the mcf file on one of the client host systems. The mcf file must be updated on all client hosts to be included in the Sun StorageTek QFS shared file system. The file system and disk declaration information must have the same data for the Family Set name, Equipment Ordinal, and Equipment Type fields as the configuration on the metadata server. The mcf files on the client hosts must also include the shared keyword. The device names, however, can change as controller assignments can change from host to host.

Examples

Example 1 - Solaris OS hosts. CODE EXAMPLE 4-1 shows how the samfsconfig(1M) command is used to retrieve device information for family set sharefs1 on client tethys. Because tethys is a potential metadata server, it is connected to the same metadata disks as titan.


CODE EXAMPLE 4-1 samfsconfig (1M) Command Example on tethys 
tethys# samfsconfig /dev/dsk/*

#

# Family Set `sharefs1' Created Wed Jun 27 19:33:50 2003

#

sharefs1                         10 ma sharefs1 on shared

/dev/dsk/c2t50020F23000065EEd0s6 11 mm sharefs1 on

/dev/dsk/c7t50020F2300005D22d0s6 12 mr sharefs1 on

/dev/dsk/c7t50020F2300006099d0s6 13 mr sharefs1 on

/dev/dsk/c7t50020F230000651Cd0s6 14 mr sharefs1 on

Edit the mcf file on client host tethys by copying the last five lines of output from the samfsconfig(1M) command into the mcf file. Verify the following:

CODE EXAMPLE 4-2 shows the resulting mcf file.


CODE EXAMPLE 4-2 mcf File for sharefs1 Client Host tethys 
# Equipment                      Eq  Eq   Family   Dev   Add

# Identifier                     Ord Type Set      State Params

# ----------                     --- ---- ------   ----- ------

sharefs1                         10  ma   sharefs1 on    shared

/dev/dsk/c2t50020F23000065EEd0s6 11  mm   sharefs1 on

/dev/dsk/c7t50020F2300005D22d0s6 12  mr   sharefs1 on

/dev/dsk/c7t50020F2300006099d0s6 13  mr   sharefs1 on

/dev/dsk/c7t50020F230000651Cd0s6 14  mr   sharefs1 on

In CODE EXAMPLE 4-2, the Equipment Ordinal numbers match those of the mcf file for metadata server titan (see Configuration Example for a Shared File System on a Solaris OS Platform). These Equipment Ordinal numbers must not already be in use on client host tethys or any other client host.

Example 2 - Solaris OS hosts. CODE EXAMPLE 4-3 shows how the samfsconfig(1M) command is used to retrieve device information for family set sharefs1 on client host mimas. Because mimas can never become a metadata server, it is not connected to the metadata disks.


CODE EXAMPLE 4-3 samfsconfig (1M) Command Example on mimas 
mimas# samfsconfig /dev/dsk/*

#

# Family Set `sharefs1' Created Wed Jun 27 19:33:50 2001

#

# Missing slices

# Ordinal 0

# /dev/dsk/c1t50020F2300005D22d0s6   12    mr   sharefs1   on

# /dev/dsk/c1t50020F2300006099d0s6   13    mr   sharefs1   on

# /dev/dsk/c1t50020F230000651Cd0s6   14    mr   sharefs1   on

In the command output, note that Ordinal 0, which is the metadata disk, is not present. Because devices are missing, the samfsconfig(1M) command comments out the elements of the file system and omits the file system Family Set declaration line. Make the following types of edits to the mcf file:

CODE EXAMPLE 4-4 shows the resulting mcf file.


CODE EXAMPLE 4-4 mcf File for Client Host mimas 
# The mcf File For mimas

# Equipment                      Eq  Eq   Family   Device Addl

# Identifier                     Ord Type Set      State  Params

------------                     --- ---- ---      -----  ------

sharefs1                         10  ma   sharefs1 on     shared

nodev                            11  mm   sharefs1 on

/dev/dsk/c1t50020F2300005D22d0s6 12  mr   sharefs1 on

/dev/dsk/c1t50020F2300006099d0s6 13  mr   sharefs1 on

/dev/dsk/c1t50020F230000651Cd0s6 14  mr   sharefs1 on

Creating the Shared Hosts File

Perform the tasks described in this section if you are configuring one of the following types of file systems:


Note - Alternatively, you can set up a shared hosts file using File System Manager. For instructions, see the "Planning a Shared File System" topic in the File System Manager online help.



procedure icon  To Create the Shared Hosts File on the Metadata Server

When you create a shared file system, the system copies information from the hosts file to the shared hosts file on the metadata server. You update this information when you issue the samsharefs(1M) -u command.

1. Use the cd(1) command to change to directory /etc/opt/SUNWsamfs.

2. Use vi(1) or another editor to create an ASCII hosts file called hosts.fs-name.

For fs-name, specify the Family Set name of the Sun StorageTek QFS shared file system.

Comments are permitted in the hosts file. Comment lines must begin with a pound character (#). Characters to the right of the pound character are ignored.

3. Use the information in TABLE 4-1 to fill in the lines of the hosts file.

File hosts.fs-name contains configuration information pertaining to all hosts in the Sun StorageTek QFS shared file system. The ASCII hosts file defines the hosts that can share the Family Set name.

TABLE 4-1 shows the fields in the hosts file.


TABLE 4-1 Hosts File Fields

Field Number

Content

1

The Host Name field. This field must contain an alphanumeric host name. It defines the Sun StorageTek QFS shared file system hosts. You can use the output from the hostname(1) command to create this field.

2

The Host IP Addresses field. This field must contain a comma-separated list of host IP addresses. You can use the output from the ifconfig(1M) -a command to create this field. You can specify the individual addresses in one of the following ways:

  • Dotted-decimal IP address form
  • IP version 6 hexadecimal address form
  • A symbolic name that the local domain name service (DNS) can resolve to a particular host interface

The metadata server uses this field to determine whether a host is allowed to connect to the Sun StorageTek QFS shared file system. If the metadata server receives a connection attempt from any interface not listed in this field, it rejects the connection attempt. Conversely, use care when adding elements here, because the metadata server accepts any host with an IP address that matches an address in this field.

Client hosts use this field to determine the metadata server interfaces to use when attempting to connect to the metadata server. A host evaluates the addresses from left to right and attempts a connection using the first responding address in the list.

Note: You should avoid using a domain name in this field, because during the reboot process, when sam-fsd is trying to contact the metadata server, naming services are likely not up. This means that a domain name may not be resolvable if it is not in the /etc/inet/ipnodes or /etc/inet/hosts file; this will cause the mount to fail and could cause the reboot to hang.

To avoid this problem, for file systems that are mounted at boot, you should add the file system's hosts to the /etc/inet/hosts or /etc/inet/ipnodes files. On clients, the names of the servers should be added; on servers, all of the file system's hosts should be added.

3

The Server field. This field must contain either a dash character (-) or an integer ranging from 0 through n. The - and the 0 are equivalent.

If the value of the Server field is a nonzero integer, the host is a potential metadata server. The rest of the row defines the server as a metadata host. The metadata server processes all the metadata modification for the file system. At any one time there is at most one metadata server host, and that metadata server supports archiving, staging, releasing, and recycling for a Sun StorageTek QFS shared file system.

If the Server field is - or 0, the host is not eligible to be a metadata server.

4

Reserved for future use by Sun Microsystems. This field must contain either a dash character (-) or a 0. The - and the 0 are equivalent.

5

The Server Host field. This field can contain either a blank or the server keyword in the row that defines the active metadata server. Only one row in the hosts file can contain the server keyword. This field must be blank in all other rows.


The system reads and manipulates the hosts file. You can use the samsharefs(1M) command to examine metadata server and client host information about a running system.

Example for Solaris OS Hosts

CODE EXAMPLE 4-5 is an example hosts file that shows four hosts.


CODE EXAMPLE 4-5 Sun StorageTek QFS Shared File System Hosts File Example 
# File /etc/opt/SUNWsamfs/hosts.sharefs1

# Host   Host IP                           Server   Not  Server

# Name   Addresses                         Priority Used Host

# ----   --------------------------------- -------- ---- -----

titan    172.16.0.129 	1        -    server

tethys   172.16.0.130 	2        -

mimas    mimas - 	-

dione    dione - 	-

This hosts file contains fields of information and comment lines for the sharefs1 file system. In this example, the number 1 in the Server Priority field defines titan as the primary metadata server. If titan is unavailable, the next metadata server is tethys, as indicated by the number 2 in this field. Note that neither mimas nor dione can ever be a metadata server.

Example for Sun Cluster Hosts

If you are configuring a Sun StorageTek QFS shared file system in a Sun Cluster environment, every host is a potential metadata server. The hosts files and the local hosts configuration files must contain node names in the Host Names field and Sun Cluster private interconnect names in the Host IP Addresses field.

CODE EXAMPLE 4-6 shows the local hosts configuration file for shared file system sharefs1. This file system's participating hosts are Sun Cluster nodes scnode-A and scnode-B. Each node's private interconnect name is listed in the Host IP Addresses field.


CODE EXAMPLE 4-6 Sun StorageTek QFS Shared File System Hosts File Example 
# File /etc/opt/SUNWsamfs/hosts.sharefs1

# Host   Host IP                           Server   Not  Server

# Name   Addresses                         Priority Used Host

# ----   --------------------------------- -------- ---- -----

scnode-A clusternode1-priv                 1        -    server

scnode-B clusternode2-priv                 2        -


procedure icon  To Create the Local Hosts File on a Client

Perform this procedure under the following circumstances:


Note - You can also use File System Manager to create or modify the hosts.fsname.local files within a Sun StorageTek QFS shared file system.


Follow these steps for each client host that you want to include in the Sun StorageTek QFS shared file system.

1. Create the local hosts configuration file on the client host.

Using vi(1) or another editor, create an ASCII local hosts configuration file that defines the host interfaces that the metadata server and the client hosts can use when accessing the file system. The local hosts configuration file must reside in the following location:

/etc/opt/SUNWsamfs/hosts.fsname.local

For fsname, specify the Family Set name of the Sun StorageTek QFS shared file system.

Comments are permitted in the local host configuration file. Comment lines must begin with a pound character (#). Characters to the right of the pound character are ignored. TABLE 4-2 shows the fields in the local hosts configuration file.


TABLE 4-2 Local Hosts Configuration File Fields

Field Number

Content

1

The Host Name field. This field must contain the alphanumeric name of a metadata server or potential metadata server that is part of the Sun StorageTek QFS shared file system.

2

The Host Interfaces field. This field must contain a comma-separated list of host interface addresses. You can use the output from the ifconfig(1M) -a command to create this field. You can specify the individual interfaces in one of the following ways:

  • Dotted-decimal IP address form
  • IP version 6 hexadecimal address form
  • A symbolic name that the local domain name service (DNS) can resolve to a particular host interface

The client hosts use this field to determine the metadata server interfaces to use when attempting to connect to the metadata server. The system evaluates the addresses from left to right and attempts a connection using the first responding address in the list that is also included in the shared hosts file.


How Metadata Server Addresses Are Obtained

The information in this section might be useful when you are debugging.

In a Sun StorageTek QFS shared file system, each client host obtains the list of metadata server IP addresses from the shared hosts file.

The metadata server and the client hosts use the shared hosts file on the metadata server and the hosts.fsname.local file on each client host (if it exists) to determine the host interface to use when accessing the metadata server. This process is as follows:


Note - The term client, as in network client, is used to refer to both client hosts and the metadata server host.


1. The client obtains the list of metadata server host IP interfaces from the file system's on-disk shared hosts file. To examine this file, issue the samsharefs(1M) command from the metadata server or from a potential metadata server.

2. The client searches for an /etc/opt/SUNWsamfs/hosts.fsname.local file. Depending on the outcome of the search, one of the following occurs:

i. It compares the list of addresses for the metadata server from both the shared hosts file on the file system and the hosts.fsname.local file.

ii. It builds a list of addresses that are present in both places, and then it attempts to connect to each of these addresses, in turn, until it succeeds in connecting to the server. If the order of the addresses differs in these files, the client uses the ordering in the hosts.fsname.local file.

Example

This example expands on FIGURE D-1 in Appendix D. CODE EXAMPLE 4-5 shows the hosts file for this configuration. FIGURE 4-1 shows the interfaces to these systems.


FIGURE 4-1 Network Interfaces

Figure of a shared Sun SAM-QFS environment showing public and private networks.Shows hosts titan, tethys, dione, and mimas connected to a public network. Shows hosts titan and tethys connected by a private network.


Systems titan and tethys share a private network connection with interfaces 172.16.0.129 and 172.16.0.130. To guarantee that titan and tethys always communicate over their private network connection, the system administrator has created identical copies of /etc/opt/SUNWsamfs/hosts.sharefs1.local on each system. CODE EXAMPLE 4-7 shows the information in these files.


CODE EXAMPLE 4-7 File hosts.sharefs1.local on Both titan and tethys 
# This is file /etc/opt/SUNWsamfs/hosts.sharefs1.local

# Host Name    Host Interfaces

# ---------    ---------------

titan          172.16.0.129

tethys         172.16.0.130

Systems mimas and dione are not on the private network. To guarantee that they connect to titan and tethys through titan's and tethys's public interfaces, and never attempt to connect to titan's or tethys's unreachable private interfaces, the system administrator has created identical copies of /etc/opt/SUNWsamfs/hosts.sharefs1.local on mimas and dione. CODE EXAMPLE 4-8 shows the information in these files.


CODE EXAMPLE 4-8 File hosts.sharefs1.local on Both mimas and dione 
# This is file /etc/opt/SUNWsamfs/hosts.sharefs1.local

# Host Name    Host Interfaces

# ----------   --------------

titan          titan.xyzco.com

tethys         tethys.xyzco.com

Verifying That the Daemons Are Running

Perform the tasks described in this section if you are configuring the following types of file systems:


procedure icon  To Verify the Daemons

Perform these steps on each host that can mount the file system.

1. Verify that the file system is mounted.

If it is not mounted, go back to Mounting the File System and follow the instructions there.

2. Use the ps(1) and grep(1) commands to determine whether the sam-sharefsd daemon is running for this file system.

For example: 


# ps -ef | grep sam-sharefsd

root 26167 26158  0 18:35:20 ?        0:00 sam-sharefsd sharefs1

root 27808 27018  0 10:48:46 pts/21   0:00 grep sam-sharefsd

This example shows that the sam-sharefsd daemon is active for the sharefs1 file system.


Note - If the sam-sharefsd daemon is active for your Sun StorageTek QFS shared file system, you need to perform some diagnostic procedures. For information about these procedures, see the Sun StorageTek QFS File System Configuration and Administration Guide.


3. If the output from this command indicates that the sam-sharefsd daemon is not running, determine whether the sam-fsd daemon is running as follows:

a. Use the ps(1) and grep(1) commands to verify that the sam-fsd daemon is running for this file system.

b. Examine the output.

CODE EXAMPLE 4-9 shows sam-fsd output that indicates that the daemon is running.


CODE EXAMPLE 4-9 sam-fsd (1M) Output That Shows the sam-fsd Daemon is Running 
cur% ps -ef | grep sam-fsd      

user1 16435 16314  0 16:52:36 pts/13   0:00 grep sam-fsd

root   679     1  0   Aug 24 ?        0:00 /usr/lib/fs/samfs/sam-fsd

4. Do one of the following:

Configuring the SUNW.qfs Resource Type

Perform the task described in this section if you are configuring a Sun StorageTek QFS shared file system on a Sun Cluster platform.


procedure icon  To Enable a Sun StorageTek QFS Shared File System as a SUNW.qfs(5) Resource

1. Log in to the metadata server as superuser.

2. Use the scrgadm(1M) -p command and search for the SUNW.qfs(5) resource type.

For example: 


metadataserver# scrgadm -p | grep SUNW.qfs

3. If the SUNW.qfs resource type is missing, issue the following command: 


metadataserver# scrgadm -a -t SUNW.qfs

4. Use the scrgadm(1M) command to set the FilesystemCheckCommand property of the SUNW.qfs(5) resource type to /bin/true.

The SUNW.qfs(5) resource type is part of the Sun StorageTek QFS software package. Configuring the resource type for use with your shared file system makes the shared file system's metadata server highly available. Sun Cluster scalable applications can then access data contained in the file system. For more information, see the Sun StorageTek QFS File System Configuration and Administration Guide.

CODE EXAMPLE 4-10 shows how to use the scrgadm(1M) command to register and configure the SUNW.qfs resource type. In this example, the nodes are scnode-A and scnode-B. /global/sharefs1 is the mount point as specified in the /etc/vfstab file.


CODE EXAMPLE 4-10 Configuring a SUNW.qfs Resource 
# scrgadm -a -g qfs-rg -h scnode-A,scnode-B

# scrgadm -a -g qfs-rg -t SUNW.qfs -j qfs-res \

	   -x QFSFileSystem=/global/sharefs1


Note - In a SAM-QFS environment, you can also configure the Sun StorageTek SAM software for high availability using Sun Cluster software. For instructions, see the Sun StorageTek Storage Archive Manager Archive Configuration and Administration Guide.


Configuring the HA Storage Plus Resource

Perform the task in this section if you are configuring a Sun StorEdge QFS highly available file system on a Sun Cluster platform.


procedure icon  To Configure a Highly Available File System as an HA Storage Plus Resource

single-step bulletUse the scrgadm(1M) command to set the FilesystemCheckCommand property of HA Storage Plus to /bin/true.

All other resource properties for HA Storage Plus apply as specified in SUNW.HAStoragePlus(5).

The following example command shows how to use the scrgadm(1M) command to configure an HA Storage Plus resource: 


# scrgadm -a -g qfs-rg -j ha-qfs -t SUNW.HAStoragePlus \

        -x FilesystemMountPoints=/global/qfs1 \

        -x FilesystemCheckCommand=/bin/true

Bringing the Shared Resource Online

Perform the task described in this section if you are configuring the following types of file systems:


procedure icon  To Bring the Shared Resource Online

1. Verify that the file system is mounted on all nodes.

If it is not mounted, go back to Mounting the File System and follow the instructions there.


Note - If you are using the SUNW.qfs resource type, you cannot use the bg mount option in the /etc/vfstab file.


2. Log in to the appropriate host.

3. Use the scswitch(1M) command to move the file system resource to another node.

For example: 


metadataserver# scswitch -Z -g qfs-rg

4. Use the scstat(1M) command to verify that the file system resource was moved successfully.

For example: 


metadataserver# scstat

< information deleted from this output >

-- Resources --

Resource Name    Node Name  State     Status Message

-------------    ---------  -----     --------------

Resource: qfs-res   ash     Online    Online

Resource: qfs-res   elm     Offline   Offline

Resource: qfs-res   oak     Offline   Offline

Verifying the Resource Group on All Nodes

Perform the task described in this section if you are configuring the following types of file systems:


procedure icon  To Verify the Resource Group on All Nodes

Perform these steps for each node in the cluster, with a final return to the original server.

1. From any node in the Sun Cluster environment, use the scswitch(1M) command to move the file system resource from one node to another.

For example: 


server# scswitch -z -g qfs-rg -h elm

2. Use the scstat(1M) command to verify that the file system resource was moved successfully.

For example: 


server# scstat

-- Resources --

Resource Name    Node Name  State     Status Message

-------------    ---------  -----     --------------

Resource: qfs-res   ash     Offline   Offline

Resource: qfs-res   elm     Online    Online

Resource: qfs-res   oak     Offline   Offline

 



  Sun StorageTek QFS Installation and Upgrade Guide 819-7936-10 Table Of ContentsPrevious ChapterNext ChapterBook Index


Copyright © 2007, Sun Microsystems, Inc. All Rights Reserved.