CVADMIN
Section: Maintenance Commands (8) Updated: August 2015
NAME
cvadmin – Administer a StorNext File System
SYNOPSIS
cvadmin [-H FSMHostName] [-F FileSystemName] [-M] [-f filename] [-e command1 -e command2…] [-x]
DESCRIPTION
cvadmin is an interactive command used for general purposeadministration of a StorNext File System including:
1. displaying file system and client status
2. activating a file system currently in stand-by mode
3. viewing and modifying stripe group attributes
4. enabling File System Manager (FSM) tracing
5. displaying disk and path information for the local system
6. forcing FSM failover
7. fetching FSM usage and performance statistics
8. temporarily enabling or disabling global file locking
9. generating a report of open files
10. listing currently held file locks
11. starting, restarting and stopping of daemon processes
12. resetting RPL information
OPTIONS
Invoke
cvadmin to start the interactive session and list the running File System Managers (FSMs). (Note: StorNext system services must be started prior to running
cvadmin. In particular, the local
fsmpm(8) process must be active.)
Then (optionally) use the select command described below to pick an FSM to connect to. Once connected, the command will display basic information about the selected file system and prompt for further commands.
Note that a few commands such as
paths,
disks,
start, and
stop require information obtained from the local
fsmpm(8) only, so there is is no need to select an FSM prior to using them.
USAGE
-H FSMHostName
Connect to the FSM located on the machine FSMHostName. By default cvadmin will attempt to connect to an FSM located on the local machine.
-F FileSystemName
Automatically set the file system FileSystemName as the active file system in cvadmin.
-M
When listing file systems with the select command, display [managed] next to each file system with DataMigration enabled. This option is currently only intended for use by support personnel.
-f filename
Read commands from filename
-e command
Execute command(s) and exit
-x
Enable extended commands.
COMMANDS
The
cvadmin commands can be used to display and modify the SNFS active configuration. When a modification is made, it exists only as long as the
FSM is running. More permanent changes can be made in the configuration file. Refer to the
snfs_config(5) man page for details. The following commands are supported.
activate file_system_name [hostname_or_IP_address]
Activate a file system file_system_name. This command may cause an FSM to activate. If the FSM is already active, no action is taken.
activate file_system_name number_of_votes
Quantum Internal only. Bypass the election system and attempt to activate the fsm on this node.
debug [[+|–] flag [ … ]]
View or set the File System Manager’s debugging flags. Entering the command with no flag will return current settings, the location of the FSM log file and a legend describing what each setting does. By entering the command with a flag list, the FSM Debugging Flags will be set accordingly. Each flag can be either a name or numeric value. Names will be mapped to their numeric value, and may be abbreviated as long as they remain unique. Numeric values are specified using a standard decimal or hexadecimal (0x) value of up to 32 bits. Using ‘+‘ or ‘–‘ enables (‘+‘) or disables (‘–‘) only the selected flags, leaving all other flags unchanged.NOTE – Setting Debugging Flags will severely impact the FSM‘s performance! Do this only when directed by an Quantum specialist.
disks [refresh]
Display the StorNext disk volumes local to the system that cvadmin is attached to. Using the optional refresh argument will force the fsmpm to re-scan all volumes before responding. If the fsmpm’s view of the disks in any file system changes compared with the FSM’s view of that client’s disks as a result of the refresh, a disconnect and reconnect to the FSM will take place to resynchronise the file system state.
disks [refresh] fsm
Display the StorNext meta-data disk volumes in use by the fsm. If the optional refresh argument is used, additional paths to these volumes may be added by the fsm.
down groupname
Down the stripe group groupname. This will down any access to the stripe group.
fail [file_system_name|index_number]
Initiate an FSM Failover of file system file_system_name. This command may cause a stand-by FSM to activate. If an FSM is already active, the FSM will shut down. A stand-by FSM will then take over. If a stand-by FSM is not available the primary FSM will re-activate after failover processing is complete.
files
Report counts of files, directories, symlinks and other objects which are anchored by a user type inode. These include named streams, block and character device files, fifos or pipes and named sockets. If the file system is undergoing conversion to StorNext 5.0, conversion progress is displayed and counters reflect the count of converted objects.
fsmlist [file_system_name] [ on hostname_or_IP_address]
Display the state of FSM processes, running or not. Optionally specify a single file_system_name to display. Optionally specify the host name or IP address of the system on which to list the FSM processes.
filelocks
Query cluster-wide file/record lock enforcement. Currently cluster-wide file locks are automatically used on Unix. Windows file/record locks are optional.If enabled, byte-range file locks are coordinated through the FSM, allowing a lock set by one client to block overlapping locks by other clients. If disabled, then byte-range locks are local to a client and do not prevent other clients from getting byte-range locks on a file, however they do prevent overlapping lock attempts on the same client.
help (?)
The help or ? command will display a command usage summary.
latency-test [index_number|all] [seconds]
Run an I/O latency test between the FSM process and one client or all clients. The default test duration is 2 seconds.
metadata
Report metadata usage. Also provide an estimate on the value of bufferCacheSize that will allow all metadata to be cached.
metadump { status | rebuild | suspend | resume }
Manage the metadump functionality of the selected FSM.The status command prints the progress of the current metadump activity, if any. If capturing a new metadump or restoring an existing one, the percentage complete will be displayed. Otherwise, the current update backlog is displayed.
The rebuild command will force the FSM to discard the existing metadump and capture a new one. This is performed online.
The suspend and resume commands are used internally to facilitate backups of the metadump files. They should not be invoked manually except under direction from support.
multipath groupname {balance|cycle|rotate|static|sticky}
StorNext has the capability of utilizing multiple paths from a system to the SAN disks.This capability is referred to as “multi-pathing”, or sometimes “multi-HBA support”. (HBA := Host Based Adaptor).
At “disk discovery” time, for each physical path (HBA), a scan of all of the SAN disks visible to that path is initiated, accumulating information such as the SNFS label, and where possible, the disk (or LUN) serial number.
At mount time, the visible set of StorNext labeled disks is matched against the requested disks for the file system to be mounted.
If the requested disk label appears more than once, then a “multi-path” table entry is built for each available path.
If the disk (or LUN) device is capable of returning a serial number, then that serial number is used to further verify that all of the paths to that StorNext labeled device share the same serial number.
If the disk (or LUN) device is not capable of returning a serial number then the device will be used, but StorNext will not be able to discern the difference between a multi-path accessible device, and two or more unique devices that have been assigned duplicate StorNext labels.
The presence of serial numbers can be validated by using the “cvlabel -ls” command. The “-s” option requests the displaying of the serial number along with the normal label information.
There are five modes of multi-path usage which can also be specified in the filesystem config file. In cases where there are multiple paths and an error has been detected, the algorithm falls back to the rotate method. The balance and cycle methods will provide the best aggregate throughput for a cluster of hosts sharing storage.
balance
The balance mode provides load balancing across all the available, active, paths to a device. At I/O submission time, the least used HBA/controller port combination is used as the preferred path. All StorNext File System I/O in progress at the time is taken into account.
cycle
The cycle mode rotates I/O to a LUN across all the available, active, paths to it. As each new I/O is submitted, the next path is selected.
rotate
The
rotate mode is the default for configurations where the operating system presents multiple paths to a
device.In this mode, as an I/O is initiated, an HBA controller pair to use for this I/O is selected based on a load balance method calculation.
If an I/O terminates in error, a “time penalty” is assessed against that path, and another “Active” path is used. If there are not any “Active” paths that are not already in the “error penalty” state, then a search for an available “Passive” path will occur, possibly triggering an Automatic Volume Transfer to occur in the Raid Controller.
static
The “default” mode for all disks other than Dual Raid controller configurations that are operating in Active/Active mode with AVT enabled.
As disks (or LUNs) are recognized at mount time, they are statically associated with an HBA in rotation.i.e. given 2 HBA’s, and for disks/LUNs:
disk 0 -> HBA 0
disk 1 -> HBA 1
disk 2 -> HBA 0
disk 3 -> HBA 1
and so on...
sticky
In this mode, the path to use for an I/O is based on the identity of the target file. This mode will better utilize the controller cache, but will not take advantage of multiple paths for a single file.
The current mode employed by a stripe group can be viewed via the “cvadmin” command “show long”, and modified via the “cvadmin” command “multipath”.Permanent modifications may be made by incorporating a “MultiPathMethod” configuration statement in the configuration file for a stripe group.
In the case of an I/O error, that HBA is assessed an “error penalty”, and will not be used for a period of time, after which another attempt to use it will occur.
The first “hard” failure of an HBA often results in a fairly long time-out period (anywhere from 30 seconds to a couple of minutes).
With most HBA’s, once a “hard” failure (e.g. unplugged cable) has been recognized, the HBA immediately returns failure status without a time-out, minimizing the impact of attempting to re-use the HBA periodically after a failure. If the link is restored, most HBA’s will return to operational state on the next I/O request.
paths
Display the StorNext disk volumes visible to the local system. The display is grouped by <controller> identity, and will indicate the “Active” or “Passive” nature of the path if the Raid Controller has been recognized as configured in Active/Active mode with AVAT enabled.
proxy [long]
Display Disk Proxy servers and optionally display the disks they serve for this file system.
proxy who hostname
The “who” option displays all proxy connections for the specified host.
qos
Display per-stripe group QOS statistics. Per-client QoS statistics are also displayed under each qos-configured stripe group.
quit
This command will disconnect cvadmin from the FSM and exit.
ras enq event “detail string”
Generate an SNFS RAS event. For internal use only.
ras enq event reporting_FRU violating_FRU “detail string”
Generate a generic RAS event. For internal use only.
repfl
Generate a report that displays the file locks currently held. Note: this command is only intended for debugging purposes by support personnel. In future releases, the format of the report may change or the command may be removed entirely. Running the repfl command will write out a report file and display the output filename.
repof
Generate a report that displays all files that are currently open on each StorNext client. Only file inode numbers and stat information are displayed, filenames are not displayed. Running the repof command will write out a report file and display the output filename. In future releases, the format of the report may change.
resetrpl [clear]
Repopulate Reverse Path Lookup (RPL) information. The optional clear argument causes existing RPL data to be cleared before starting repopulation. Note: resetrpl is only available when cvadmin is invoked with the -x option. Running resetrpl may significantly delay FSM activation. This command is not intended for general use. Only run resetrpl when recommended by Technical Support.
restartd daemon [once]
Restart the daemon process. For internal use only.
select [file_system_name|N]
Select an active FSM to view and modify. If no argument is specified, a numbered list of FSMs and running utilities will be displayed. If there is only one active file system in the list, it will automatically be selected.When a running utility is displayed by the select command, it will show the following information. First the name of the file system is displayed. Following that, in brackets “[]”, is the name of the utility that is running. Third, a letter indicating the access type of the operation. The options here are (W) for read-write access, (R) for read-only access and (U) for unique access. Finally, the location and process id of the running utility is displayed.
If file_system_name is specified, then cvadmin will connect to the current active FSM for that file system. If N (a number) is specified, cvadmin will connect to the Nth FSM in the list. However, only active FSMs may be selected in this form.
show [groupname] [long]
Display information about the stripe groups associated with the selected file system. If a stripe group name groupname is given only that stripe group’s information will be given. Omitting the groupname argument will display all stripe groups associated with the active file system. Using the long modifier will additionally display detailed information about the disk units associated with displayed stripe groups.
start file_system_name [on hostname_or_IP_address]
Start a File System Manager for the file system
file_system_name. When the command is running on an MDC of an HA cluster, the local FSM is started, and then an attempt is made to start the FSM on the peer MDC as identified by the
/usr/cvfs/config/ha_peer file. When the optional
hostname_or_IP_address is specified, the FSM is started on that MDC only. The file system’s configuration file must be operational and placed in
/usr/cvfs/config/<file_system_name>.cfgx before invoking this command. See
snfs_config(5) for information on how to create a configuration file for an SNFS file system.
startd daemon [once]
Start the daemon process. For internal use only.
stat
Display the general status of the file system. The output will show the number of clients connected to the file system. This count includes any administrative programs, such as cvadmin. Also shown are some of the static file-system-wide values such as the block size, number of stripe groups, number of mirrored stripe groups and number of disk devices. The output also shows total blocks and free blocks for the entire file system.
stats client_IP_address [clear]
Display read/write statistics for the selected file system. This command connects to the host FSMPM who then collects statistics from the file system client. The ten most active files by bytes read and written and by the number of read/write requests are displayed. If clear is specified, zero the stats after printing.
stop file_system_name [on hostname_or_IP_address]
Stop the File System Manager for file_system_name. This will shut down the FSM for the specified file system on every MDC. When the optional hostname or IP address is specified, the FSM is stopped on that MDC only. Further operations to the file system will be blocked in clients until an FSM for the file system is activated.
stopd daemon
Start the daemon process. For internal use only.
up groupname
Up the stripe group groupname. This will restore access to the stripe group.
who
Query client list for the active file system. The output will show the following information for each client.SNFS I.D. – Client identifier Type – Type of connection. The client types are: FSM – File System Manager(FSM) process ADM – Administrative(cvadmin) connection CLI – File system client connection. May be followed by a CLItype character: S – Disk Proxy Server C – Disk Proxy Client H – Disk Proxy Hybrid Client. This is a client that has been configured as a proxy client but is operatingas a SAN client. Location – The clients hostname or IP address Up Time – The time since the client connection was initiated License Expires – The date that the current client license will expire
EXAMPLES
Invoke the cvadmin command for FSM host cornice, file system named default.
spaceghost% cvadmin -H k4 -F snfs1
StorNext File System Administrator
Enter command(s)
For command help, enter "help" or "?".
List FSS
File System Services (* indicates service is in control of FS):
1>*snfs1[0] located on k4:32823 (pid 3988)
Select FSM "snfs1"
Created : Fri Jul 25 16:41:44 2003
Active Connections: 3
Fs Block Size : 4K
Msg Buffer Size : 4K
Disk Devices : 1
Stripe Groups : 1
Mirror Groups : 0
Fs Blocks : 8959424 (34.18 GB)
Fs Blocks Free : 8952568 (34.15 GB) (99%)
Show all the stripe groups in the file system;
snadmin (snfs1) > show
Show stripe group(s) (File System "snfs1")
Stripe Group 0 [StripeGroup1] Status:Up,MetaData,Journal
Total Blocks:8959424 (34.18 GB) Free:8952568 (34.15 GB) (99%)
MultiPath Method:Rotate
Primary Stripe 0 [StripeGroup1] Read:Enabled Write:Enabled
Display the long version of the RegularFiles stripe group;
snadmin (snfs1) > show StripeGroup1 long
Show stripe group "StripeGroup1" (File System "snfs1")
Stripe Group 0 [StripeGroup1] Status:Up,MetaData,Journal
Total Blocks:8959424 (34.18 GB) Free:8952568 (34.15 GB) (99%)
MultiPath Method:Rotate
Stripe Depth:1 Stripe Breadth:16 blocks (64.00 KB)
Affinity Set:
Realtime limit IO/sec:0 (~0 mb/sec) Non-Realtime reserve IO/sec:0
Committed RTIO/sec:0 Non-RTIO clients:0 Non-RTIO hint IO/sec:0
Disk stripes:
Primary Stripe 0 [StripeGroup1] Read:Enabled Write:Enabled
Node 0 [disk002]
Down the stripe group named stripe1;
snadmin (snfs1) > down stripe1
Down Stripe Group "stripe1" (File System "snfs1")
Stripe Group 0 [stripe1] Status:Down,MetaData,Journal
Total Blocks:2222592 (8682 Mb) Free:2221144 (8676 Mb) (99%)
Mirrored Stripes:1 Read Method:Sticky
Primary Stripe 0 [stripe1] Read:Enabled Write:Enabled
FILES
/usr/cvfs/config/*.cfgx
SEE ALSO
