NetApp performance Data Collection Command

http://vipulvajpayeestorage.blogspot.com/2012/07/netapp-performance-data-collection.html

NetApp performance Data Collection Command & Tools(NFS)

The following Data Ontap tools can be used to collect the performance data:

Sysstat, nfsstat, nfs.mounted.trace, netstat, ifstat, nfs_hist, stats, statit, netdiag and wafl_susp, pktt

And from client side.

Ethereal, netapp-top.pl, perfstat, sio.

nfsstat : Display statistical information about NFS and remote procedure call (RPC) for storage system. Per client stats can be collected and displayed vianfs.per_client_stats.enable

nfsstat options

nfsstat –h : Display per-client statistics since last zeroed.

nfsstat –l : Display list of clients whose statistics were collected on per-client basis.

nfsstat –z: zeroes current cumulative and per-client statistics

nfsstat –c: includes reply cache statistics.

nfsstat –t: Display incoming messages in addition to reply cache statistics.

nfsstat –C: Dislpay number and type of NFS v2, v3 requests received by all FlexCache volumes.

nfsstat –d : this diagnostic option allows for debugging of all NFS-related traffic on the network. this cmd is most commonly used option used to decode export and mountd problem.

NFS Mounting monitoring

NFS mountd traces enables tracing of denied mount requests against the storage system

-nfs.mountd.trace

-Enable option only during session as there is a possibility of numerous syslog hits during DOS attacks.

This option should only be enabled during a debug session.

“options nfs.mountd.trace on”

nfs_hist command Overview

Advanced command that display NFS delay time distributions

Syntax nfs_hist [-z]

-z reinitializes delay distributions so that subsequent use displays information about messages processed since distributions were zeroed.

This is advanced mode command.

In takeover mode this display combined delay distributions for the live storage system and failed system.

This command is good to understand that how the system is working when one is attempting to understand NFS performance issue.

pktt overview

 It is Data ontap utility for packet capture and captures data for further analysis by support personal

Syntax: pktts start  <if>|all [-d dir] [-m pklen] [-b bsize] [-i ipaddr –i…]

For ex:

pktt start fa3 –d / -s 100m –b 128k

This starts capturing traffic on the “fas3” interface , writing to a file called  “/fa3.trc”, which will be allowed to grow to a maximum size of 100MB with a 128KB buffer.

Netapp-top.pl Script Overview

A perl script , downloadable from the NOW site and this display top NFS Clients currently most active for the storage system.

-Assists in identifying problematic clients by providing per-client NFS operation statistics.

-Use on Solaris, Linux or other versions of UNIX

-Patches also available for unresolved hostnames and for the limitation of intervals recalculation.

To use it

-Download netapp-top.pl script to your UNIX home directory

-Run it from the shell prompt , specifying the storage system you want.

Recommended statistics to collect

From the client

# nfsstat –z (zero the NFS statistics at the client)

# netstat –I (network statistics before the tests)

-Mount the storage system volume with rsize, wsize =32768

Syntax

# mount –o rsize=32768,wsize=32768 storagesystem:/,export><mountpoint>

# cd <mountpoint>

# nfsstat –m (output of the mountpoints and the mount flags)

Time the mkfile command

# time mkfile 1g test (write test)

Time the dd command

# time dd if=/<mountpoint>/test of=/test (read test)

Time the copy command

#time cp test test1 (read and write test)

Verify nfsstat output

nfsstat –c

check the following parameter;

1.       Timeout>5% request timing out before the server can answer them.

2.       Badxid~timeout server slo. Check nfsstat –m

3.       Badxid~0 and timeouts>3% Packets lost in the network , check netstat.  If this numbet is the same as bad calls the network is congested.

4.       Retrans. May indicate network or routing problem if retransmit>5%

sio utility

Overview

-Acronym for simulated I/O

_general-purpose load generator

-Allow for different block size read and write ops

-Performs synchronous I/Os to the specified file(s)

-collects basic statistics

Syntax

sio Read% Rand% Blk_Size File_Size Seconds Thread Filename [Filename]