Saturday, December 13, 2008

Diagnostic command line tools

OS X comes with a number of command line tools to measure system performance and individual application performance. Using these tools, you can track down bottlenecks and discover misbehaving applications -- those acting like rude, thoughtless, little pigs.

top

The top program shows a full screen of stats on overall CPU usage, memory usage, load average, and how many resources each process is using. The default mode for top is an interactive display that updates once per second. Applications are sorted by CPU usage in descending order. To quit while in interactive mode, type q.

Processes:  64 total, 3 running, 1 stuck, 60 sleeping... 204 threads   16:21:14
Load Avg: 0.52, 0.43, 0.25 CPU usage: 9.3% user, 22.9% sys, 67.8% idle
SharedLibs: num = 27, resident = 11.0M code, 1.02M data, 3.44M LinkEdit
MemRegions: num = 11872, resident = 388M + 9.48M private, 111M shared
PhysMem: 117M wired, 393M active, 374M inactive, 885M used, 138M free
VM: 5.57G + 21.5M 284085(0) pageins, 19704(0) pageouts

PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
23002 mdimport 0.1% 0:00.38 4 62 52 996K 4.27M 3.01M 40.0M
22998 top 12.3% 0:24.47 1 19 22 476K 476K 900K 27.0M
22977 lookupd 0.0% 0:00.09 2 34 37 400K 1.06M 1.16M 28.5M
22971 mdimport 0.0% 0:00.31 3 60 46 864K 3.37M 2.76M 39.0M
19117 firefox-bi 10.1% 12:34:42 10 196 781 143M 59.6M 155M 983M
18711 bash 0.0% 0:00.14 1 14 17 216K 884K 748K 27.2M
18710 login 0.0% 0:00.02 1 16 37 124K 516K 460K 26.9M
17409 httpd 0.0% 0:00.00 1 11 87 104K 2.47M 476K 31.9M
7211 usbmuxd 0.0% 0:00.02 2 23 23 164K 436K 284K 27.0M
7193 iTunesHelp 0.0% 0:01.38 2 55 99 608K 4.67M 1.55M 108M
4618 Terminal 4.2% 27:15.18 10 173 228 3.91M+ 16.5M+ 9.18M+ 135M+
717 AppleSpell 0.0% 0:00.28 1 32 36 284K 2.06M 860K 37.7M
712 System Eve 0.0% 0:03.10 1 62 109 928K 7.02M 1.30M 109M
682 cupsd 0.0% 0:58.79 2 30 25 568K 1.03M 844K 27.8M
380 DashboardC 0.0% 0:56.12 3 78 160 3.59M 6.57M 4.13M 116M
379 DashboardC 0.0% 0:07.55 4 86 158 5.75M 6.66M 6.56M 115M

Top can also be run in batch mode where it writes output to the terminal a set number of times, then quits. This is ideal for feeding data to a parsing program for generating your own stats or filtering. For example, to show top output once, use:

top -l 1
note: top minus el one

To sort the output by CPU usage, use:

top -l 1 -o cpu

To sort the output by resident memory usage (physical memory), use:

top -l 1 -o rsize

To sort the output by virtual memory usage (total memory), use:

top -l 1 -o vsize

fs_usage

Report system calls and page faults related to filesystem activity in real-time. It is a continuous display and if unfiltered, is overwhelming. It is usually best to limit the output to a single process or to watch network activity. It must be run as root.

To limit the output to a process or command name, include it at the end of the command:

fs_usage 19117

To limit the output to just network related system calls, use:

fs_usage -f network

To limit the output to just filesystem related system calls, use:

fs_usage -f filesys

See also lsof

vm_stat

The vm_stat program displays Mach (the kernel) virtual memory statistics. It is useful to seeing if your system is memory starved.

Here is the output from vm_stat:

Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free: 63416.
Pages active: 103005.
Pages inactive: 68102.
Pages wired down: 27621.
"Translation faults": 308932364.
Pages copy-on-write: 7961115.
Pages zero filled: 204818181.
Pages reactivated: 1571266.
Pageins: 279532.
Pageouts: 19704.
Object cache: 1693448 hits of 4581511 lookups (36% hit rate)
If Pages free is low, you need more memory to run all your applications.

You can add an interval (in seconds) as a parameter to vm_stat and it will show your memory and paging activity in real time. For example, vm_stat 1 will output a new line every second -- useful to see if memory is taking a beating with a lot Pageins/Pageouts, but it won't pinpoint an offending application.