Very High Number of Major Faults

From: Hackett, Peter (PHackett@talisman-energy.com)
Date: Mon Mar 12 2007 - 17:35:29 EST


Good day.

I have a Sun Fire V490 (8 CPU/16GB Mem) server running Sol 10 6/06 and
am seeing some concering metrics when running mpstat, especially major
faults. The server acts mostly as an NFS server as well as a samba
server for user home and data directories.

Any thoughts on how to proceed or references appreciated.

Thanks,
Peter

--
box:# mpstat 1 5
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt
idl
  0  115 3000773253  284    14    7  526    2  179   45    0   579    1
5   0  94
  1   96 4221576187  317 14242   50  769    3  211   55    0   788    1
6   0  93
  2  114 2728721864  414   130  122  459    2  172   46    0   490    1
5   0  94
  3  122 5284038486  373   255  149  928    3  247   60    0   561    1
7   0  92
 16  109 2163004186  427   195   34  696    3  163   54    0   568    1
6   0  94
 17   94 5062157988  312    15    7  668    3  226   47    0   905    1
5   0  94
 18   79 2554937924  291    23   13  716    2  170   54    0   789    1
6   0  94
 19   74 3968731246  339  1941 1934  306    2  121  125    0   905    1
9   0  90
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt
idl
  0    6 12757328602  277    24    7 2806    9  912  395    0   389    0
22   0  78
  1    0 12757328602  126 54715   42 2778   16  837  328    0   719    1
21   0  78
  2    0 25514657204  287    80   61 2507   11  898  355    0   450    1
20   0  79
  3  148 12884901888 1339   381  266 3480    8  989  376    0   656    1
24   0  75
 16    0 4252442867 4886  1581  394 3296    8  818  436    0   329    1
25   0  74
 17    0   0  518    36    6 2446   22  856  295    0   599    1  19   0
80
 18    0   0  119    19    6 3598    4  863  474    0   321    1  25   0
74
 19    0 4252442867  884  7330 7316 1153    9  458  554    0   605    1
37   0  62
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt
idl
  0    0 8504885735  137    35    6 2698   20  910  401    0   968    3
23   0  74
  1    0 4252442867  123 58759   32 2746    4  834  455    0   140    1
24   0  75
  2    0 4252442867  173    84   62 2491   11  929  422    0   810    2
24   0  74
  3    1   1 1601   450  339 3396    8  949  457    0    66    1  26   0
73
 16   46 4252442867 6649  1821  439 2930   16  766  458    0  1650    2
28   0  70
 17    0   0  124    29    7 2573   14  841  344    0   548    0  23   0
77
 18   38   0  254    30    8 3482   12  871  458    0   823    1  28   0
71
 19    0 4252442867  283  7537 7526 1019    3  391  576    0    31    0
41   0  59
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt
idl
  0    0 17179869184  177    35    7 2713   11  898  394    0   552    0
22   0  78
  1    0 17009771469  144 46591   36 2477    4  759  313    0   175    0
19   0  81
  2    0   0  163    66   39 2456   13  890  357    0   527    1  20   0
79
  3    0 29767100071  488   301  188 3203    4  866  390    0   111    0
22   0  78
 16    0   0 6115  1655  481 2996   12  734  448    0   506    1  24   0
75
 17    0 17009771469  121    17    6 2342    4  820  355    0   220    1
18   0  81
 18    0 17179869184  195    28    7 3295    8  819  419    0   483    1
24   0  75
 19    0 8504885735  336  7267 7251 1170    8  461  433    0   493    0
32   0  68
CPU minf mjf xcal  intr ithr  csw icsw migr smtx  srw syscl  usr sys  wt
idl
  0    0 50529027012  172    18    6 2013    7  668  242    0   161    1
16   0  83
  1    0 21262214337   59 41316   36 2317    1  664  232    0    57    0
16   0  84
  2    0 38271985806  211    50   39 1815    2  660  236    0   263    0
15   0  85
  3    0 38271985806  312   285  182 3041    1  793  264    0    41    0
19   0  81
 16    0 17009771469 4362  1263  370 2691    1  558  248    0   132    0
18   0  82
 17    0 34019542939  153    12    6 2159    2  725  181    0    70    0
15   0  85
 18    0 37896770259  115    26    6 2797    9  634  269    0   238    1
20   0  79
 19    0 42949672960  260  6750 6741 1003    2  406  389    0    70    0
26   0  74
box:#
---
box:# vmstat -s
        0 swap ins
        0 swap outs
        0 pages swapped in
        0 pages swapped out
179322896 total address trans. faults taken
22267588658809564 page ins
    24853 page outs
93602417010661737 pages paged in
    43523 pages paged out
 52907576 total reclaims
 52902100 reclaims from free list
        0 micro (hat) faults
179322896 minor (as) faults
6491323377071110 major faults
 51911625 copy-on-write faults
 24605016 zero fill page faults
    56316 pages examined by the clock daemon
        0 revolutions of the clock hand
    38042 pages freed by the clock daemon
  1573402 forks
   276553 vforks
  1740918 execs
1135839472 cpu context switches
3770341532 device interrupts
222560254 traps
1247049897 system calls
480851560 total name lookups (cache hits 100%)
  1629557 user   cpu
 10999507 system cpu
166008176 idle   cpu
        0 wait   cpu
box:#
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers


This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:41:45 EDT