From: Siebert, Aaron (aaron.siebert@nagrastar.com)
Date: Tue Apr 27 2004 - 09:33:03 EDT
Dr. Blinn, I am always excited to see your response.
I have read the advfs documentation but it is unclear about if the
parameters are related to UBC or not. My hope was that they were so that
reducing the UBC values would effectively reduce the percentage of
memory used by advfs. ;
AdvfsAccessMaxPercent Default value: 80 (percent)
The maximum percentage of the malloc pool that can be allocated for
access structures.
AdvfsCacheMaxPercent Default value: 7 (percent)
The percentage of system memory that is allocated to the AdvFS buffer
cache.
>From System configuration and tuning:
To free memory resources, you may want to decrease the amount of memory
allocated to the AdvFS buffer cache. Decreasing the cache size also
decreases the overhead associated with managing the cache. The advfs
subsystem attribute AdvfsCacheMaxPercent determines the maximum amount
of memory that can be used for the AdvFS buffer cache. The default is 7
percent of physical memory. The minimum is 1 percent, and the maximum is
30 percent. If you are not using AdvFS or if you do not reuse many
files, decrease the cache size to 1 percent. If you are using AdvFS, but
you have a VLM system, you may also want to decrease the cache size.
However, decreasing the size of the AdvFS buffer cache may adversely
affect AdvFS I/O performance if you access and then reuse many files.
As you can see that the decription of the parameter from the man pages
doesn't say anything about UBC however the description from the guide
does. Either way, what I don't understand is how the system consumes
over 4GB of memory over a week and doesn't seem to reclaim it.
After some tuning of ubc and sga
Virtual Memory Statistics: (pagesize = 8192)
procs memory pages intr cpu
r w u act free wire fault cow zero react pin pout in sy cs us
sy id
5 386 40 135K 1M 212K 887K 163K 305K 324 151K 0 821 4K 2K 3
5 92
5 384 40 143K 1M 220K 38K 5571 22K 0 4931 0 843 2K 4K 3
4 94
6 386 39 152K 1M 229K 40K 7096 18K 0 6140 0 1K 5K 5K 4
6 90
5 387 40 160K 1M 238K 15K 325 9762 0 621 0 1K 909 4K 2
3 96
5 385 42 173K 1M 239K 40K 7266 18K 0 6354 0 896 5K 4K 3
6 91
5 387 40 190K 1M 239K 11K 131 8821 0 246 0 883 825 4K 1
3 96
6 387 39 207K 1M 239K 38K 6975 18K 0 5885 0 1K 4K 4K 3
6 91
5 387 40 223K 1M 239K 14K 262 8788 0 493 0 1K 933 4K 1
3 96
5 387 40 239K 1M 239K 35K 3392 14K 0 3057 0 1K 4K 7K 3
7 91
5 387 40 252K 1M 239K 34K 4028 13K 0 3542 0 1K 2K 6K 2
5 93
5 387 40 263K 1M 240K 15K 384 6544 0 389 0 1K 2K 5K 1
4 95
Current, Here is an example of the system consuming all the memory.
r w u act free wire fault cow zero react pin pout in sy cs us
sy id
5 567 40 836K 55 1M 10K 467 1719 7322 665 13 1K 2K 6K 6
34 60
5 566 41 836K 655 1M 36K 6815 13K 17K 6075 0 1K 5K 6K 8
8 83
13 559 41 836K 164 1M 11K 885 5073 16K 1101 0 4K 6K 6K 16
45 39
6 567 40 836K 57 1M 38K 6862 12K 13K 6489 28 6K 9K 7K 17
72 11
5 565 41 836K 125 1M 8664 686 1261 4246 610 0 3K 2K 6K 13
6 81
9 569 41 836K 130 1M 29K 4560 9684 7041 4319 3 1K 6K 6K 10
9 81
5 565 41 836K 124 1M 20K 3114 5390 4578 2907 7 1K 1K 7K 11
5 84
5 563 42 833K 218 1M 11K 693 1824 4737 726 0 1K 2K 6K 9
5 86
6 565 40 830K 127 1M 35K 6767 11K 13K 6092 0 1K 4K 5K 8
7 84
16 557 43 828K 222 1M 6518 325 2342 21K 360 0 2K 2K 4K 6
53 41
18 554 42 832K 174 1M 5707 653 2307 7135 725 0 711 3K 1K 1
99 0
12 565 40 836K 200 1M 0 0 0 0 0 0 511 1K 1K 1
99 0
6 569 41 838K 57 1M 89K 17K 25K 26K 14K 70 1K 9K 3K 3
92 5
Some other suspect parameters
Vfs-bufcache:
At boot time, the kernel allocates wired memory for the metadata buffer
cache. The cache acts as a layer between the operating system and disk
by storing recently accessed UFS and CDFS metadata, which includes file
header information, superblocks, inodes, indirect blocks, directory
blocks, and cylinder group summaries. Performance is improved if the
data is later reused and a disk operation is avoided. The metadata
buffer cache uses bcopy routines to move data in and out of memory.
Memory in the metadata buffer cache is not subject to page reclamation.
The size of the metadata buffer cache is specified by the value of the
vfs subsystem attribute bufcache.
Vm-syswirepercent:
You can reduce the amount of dynamically wired memory by reducing the
value of the vm subsystem attribute vm-syswiredpercent. The default
value is 80 percent.
Thoughts?
Regards,
Aaron Siebert
-----Original Message-----
From: Dr Thomas.Blinn@HP.com [mailto:tpb@doctor.zk3.dec.com]
Sent: Monday, April 26, 2004 6:51 PM
To: Siebert, Aaron
Subject: Re: Disappearing free pages in memory
I hate to say it, but what you are describing about how the system
uses the available memory (instead of, say, paging to disk which is
a losing proposition any time you can avoid it) is exactly how it's
supposed to work, from my understanding.
Have you looked at the sys_attrs_advfs reference page? It should
help explain the AdvFS parameters you are asking about (I won't
claim to know much more than it says).
Tom
> Es40 4.0G 16 GB memory 4 cpu
>
> We are fighting a disappearing free memory issue with a large oracle
> server. Upon boot and mount of the db we have 1M free pages. This
> decreases rapidly over time. Until it reaches the thresholds set by
the
> vm subsystem attributes for minimum free pages.
>
> Oracle SGA is set to 6 GB. This leaves 10 GB physical memory free. We
> have reduced the ubc values below:
> vm:
> vm-maxvas = 7516192768
> vm-mapentries = 400
> ubc-minpercent = 1
> ubc-maxpercent = 5
> ubc-borrowpercent = 3
> vm-vpagemax = 917504
> vm-aggressive-swap = 1
>
> Here are the advfs settings, I am curious about the CacheMax and
> AccessMax values:
>
> advfs:
> AdvfsCacheMaxPercent = 7
> AdvfsAccessMaxPercent = 80
>
> 1. Are these 2 values in relation to UBC, total system memory?
> 2. Is the AdvfsAccessMaxPercent value in relation to
> AdvfsCacheMaxPercent?
> 3. Kernel is allocating by ps -ef -o vsize more than 4.5GB
> 4. Does anyone have any other information that might help us to
> determine why our system over time continues to lose free pages?
>
> Thanks for everyones help
>
> Aaron Siebert
> Nagrastar Customer Support Engineer
> 303-706-5492 fax 303-706-5719
> Aaron.Siebert@NagraStar.com
Tom
Dr. Thomas P. Blinn + Tru64 UNIX Software + Hewlett-Packard Company
Internet: tpb@zk3.dec.com, thomas.blinn@compaq.com, thomas.blinn@hp.com
110 Spit Brook Road, MS ZKO3-2/W17 Nashua, New Hampshire 03062-2698
Alpha Tru64 UNIX kernel support Phone: (603) 884-0646
ACM Member: tpblinn@acm.org PC@Home: tom@felines.mv.net
Worry kills more people than work because more people worry than work.
Keep your stick on the ice. -- Steve Smith ("Red Green")
My favorite palindrome is: Satan, oscillate my metallic sonatas.
-- Phil Agre, pagre@alpha.oac.ucla.edu
Yesterday it worked / Today it is not working / UNIX is like that
-- apologies to Margaret Segall
Opinions expressed herein are my own, and do not necessarily represent
those of my employer or anyone else, living or dead, real or imagined.
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:49:57 EDT