From: Taylor, Vince (TaylorV@logica.com)
Date: Tue Jul 30 2002 - 07:56:09 EDT
Hi all,
Original message at the end. Thanks to everyone who offered suggestions,
far too many people to thank individually. A quick summary is that it is
far too difficult to try and solve this sort of problem unless you can get
access to the machines and a lot more information. Bill Hassell probably
summed it up best...
-----
Hi,
Without a *LOT* of additional information, this would be impossible to
troubleshoot. The processes may have scaled themselves incorrectly and used
much more than the available memory, causing severe swapping, or a timing
problem between the processes has shown up due to less swapping. The biggest
questions is: did you use the additional RAM (what did Glance report?)
Bill
-----
We did use a bit of the extra RAM (0.8GB according to Glance) but we were
not swapping/paging before we added the extra memory. We also reduced the
dynamic buffer cache by half (now set to max 10% - we could reduce further
really) to keep the same amount of physical buffer available. The Informix
database engine was running on the other machine, so did not affect this
machine with the extra 4GB RAM.
Several suggestions were to increase the real swap space to be about 1.5
times the amount of RAM. By default I would do this, but we have no
available disk space on which to do this. However, my understanding of the
swapmem_on=1 kernel parameter was to make it possible (and improve the
performance) when there is more RAM than physical swap space.
One suggestions was that we may have swapped one bottleneck for another
(added extra memory, but then ran out of CPU/disk performance) However we
were rarely if ever paging/swapping before we added memory, we originally
had about 90% memory utilisation and this dropped when we added the extra
4GB (as you would expect) to about just over 55%, and there was still no
paging/swapping. No additional processes were added, no other kernel
parameters were changed (apart from the buffer cache ones)
I also received a few links to the old archive stuff on memory problems,
which I have included here (although they are quite old, they are
interesting but unfortunately did not solve the problem)
http://www.dutchworks.nl/htbin/hpsysadmin?h=3&dn=14361&q=adding%20memory&fh
http://www.dutchworks.nl/htbin/hpsysadmin?h=3&dn=42071&q=adding%20memory&fh
http://www.dutchworks.nl/htbin/hpsysadmin?h=3&dn=50814&q=adding%20memory&fh
Another suggestion was that HP had inserted the memory in the wrong order,
or that the smaller memory SIMMs were in the lower slots. As far as I know
this was not the case, but I was not on site when these changes were made,
and I have no reason to suspect that the HP engineer was at fault. In the
past they have always be very good. But worth checking just in case.
In summary, thanks for your help, but this seems to be a "do it and diagnose
it" at the time. Unfortunately I am working on this from a different
country to where to hardware is, and it is a mission critical system, so
redoing this and then looking to solve it is not a possibility. I think we
will have to invest in 4GB of extra RAM for our testbed systems and see if
we can work it out there.
Cheers,
Vince
-----Original Message-----
From: Taylor, Vince [mailto:TaylorV@logica.com]
Sent: Wednesday, July 24, 2002 15:36
To: hpux-admin@dutchworks.nl
Subject: [HPADM] Strange performance problems when adding memory...
Hi all,
We have an N class running HPUX 11, 8 CPU machine, 4 GB RAM, 5GB swap,
kernel parameter swapmem_on set to 1. We are running a telecoms application
which involves an Informix Database (running on a seperate machine via
socket connections) and multiple call handling processes.
The customer has insisted that we increase the RAM from 4GB to 8GB. We
disagree, but to keep them happy, we did add the extra memory. HP went on
site and did the memory upgrade (and downgrade later!) and are convinced
that they put the memory in the correct slots etc. etc. I have no reason to
disbelieve them, as we have never had a problem with their field engineers
in the past. However, once added, the performance of the machine slumped
dramatically, and was performing significantly worse after this memory was
added (it was taking 10 seconds to do what it would normally do in 2 seconds
- that sort of magnitude). We have since removed the memory, and it is now
running fine again. However, the customer still wants the memory adding.
We are trying to find an explanation as to why this is happening. Anyone
any clues, or wants more info to help me work this out?
I will summarise
Cheers,
Vince
This e-mail and any attachment is for authorised use by the intended
recipient(s) only. It may contain proprietary material, confidential
information and/or be subject to legal privilege. It should not be copied,
disclosed to, retained or used by, any other party. If you are not an
intended recipient then please promptly delete this e-mail and any
attachment and all copies and inform the sender. Thank you.
--
---> Please post QUESTIONS and SUMMARIES only!! <---
To subscribe/unsubscribe to this list, contact
majordomo@dutchworks.nl
Name: hpux-admin@dutchworks.nl Owner:
owner-hpux-admin@dutchworks.nl
Archives: ftp.dutchworks.nl:/pub/digests/hpux-admin (FTP, browse
only)
http://www.dutchworks.nl/htbin/hpsysadmin (Web, browse &
search)
This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you.
--
---> Please post QUESTIONS and SUMMARIES only!! <---
To subscribe/unsubscribe to this list, contact majordomo@dutchworks.nl
Name: hpux-admin@dutchworks.nl Owner: owner-hpux-admin@dutchworks.nl
Archives: ftp.dutchworks.nl:/pub/digests/hpux-admin (FTP, browse only)
http://www.dutchworks.nl/htbin/hpsysadmin (Web, browse & search)
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 11:02:17 EDT