Summary: E-450 slows down

Thanks a lot for so many reponses. This is definitely the most useful
mailing lists I have ever used.

Mainly the suggestions were to check the CPU and Swap usage and Paging using
SAR/TOP/vmstat/iostat. Besides that some responses were as follows(The one
by Karl is most different).

Thanks again,
Gaurav

----------
Sudarshan

1. Seems like a classic case of memory leak. You can monitor this using sar
2. Check to see if there is sufficient swap space, Is the disk cache getting
eaten up? If so, it would seem like "memory leak" even if there is none.
3. How rapidly is the database growing? Depending on the nature of your load
scripts, the performance could be affected adversely proportional to the
growth of the database.
4. Use ps to monitor for spurious processes, and top to sort by memory usage
5. Look up Oracle user groups for known memory leak patches

-------
ANDY LEE
check your not being caught by 108089-04 patch.

There is a version of tail (fixed by patch above) that if you do a tail -f
on a
file and you have more
than 2GB swap it nicks 2 gb of swap.

Is your scan rate running high during the slow down ?

Does ps show a process having a strangly high process size ?



-------
Karl Vogel               ASC/YCOA, Wright-Patterson AFB, OH 45433
vogelke_at_dnaco.net                   http://www.dnaco.net/~vogelke

What we should have fought for was representation without taxation.
                                                  --Sam Levenson

----------------------------------------------------------------------------
-
You won't be able to get this unless you have a SunSolve Online account,
but it's worth a look.

http://sunsolve.sun.com/private-cgi/retrieve.pl?type=2&doc=stb/1442
White Papers/Tech Bulletins 1442
Delivering Performance on Sun: System Tuning
Greg Schmitz and Allan Esclamado
30-Apr-1999

This document focuses on techniques for performance tuning for the Sun
computing environment.  It is aimed at system administrators.  Each chapter
concentrates on a different subsystem of the computing environment (e.g.,
Tuning the Solaris Kernel, Memory, Tuning Disk Subsystems, etc.) and the
specific things that can be done to increase performance.

-------------------------------------------------------------------------
If you have an application that's an incredible swap hog, or the system is
really slowing down, try adding the lines below to /etc/system and
rebooting.  I run with these settings and they've never caused me trouble.

*
* Swap
*   System keeps 1/8th of all memory for swap, which is too much for
*   a 4GB system.  Reduce that to 32 Mbytes (4096 8K pages).
set swapfs_minfree=4096
*
* Memory management
*
*   http://www.carumba.com/talk/random/tuning-solaris-checkpoint.txt
*   Tuning Solaris for FireWall-1
*   Rob Thomas robt@cymru.com
*   14 Aug 2000
*
*   On firewalls, it is not at all uncommon to have quite a bit of
*   physical memory.  However, as the amount of physical memory is
*   increased, the amount of time the kernel spends managing that
*   memory also increases.  During periods of high load, this may
*   decrease throughput.
*
*   To decrease the amount of memory fsflush scans during any scan
*   interval, we must modify the kernel variable autoup.  The default
*   is 30.  For firewalls with 128MB of RAM or more, increase this
*   value.  The end result is less time spent managing buffers,
*   and more time spent servicing packets.
set autoup = 120

-------------------------------------------------------------------------
Check your most popular applications (using truss) for the following:

   * lots of kernel-level system calls, like open(), read(), write(); all
     of these require an interrupt plus kernel attention.

   * lots of fork() or exec() calls to start new processes; fork() under
     Solaris is extremely expensive.

   * lots of open files; a program called "lsof" can tell you how
     many file descriptors are being used by anything on your system.

   * any opendir()/readdir() calls for walking through directories to find
     a file; any given directory is stored in a hash table, but the
     contents of the directory have to be scanned linearly, so files in
     large directories (~1000 or more files) will take much longer to
     open or close.

   * size of your inode caches, which keep track of previously-accessed
     files.  Run the DNLC script below as root to see your hit-rate
     percentage.  If it's under 90-95%, you need to up the cache size.

Run "mount" to see how your filesystems are set up.  I'm pretty sure
you can mount filesystems with "noatime" turned on, meaning don't bother
updating the access time whenever a file is opened.  We use this under
Solaris-8, and it makes a *huge* difference if you're doing something
to a large number of small or medium sized files.

---------------------------------------------------------------------------
#!/bin/sh
#
# NAME:
#    dnlc
#
# SYNOPSIS:
#    dnlc
#
# DESCRIPTION:
#    "dnlc" reports on Directory name lookup cache statistics from
#    the kernel.  This corrects a bug in vmstat.
#
#    To change the kernel values, add something like this
#    to /etc/system and reboot.  Both "nnn" numbers should be the
#    same.
#
#         set ncsize = nnn
#         set ufs_ninode = nnn
#
# AUTHOR:
#    Kimberley Brown - UKAC Kernel Support
#    comp.unix.solaris

PATH=/bin:/usr/bin:/usr/local/bin
export PATH

adb -k /dev/ksyms /dev/mem <<END
="**  Directory/Inode Cache Statistics  **"
="----------------------------------------"
ufs_ninode/D"Inode cache size"
ncsize/D"Directory name cache size"
ncstats/D"# of cache hits that we used"
+/D"# of misses"
+/D"# of enters done"
+/D"# of enters tried when already cached"
+/D"# of long names tried to enter"
+/D"# of long name tried to look up"
+/D"# of times LRU list was empty"
+/D"# of purges of cache"
*ncstats%1000>a
*(ncstats+4)%1000>b
*(ncstats+14)%1000>c
<a+<b+<c>n
<a*0t100%<n=D"Hit rate percentage"
="(See /usr/include/sys/dnlc.h for more information)"
END

exit 0
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers