SUMMARY:Solaris 8 on UltraEnterprise 450 install

Hi,

Thanks to Simon Kuhn, Jay Lessert, Adam Levin, Krister Bergman, Thomas Anders, 
and Karl Vogel for their answers.

Adam thought that maybe I was pushing it a bit, having so many disks attached 
to the poor beast.  Since it's all we're gonna get for a while, there's no 
alternative.  He pointed me to a page
http://www.westnet.com/~levins/e450scsi.html
describing how the PCI busses and SCSI controlers are assigned in a 450.  The 
least I could do was try to balance the load among the several PCI controlers.

Karl provided some pointers to reference material (white papers) about solaris 
tuning, and some more meterial pasted below:
---
http://sunsolve.sun.com/private-cgi/retrieve.pl?type=2&doc=stb/1442
White Papers/Tech Bulletins 1442
Delivering Performance on Sun: System Tuning
Greg Schmitz and Allan Esclamado
30-Apr-1999

This document focuses on techniques for performance tuning for the Sun
computing environment.  It is aimed at system administrators.  Each chapter
concentrates on a different subsystem of the computing environment (e.g.,
Tuning the Solaris Kernel, Memory, Tuning Disk Subsystems, etc.) and the
specific things that can be done to increase performance.

-------------------------------------------------------------------------
If you have an application that's an incredible swap hog, or the system is
really slowing down, try adding the lines below to /etc/system and
rebooting.  I run with these settings and they've never caused me trouble.

*
* Swap
*   System keeps 1/8th of all memory for swap, which is too much for
*   a 4GB system.  Reduce that to 32 Mbytes (4096 8K pages).
set swapfs_minfree=4096
*
* Memory management
*
*   http://www.carumba.com/talk/random/tuning-solaris-checkpoint.txt
*   Tuning Solaris for FireWall-1
*   Rob Thomas robt@cymru.com
*   14 Aug 2000
*
*   On firewalls, it is not at all uncommon to have quite a bit of
*   physical memory.  However, as the amount of physical memory is
*   increased, the amount of time the kernel spends managing that
*   memory also increases.  During periods of high load, this may
*   decrease throughput.
*
*   To decrease the amount of memory fsflush scans during any scan
*   interval, we must modify the kernel variable autoup.  The default
*   is 30.  For firewalls with 128MB of RAM or more, increase this
*   value.  The end result is less time spent managing buffers,
*   and more time spent servicing packets.
set autoup = 120

-------------------------------------------------------------------------
Check the size of your inode caches, which keep track of previously-
accessed files.  Run the DNLC script below as root to see your hit-rate
percentage.  If it's under 90-95%, you need to up the cache size.
The easiest way to do that is change maxusers in /etc/system to a nice
high number like 2048.

Run "mount" to see how your filesystems are set up.  You can mount
filesystems with "noatime" and journalling ("logging") turned on,
meaning don't bother updating the access time whenever a file is opened.
We use this under Solaris-8, and it makes a *huge* difference if you're
doing something to a large number of small or medium sized files.

---------------------------------------------------------------------------
#!/bin/sh
#
# NAME:
#    dnlc
#
# SYNOPSIS:
#    dnlc
#
# DESCRIPTION:
#    "dnlc" reports on Directory name lookup cache statistics from
#    the kernel.  This corrects a bug in vmstat.
#
#    To change the kernel values, add something like this
#    to /etc/system and reboot.  Both "nnn" numbers should be the
#    same.
#
#         set ncsize = nnn
#         set ufs_ninode = nnn
#
# AUTHOR:
#    Kimberley Brown - UKAC Kernel Support
#    comp.unix.solaris

PATH=/bin:/usr/bin:/usr/local/bin
export PATH

adb -k /dev/ksyms /dev/mem <<END
="**  Directory/Inode Cache Statistics  **"
="----------------------------------------"
ufs_ninode/D"Inode cache size"
ncsize/D"Directory name cache size"
ncstats/D"# of cache hits that we used"
+/D"# of misses"
+/D"# of enters done"
+/D"# of enters tried when already cached"
+/D"# of long names tried to enter"
+/D"# of long name tried to look up"
+/D"# of times LRU list was empty"
+/D"# of purges of cache"
*ncstats%1000>a
*(ncstats+4)%1000>b
*(ncstats+14)%1000>c
<a+<b+<c>n
<a*0t100%<n=D"Hit rate percentage"
="(See /usr/include/sys/dnlc.h for more information)"
END

exit 0
---

A while ago, I said:
> I will soon be upgrading an UltraEnterprise 450 machine to Solaris 8 2/
> 02, and  I still have a few nagging doubts...

> The machine will have 4 GB RAM, about 1 TB disks, and Gigabit ethernet
> (Sun Vector Gigabit Ethernet PCI NIC).  It is used mainly as an NFS
> file server, and as a web server.  The main data partition will be a
> SDS concat/stripe of about 900 GB.  The disks are in the built-in
> racks, and in three D1000 cabinets.

> doubt #1: is the vge driver included in the Solaris 8 2/02 release,
> and part  of the normal install ?

Krister warned me that my vge board was not supported in Solaris 8, and was 
quite right about it.  I subsequently opened a call (the machine has a Silver 
support contract) and Sun Switzerland was nice enough to send me a replacement 
GE board for free.

> doubts #2: I have read through the "Solaris Tunable Parameters
> Reference  Manual", and came to the part on Per-Route Metrics, which
> explains that for  gigabit ethernet devices, tcp_recv_hiwat should be
> increased.  Question is:  how much ?  Also, should tcp_xmit_hiwat be
> increased as well ?

No specific answer.

> doubt #3: Am I correct in assuming that tcp_max_buf should be set
> around  104,857,600 (i.e., 100 times its default value) ?

No specific answer.

> doubt #4: the machine currently has 1.7 GB RAM and Solaris 2.6 (+
> patches).   If I look at vmstat, the sr column appears to stay at 0.
> Should I then set  dopageflush to 0 ?  Or am I better off increasing
> autoup ?

No specific answer.

> doubt #5: it sometimes happens in the current setting that a run-away
> perl script from the web server eats up all the swap space by filling
> up /tmp (mounted from swapfs) and the system pretty much hangs at that
> point.  Is setting tmpfs:tmpfs_minfree to some megabytes the right way
> to prevent the  freeze and allow root logins to kill the bad process ?
>  Is the default value  really 256 *bytes* ?

Jay had a simple solution:
Keep it simple.  Just limit /tmp size in vfstab:
    swap        -       /tmp    tmpfs   -       yes     size=500m

> doubt #6: what else am I missing ? ;) 

Simon said to be a bit weary of the 02/02 release, as the statd daemon has a 
known bug that will cause it to fail every now and then.  I can confirm this 
is the case, and it has to be restarted about once daily.  There's supposed to 
be a patch available in mid-May.

Regards,
					Christian
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers