SUMMARY: Really large memory models and i/o performance

I got responses from Kevin Buterbaugh and Al Hopper.  Al gave me some
detailed
settings for individual disk drives and Kevin pointed me to the "Solaris
Tunable Parameters Reference Manual" on docs.sun.com and the 2nd edition of
"System Performance Tuning."  Both excellent references. I also obtained the
latest version of SE Tools.

The results are:
There is a significant performance bug with VxFS and global file systems. Sun
doesn't know when there will be a fix. We stopped using VxFS. DO NOT USE
Veritas File System in Sun Cluster 3.0.  Use UFS instead and turn on logging.

Virtual Adrian complained about fsflushsr (file system flusher) and suggested
setting tune_t_fsflushr and autoup higher. It took two changes but fsflushr
is
no longer hogging the system.

There is a "bug" (4487602)that talks about an enhancement for making sync run
faster.  It is not scheduled until Solaris 10 (9 is now locked).  The support
types have heard that Sun engineering is looking into making this available
in
8.

After a significant I/O (e.g. backup or restore) sync's will take a long
time.
 This is due to how sync walks the memory looking for dirty pages.  The main
effect will be reboots. My reboots ranges from about 5 minutes to close to an
hour.  The sync time seems to get shorter the less I/O there is, but it takes
quite a while.

We started using "plaiding" (hardware raid 5 LUN's in T3's, stripes across
the
LUN's). Our initial testing shows we can now hump some serious data.  We're
still testing.

Don't bother asking Sun Service for tuning suggestions.  Their mantra is
always "You have to contract with Sun PS."


*************************************************************************
Original Message
We have 6800's in a 2-way cluster (Sun Cluster 3.0) with 18 CPU's and 48G Of
memory to which we are moving an existing application of ~800G of data. While
doing the restore we found that 1)VxFS was about 1/8 the speed of UFS and 2)
that free memory dropped to about 300M (yes 300M). If we aborted the restore
it took over 30 minutes before the disks stopped writting, apparently because
the system was flushing the over 40G of memory to disks (T3+). We are
investigating changing the disk layout but my real question is about tuning
the system for the large memory model. The current literature talks about
large memory as less that 10G and really doesn't discuss Solaris 8. Has
anyone
have CURRENT literature that discusses the new large memory systems (i.e.
large E10K's,6800's, 12K's and 15K's) running Solaris 8? I find information
about settings for fastscan, handspreadpages, maxpgio, autoup and others but
each seems to be for Solaris 7 and older.

As a side note, once we get the data loaded these puppies scream.
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers