SUMMARY: Slow Network Performance On V880

Thanks to all who responded, especially to Kevin Pringge who pointed out
to me the possible effects of a 33MHz bus compared to a 66MHz bus as
discussed in the article at
http://www.beowulf.org/archive/2001-December/005647.html which shows
that performance around 30MB/s may be the practical limit on a 33MHz
slot.  To test the theory that a GigE interface (ge) is drastically
limited when used in a 33MHz slot, I ran a test on a Blade 1500
workstation which had both a 33MHz and 66MHz slot available.  The
results were dramatic in that performance on the 66MHz slow was almost
2.5 times greater. The test was performed with the GE card first in the
33MHz slot which had a maximum throuput of ~35MB/sec. When the card was
moved to the 66MHz slot, I was able to achieve ~80MB/sec, which is about
what I was expecting for a gig connection.

The following is the output from the TTCP bandwidth testing tool:

GE Interface in 33MHz slot:
java ttcp -t 10.0.101.240 -n 65536
Transmit: buflen= 8192  nbuf= 65536 port= 5001 Transmit connection:
  Socket[addr=10.0.101.240/10.0.101.240,port=5001,localport=64869].
Transmit: 536870912 bytes in 15142 milli-seconds = 35455.746 KB/sec
(283645.97 K bps).

GE Interface in 66MHz slot:
java ttcp -t 10.0.101.240 -n 65536
Transmit: buflen= 8192  nbuf= 65536 port= 5001 Transmit connection:
  Socket[addr=10.0.101.240/10.0.101.240,port=5001,localport=64849].
Transmit: 536870912 bytes in 6676 milli-seconds = 80418.055 KB/sec
(643344.44 Kb ps).

Further to this, I was able to find a SunSolve document titled "Sun[TM]
GigaSwift Ethernet (ce) performance slow." (DocID 81991) which did
reveal an interface kernel counter 'rx_ov_flow' incrementing. A sign
that the 33MHz slow is potentially the problem.

Once again, thanks to those who responed. Original question below.

Peter


> -----Original Message-----
> From: Hackett, Peter
> Sent: Friday, November 04, 2005 12:06 PM
> To: 'sunmanagers@sunmanagers.org'
> Subject: Slow Network Performance On V880
>
>
> Good Day,
>
> I have a Solaris 9 V880 NFS server w/4 X 900MHz CPU's and 8GB of
> Memory.  It's configured with 5 HBA's for various disk and SAN
> connectivy, and a CE interface (@1000FD) for network connectivity. The

> problem I'm experiencing is that I cant seem to get more than 30 MB
> (240Mbps) through the network interface where I am expecting around
> 85MB (680Mbpb). I'm using TTCP and simply FTP to measure.  I am trying

> to find where the problem lies and where the bottleneck is.  What I've

> looked at so far is SE-Toolkit's nx.se which indicates 0 Coll%, 0
> NoCP/s, and 0 Defr/s, and I've looked at nicstat (from
> brendangregg.com) which also show very low utilization and zero
> saturation.  I also did a switch from ge to ce to no avail. So, this
> leaves me with the though that maybe there's not enough CPU resources
> to drive all this I/O, but CPU's typically don't run greater than
> ~70%, with %sys not much greater than 25% or so, which is where I
> though I would see problems if CPU was the factor.  Anyway,  I'm
> looking for how I determine where and why my network throughput is
> about a third of what it should be? What metrics / tools can I use to
> isolate  and pinpoint the bottleneck? Any and all help and insight
> would be appreciated.
>
> Peter
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers