SUMMARY: Setting tcp_recv_hiwat but not seeing the result on the wire

OK, I've been doing this the hard way.  Here's an easier way.

If one sets tcp_recv_hiwat *prior* to Samba loading, then Samba does the right 
thing automatically, no need for anything special in smb.conf

Add this to/etc/system and reboot to increase the maximum allowable TCP Receive 
Window.  [Other applications may be able to take advantage of even larger 
windows, but SMBv1 has a ~64K application buffer and so cannot take advantage of 
anything larger.]

set ndd:tcp_recv_hiwat=65535

/etc/samba/smb.conf
[global]
	socket options = TCP_NODELAY

--sk

-------- Original Message --------
Subject: SUMMARY: Setting tcp_recv_hiwat but not seeing the result on the wire
Date: Mon, 09 Aug 2010 15:59:05 -0700
From: Stuart Kendrick <skendric@fhcrc.org>
To: sunmanagers@sunmanagers.org

Per several correspondents, setting the maximum possible TCP buffer is a
good start, but the application may or may not take advantage of this.

In Samba's case, one can tell Samba to grab a larger buffer:

/etc/samba/smb.conf
[global]
         socket options = SO_RCVBUF=65536

And in fact, I can see the Solaris box advertising a TCP Window size of
64,240 bytes now, when I examine packet traces of file copies over SMB.

A more complete socket options line might look as follows:
[global]
         socket options = TCP_NODELAY SO_SNDBUF=65536 SO_RCVBUF=65536

Thank you for your assistance,

--sk

On 8/8/2010 5:09 PM, Stuart Kendrick wrote:
> Hi folks,
>
> I'm setting TCP Receive Window size (along with other parameters) in a
> custom /etc/init.d script:
>
> /usr/sbin/ndd -set /dev/tcp tcp_max_buf 4096000
> /usr/sbin/ndd -set /dev/tcp tcp_xmit_hiwat 4096000
> /usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 4096000
> /usr/sbin/ndd -set /dev/tcp tcp_wscale_always 1
> /usr/sbin/ndd -set /dev/tcp tcp_tstamp_if_wscale 1
> /usr/sbin/ndd -set /dev/tcp tcp_cwnd_max 4096000
>
> Seems successful.
>
>  > /usr/sbin/ndd -get /dev/tcp tcp_recv_hiwat
> 4096000
>  >
>
> But, for grins, I reboot anyway.
>
>  > /usr/sbin/ndd -get /dev/tcp tcp_recv_hiwat
> 4096000
>  >
>
> However, when I negotiate a TCP session with the box, I see TCP Receive
> Window sizes ranging around 48-49K, as seen in a packet trace, same as I
> what I saw prior to fiddling with tcp_recv_hiwat
>
> https://vishnu.fhcrc.org/fred/telnet-to-fred5.pcap
> https://vishnu.fhcrc.org/fred/shiva-to-fred5-smb-negotiate.pcap
>
>
> I would expect the box to advertise a TCP Receive Window size of
> 4096000, not 49232 and 49202 (telnet/TCP negotiation) or 49680 and 49532
> and 48300 (SMB/TCP negotiation). And for it to use RFC1323 window
> scaling to do so.
>
> Any ideas why not?
>
>  > uname -a
> SunOS fred5 5.10 Generic_142900-05 sun4v sparc SUNW,Sun-Fire-T200
>  >
>
> --sk
>
> Stuart Kendrick
> FHCRC
>
> P.S. The box is slow at receiving files ... I can see TCP Receive Window
> size dropping to 0 regularly ...
> http://vishnu.fhcrc.org/fred/comparing-tcp-window-size-io-graphs.pdf
> This might, of course, stem from slow storage or some other cause, such
> that the box cannot drain the TCP buffer fast enough. But then again,
> this is a 1GigE environment, with fast clients, and by my BDP
> calculation, I need at least a 64K window size in order to keep up with
> them, possibly closer to 128K. Ergo my efforts to fiddle with window size.
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers