From: DRoss-Smith@reviewjournal.com
Date: Thu Dec 06 2007 - 10:23:16 EST
Last week I'd asked for help on speeding up NFS over gigabit for large
file transfers (several sybase databases over 100-500GB each)
dumped via NFS to v240s.
Thanks to Joe Fletcher and Vitaly Porotikov for useful information.
Joe suggested that I just dump my v880's and go to x64.
Vitaly suggested jumbo frames.
I can't do either at the moment but I can look at their other suggestions:
Joe suggested tweaking maxphys
set maxphys=1048576
set md:md_maxphys=1048576
which I already had set and he has the following tweaks in a startup
script:
#!/bin/sh
#
#
# Setup for Giga Bit Ethernet
#
case "$1" in
'start')
/usr/sbin/ndd -set /dev/tcp tcp_xmit_hiwat 65535
/usr/sbin/ndd -set /dev/tcp tcp_recv_hiwat 65535
/usr/sbin/ndd -set /dev/udp udp_xmit_hiwat 65535
/usr/sbin/ndd -set /dev/udp udp_recv_hiwat 65535
/usr/sbin/ndd -set /dev/tcp tcp_max_buf 262144
/usr/sbin/ndd -set /dev/tcp tcp_deferred_ack_interval 100
/usr/sbin/ndd -set /dev/tcp tcp_dupack_fast_retransmit 2
/usr/sbin/ndd -set /dev/tcp tcp_slow_start_initial 4
echo gbit_ndd done
;;
*)
echo "Usage: /etc/init.d/gbit_ndd { start | stop }"
;;
esac
exit 0
In his testing (he sent me the results) he get's full speed gig-e
(100MB/sec) over the wire.
------
Vitaly sent the following tips addressing NFS tuning and TCP tuning:
2. adjust nfs kernel:
- nfs3_nra -> 16~32
- nfs3_bsize -> 65536 and hence you can increase rsize (nfs mount
option) -> 65536
- nfs3_max_transfer_size -> 65536
- clnt_max_conns -> 16~32
- sq_max_size -> 25~100
use 'mdb -kw' to change on the fly
3. adjust tcp parameters:
tcp_max_buf -> 1~4MB
tcp_cwnd_max->1~4M
tcp_xmit_hiwat -> 131072~1M
tcp_recv_hiwat -> 131072~1M
tcp_wscale_always -> 1
use ndd to change
4. use 'nocto', 'noac' option for nfs mount
Dean Ross-Smith
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:42:35 EDT