Gigabit ethernet performance (ce/Cassini NIC)

From: Oscar Olsson (spam1@qbranch.se)
Date: Thu Jan 06 2005 - 13:57:59 EST


I have a V440 that acts as a legato networker backup server, running
Solaris 9. It has two ce interfaces, running Sun Trunking 1.3. The system
has recent patches for all (hopefully) software/hardware. The system has 4
1,28GHz Ultrasparc IIIi CPUs. The load is fairly evenly spread over both
NICs.

In short, the CPUs run at 100%, approx 80% kernel and 20% user CPU, when
running backups during peak periods. The throughput at that time is about
900mbit network-wise. When running local backups to all drives, they get
fed a total of 1.5Gbit of data, and the system has CPU to spare, so the
performance problem is likely network-related. I have already read through
various ce and TCP tuning guides for Solaris, and that gave an additional
5-10% performance increase, but that's not nearly enough. Upgrading the
CPUs is pretty expensive, and will probably only yield another 20% or so.

I've read the following posts:

http://www.sunmanagers.org/pipermail/sunmanagers/2003-November/027205.html
http://www.sunmanagers.org/pipermail/sunmanagers/2002-July/015645.html

What I can tell from this is the following:

1. The Cassini NIC has TCP offloading, which I wasn't sure of before.
2. Jumbo frames seem to decrease CPU load for the same throughput on
various systems.

However, running Jumbo frames in my scenario is not an option, since
clients are mostly connected to 10/100mbit interfaces on Cisco switches
that don't support that MTU on that type of interface.

However, I'd still like a bit more information on a few questions:

1. I've been looking at some 3rd-party Solaris SPARC compatible gigabit
NICs. Will any of these two yield better performance when it comes to CPU
useage per megabit? For instance through better drivers, or better
chipsets that have better offloading or are more optimized when it comes
to putting data on the PCI bus?

http://www.sbei.net/Products/LAN/lanPCI-2GC.htm
http://www.syskonnect.com/syskonnect/products/sk-9821v20.htm

2. How CPU-expensive is the use of Sun Trunking v1.3? Either way, since I
already peak at 900Mbit+, it might not be an option to run the system
without it. Although I admit it would be interesting to do that for
testing purposes, in order to see if the load decreases.

And in general, is it supposed to require 4 1,28GHZ UltraSPARC IIIi CPUs
just to drive about a gig of incoming traffic?

Ofcourse, any comments about what could be causing this performance
bottleneck is welcome, it doesn't have to be a specific reply to my
question.

Since the policy list requires me to summarize all answers, reply off-list
and I will do just that if I get some interesting comments. :)

//Oscar
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers



This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:29:57 EDT