From: Robert E. Lee (robert@dyadsecurity.com)
Date: Fri Jul 07 2006 - 11:55:54 EDT
On Wed, 5 Jul 2006 09:49:39 -0400
"Security Tester" <pentestrbk@gmail.com> wrote:
> While running my scans (basic TCP port scans and such)
> they claim that their load balancers basically rolled-over and died
> causing Internet outages (inbound and outbound).
> Has anyone else seen this kind of behavior?
I believe their claim. From personal experience, many devices will roll-over during port scans. I have caused very high-end devices with "syn-flood protection" enabled to roll-over with a small fraction of the bandwidth they had available. It usually has to do with how much work the device has to do when dealing with connection state changes.
To safely and efficiently scan these devices we've used a scanner engine than provides a packet per second rate setting (in our case, unicornscan). Start with a low rate of scanning (100 packets per second or so), and work your way up to a higher rate that does not cause the outage on the remote side.
> Are their configuration settings on the Link Proof that would prevent
> this or is this simply a vulnerability with the load balancers?
To the best of my knowledge, this is a vulnerability with any inline device that keeps track of state changes and passes traffic. That is, Load Balancers, Firewalls, IPS, etc.
My suggestion for load-balancer configuration changes would be to disable any extra "security" features. It seems counter intuitive, but we've found that disabling many of the IDS/IPS/Firewall/Protection features on a load-balancer type device greatly increased it's availablity because it consumes much less CPU time.
How many VIP's was the load balancer responsible for (VIP being front-end IP/port to back-end IP/port)? How many IP's/Ports that the load-balancer was balancing were you scanning?
The state table for a load balancer may look something like "PROTO:SRC_IP:SRC_PRT|DST_IP:DST_PRT|NAT_IP:NAT_PRT|STATE". If they have a high number of back-end IP's/Ports or a high number of VIP's, this outage would likely be easier to trigger than it otherwise would be. You may end up recommending that they use fewer VIP's per load-balancer (IE, buy more load-balancers).
> it just seems strange to me that my scans were able to do this considering that they have a couple
> of DS3s of bandwidth coming in.
The resource I'm guessing you exceeded was the devices ability to track state changes, which is usually much lower than the available bandwidth. Remember these devices were tuned to deal with passing normal network traffic which is relatively a lower number of packets with much higher data content. Port scans are not what these devices are tuned for :).
Robert
-- Robert E. Lee Chief Information Officer http://www.dyadsecurity.com phone: (949) 394-2033 fax : (949) 486-6601 email: robert@dyadsecurity.com ------------------------------------------------------------------------------ This List Sponsored by: Cenzic Concerned about Web Application Security? Why not go with the #1 solution - Cenzic, the only one to win the Analyst's Choice Award from eWeek. As attacks through web applications continue to rise, you need to proactively protect your applications from hackers. Cenzic has the most comprehensive solutions to meet your application security penetration testing and vulnerability management needs. You have an option to go with a managed service (Cenzic ClickToSecure) or an enterprise software (Cenzic Hailstorm). Download FREE whitepaper on how a managed service can help you: http://www.cenzic.com/news_events/wpappsec.php And, now for a limited time we can do a FREE audit for you to confirm your results from other product. Contact us at request@cenzic.com for details. ------------------------------------------------------------------------------
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:56:14 EDT