FW: IDS Testing

From: Robert E. Lee (robert@dyadsecurity.com)
Date: Wed Mar 10 2004 - 15:24:36 EST


> Do any of you have any suggestions as to what might be a good
> technique/tool to test the responses of the IDS systems, apart from
> performing the attacks yourself. I am really looking for some sort
of
> way to replay the attack data on the wire, but not actually target any

> machines.

Manual IDS testing with access to the logs in passive environments is
the only thorough way I've found to perform IDS testing. We use the
knowledge of how pattern matching and anomaly matching technology work
and try to find interesting ways of triggering IDS. If the IDS is doing
its job it should identify when you're doing "bad" stuff as defined by
what the IDS admin wants to know about. If you have access to the logs
you can see exactly what triggered. Knowing what didn't trigger is
equally important if you feel your actions should have been interesting
enough to notify the IDS admin.

You can in theory automate some aspects of IDS testing without logs if
the IDS is in an "active" configuration, meaning that it updates acl
rules or send tcp rst's inline. If this is the case, you can map out
things like sensitivity (3 syns to successive ports on the same ip in 30
seconds == lock out for 5 minutes... etc etc).

> I have also used stick and snot in the past, but these get old, and
quite > frankly they really don't test the detection capability of the
sensor.
> They are however great tools for spamming the sensors and slipping in
> below the radar.

Those types of tools are really noisy and not really useful in mapping
out the features/sensitivity. That being said, I also have used stick
and snot... but I do so more as a wake up call to see if the IDS admins
are really watching the logs or not. If I let it generate 10+ minutes
of pattern match EVERYTHING and I don't get a call, I'm suspicious. If
I then have a status update call and they don't mention it I ask probing
questions until I find out how often they actually look at the logs.

> I am currently looking at different methods that can test the
> functionality and response of production IDS sensors.

A good method for testing IDS's manually would be (as quoted from
ISECOM's OSSTMM - www.osstmm.org):

Module 9 . Intrusion Detection System Testing
This test is focused on the performance and sensitivity of an IDS. Much
of this testing cannot be properly achieved without access to the IDS
logs. Some of these tests are also subject to attacker bandwidth, hop
distance, and latency that will affect the outcome of these tests.

Reviewing the server logs is needed to verify the tests performed on the
Internet presence especially in cases where results of the tests are not
immediately visible to the tester. Many unknowns are left to the analyst
who has not reviewed the logs and alerts.

Expected Results:
Type of IDS
Note of IDS performance under heavy load
Type of packets dropped or not scanned by the IDS
Type of protocols dropped or not scanned by the IDS
Note of reaction time and type of the IDS
Note of IDS sensitivity
Rule map of IDS
List of IDS false positives
List of IDS missed alarms
List of unmonitored paths into the network

IDS and features identification
1. Verify the IDS type with information collected from intelligence
gathering.
2. Determine its sphere of protection or influence.
3. Test the IDS for alarm states.
4. Test the signature sensitivity settings over 1 minute, 5 minutes, 60
minutes, and 24 hours.

Testing IDS configuration
5. Test the IDS for configured reactions to multiple, varied attacks
(flood and swarm).
6. Test the IDS for configured reactions to obfuscated URLs and
obfuscated exploit payloads.
7. Test the IDS for configured reactions to speed adjustments in packet
sending.
8. Test the IDS for configured reactions to random speed adjustments
during an attack.
9. Test the IDS for configured reactions to random protocol adjustments
during an attack.
10. Test the IDS for configured reactions to random source adjustments
during an attack.
11. Test the IDS for configured reactions to source port adjustments.
12. Test the IDS for the ability to handle fragmented packets.
13. Test the IDS for the ability to handle specific system method
attacks.
14. Test the effect and reactions of the IDS against a single IP address
versus various addresses.

Reviewing IDS logs and alerts
15. Match IDS alerts to vulnerability scans.
16. Match IDS alerts to password cracking.
17. Match IDS alerts to trusted system tests.

While any sane person would automate as much as possible, I still
strongly believe that artificial intelligence is artificial. If your
organization has the need to have a thorough test of the IDS, I would
strongly consider a manual effort over any product/tool alone.

Cheers,

Robert

Robert E. Lee
CTO, Dyad Security
(800) 644-DYAD
http://www.dyadsecurity.com

---------------------------------------------------------------------------
Ethical Hacking at the InfoSec Institute. Mention this ad and get $545 off
any course! All of our class sizes are guaranteed to be 10 students or less
to facilitate one-on-one interaction with one of our expert instructors.
Attend a course taught by an expert instructor with years of in-the-field
pen testing experience in our state of the art hacking lab. Master the skills
of an Ethical Hacker to better assess the security of your organization.
Visit us at:
http://www.infosecinstitute.com/courses/ethical_hacking_training.html
----------------------------------------------------------------------------



This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:53:50 EDT