From: James Fields (jvfields@tds.net)
Date: Fri Jan 23 2004 - 17:20:31 EST
Agreed. My company has purchased testing from an extremely well-known and
respected firm. The tests are minimally useful in pointing out an
occassional flaw or a missed configuration step on a web server. However,
the reports are not detailed at all, and fail to give an accounting of
everything that was tested. This is important to me and my company - we pay
a LOT of money for these tests, and not knowing what was tested leaves us
blind as to what we're really paying for. It certainly isn't a trophy hunt.
I personally need to know all that was tried, and when, so that I can
compare the test activity to my firewall and IDS logs and see if I logged
all that I should have, or if I interpreted correctly what was being done
during the test. It's one thing I really like about the OSSTMM - you have
to lay it all out, whether you successfully exploit anything or not.
Full-disclosure is good for the tester...
----- Original Message -----
From: "Pete Herzog" <pete@isecom.org>
To: "Jeff Shawgo" <jeff.shawgo@verizon.net>; <pen-test@securityfocus.com>
Sent: Friday, January 23, 2004 3:32 PM
Subject: What a security test should do?- from thinking about: Ethical
Hacking Training
> What does a pen test fail to provide?
>
> I had to think about this for a little while because it's not so much to
me
> what someone needs to know to be a security manager, CISO, or security
> consultant, but rather what do we expect from a security test?
>
> I know what pen-tests have been used for but I think a lot of that is also
> under-analyzing the results of a pen-tset. As an auditor of pen-test
> reports for some companies, I see many of these reports focusing on
software
> vulnerabilities, the occassional rooting of boxes, and the holy trilogy of
> web app hacks (XSS, Command Injection, Buffer Overflows). Most reports
will
> have a traceroute to each host in the network but not even say why or what
> that is useful for. So in the end these reports leave a lot of analysis
up
> to the client and if they are not capable of this kind of analysis, the
> report has much less worth.
>
> I have felt that security tests should do more. They should test
> configurations and policies as well. A test may tell you, for example,
> about patch management, which department influences the company's Internet
> presence, and if the firewall admin has top-level support or a policy to
> follow regarding opening new ports. All of these things may negatively
> influence the strength of network security in ways that make it just as
> vulnerable as a remote service exploit.
>
> As Jeff mentions here, there is a lot more to network security than
> pen-testing but for the most part, testing should be also able to verify
> when the foundation is rotten.
>
> So my question is, what parts of security can't be verified in a security
> test? No flames please-- I'm just trying to make the OSSTMM (osstmm.org)
> better.
>
> Sincerely,
> -pete.
>
> Pete Herzog, Managing Director
> Institute for Security and Open Methodologies
> www.isecom.org - www.osstmm.org
> www.hackerhighschool.org - www.isestorm.org
>
>
> > -----Original Message-----
> > From: Jeff Shawgo [mailto:jeff.shawgo@verizon.net]
> > Sent: Tuesday, January 20, 2004 18:46 PM
> > To: pen-test@securityfocus.com
> > Subject: Re: Ethical Hacking Training
> >
> > On the other hand, most people also forget that knowing how to
> > perform a pen-test or exploit is only one very very tiny aspect
> > of security. The organization that has a solid policy,
> > coordinated antivirus, well-managed firewalls, patch management
> > policy, e-mail and web filtering, code review, and basic system
> > hardening is likely to be many times more secure than the
> > organization that focuses on *any* one individual's skill as a
> > pen-tester.
> >
> > If the security foundation is rotten, it does little good to
> > point out that the windows are unlocked.
> >
> > Pen-testing is important, but the basics need to be there first.
> > That's the message most people are missing - probably because
> > it's not as attractive.
> >
> > ~Jeff
> >
> > ------------------------------------------------------------------
> > ---------
> > ------------------------------------------------------------------
> > ----------
> >
> >
> >
>
>
> --------------------------------------------------------------------------
-
> --------------------------------------------------------------------------
-- > --------------------------------------------------------------------------- ----------------------------------------------------------------------------
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:53:46 EDT