From: Omar A. Herrera (omar.herrera@oissg.org)
Date: Sun May 21 2006 - 17:33:47 EDT
Hi, my answers below...
> -----Original Message-----
> From: mr.nasty@ix.netcom.com [mailto:mr.nasty@ix.netcom.com]
>
> Ivan Arce is correct.
>
>
> "The original author (Mr. Nasty) equated defining the scope of a
> penetration test to committing (or attempting to commit) fraud on the
> basis that if you define a precise scope then you are purposely leaving
> out things that may be important to the general public (I am assuming that
> he intended to apply that rational to government,public service
> organization and public companies).
>
>
> So you are talking about a different thing: Fraud (or is it phraud?)
> ommitted by the penetration tester because she exceed the scope of what
> she
>
> was allowed to do, whereas Mr. Nasty proposed that having a scope defined
> by the organization subject to the test is somehow equivalent to fraud (if
> the
>
> results of the test are not made public)"
>
>
> The only rational that I can see from what Ivan's written is that he has
> been there. Most others have not. That's why there is a complete
> disconnect between logic and reason.
So, should we assume that by recognizing that Ivan has been "there" you are
at least taking his comments more seriously? (I would, he is a well
respected, well known, information security professional with a lot of
experience).
If that's the case, what is your opinion on these comments that were also
posted by Ivan in the same email that contained the comments that you quoted
above? :
" I submit that scope definition prior to a penetration test is a good thing
because it syncs both the tester and the testee on what is considered
important, valid, desired, etc., ant helps to plan resource allocation
accordingly and to understand and align expectations.
BTW you can still define the scope as: "Anything goes, no restrictions
whatsoever" but then you would be letting the penetration tester do whatever
she feels like doing and unless both parties have a good and long standing
relationship it becomes harder for both to assess the costs and the value of
the work. "
Do you agree or disagree?
> Since I receive information on specific audit requirements here is the
> most recent from ISACA;
>
> The Standards Board has issued the following IS Auditing Standards, which
> become effective for IS audits commencing after 1 July 2006:
>
> . S12 Audit Materiality
>
> . S13 Using the Work of Other Experts *****
>
> . S14 Audit Evidence
>
>
> My concerns with ROE's are defined within S13. Any big 4 or maybe big 3
> now, manager should know this. Audit Managers are brought to the back room
> by the CFO or CEO presented a pentest within the past 12 months that
> covered dialup issues. The Everyone smiles and the Audit Manager is lead
> out of the room with the cover letter stating that the pen-test performed
> was in conformance with all ROE. The Audit Manager, knowing he has to cut
> costs or it's coming out of his budget, will accept the pen-test as
> support and reduce the confidence sample.
Yes, ISACA sends notice on drafts to its members and I also got that one.
No, I'm not an auditor anymore and I do "suffer" audits as well as a
security officer, just like you, so don't think of yourself as a unique,
lucky human being just because you get information from ISACA.
Yes, any audit manager should be aware of this stuff, even if they don't
work for a big firm. What's the big deal?
Yes, these standards have not yet become effective.
Yes, many of us are concerned about how vague some of these drafts are and
that's why we send comments to ISACA on the drafts before they are approved
and sometimes even after. And we are all aware of the Disclaimer section
contained in each of these documents, specifically the part talking about
"minimum level of acceptable performance". Aren't we?
And yes, things like the one you tell us do happen. So what? Do you believe
that even with the vague definitions in S13 this action would comply with
S13? Is there evidence this would pass the criteria specified by point 06 of
this standard (and I make special emphasis on the requirement that this
external work must be considered "complete").
The fact that a certain auditor is potentially being negligent is not an
excuse for asking that every auditor out there refuses to accept any pentest
or any other security assessment with a precise scope.
So, according to Ivan's clarification that you quoted, and since you seem to
have agreed with that, let me ask you:
1) What proof do you have that by removing any precise scope (i.e. allowing
the pentester to test all an everything) will guarantee that all "things
that may be important to the general public will be covered"? (Read Ivan's
comments on a loose scope).
2) What proof do you have that a complete and detailed pentest (as detailed
and complete as you can imagine) can be completed within a reasonable time
for any organization? (Don't forget multinational corporations that have
offices and systems in most countries of the world and institutions owning
class B ip address ranges, those get audited too). Well at least I would
expect them to be completed within a year.
3) Have you ever done or even attempted such and extremely detailed and
complete pentest yourself with a huge corporation? This is what you are
asking for, isn't it?
4) Wouldn't a very light (i.e. non-detailed) yet broad pentest without any
precise scope, covering every single thing that you, the stakeholders the
general public and every human being on this planet might think of being
important in relation to a big organization, fit your requirement to avoid
fraud under this definition?
After all, scope does not only restrict the things checked, but also how
detailed are the tests on these things required to be.
If there is no scope, how would you compare if the level of detail was
adequate if there is no metric? Wouldn't that metric/threshold be considered
as a "precise" scope as well? (Remember the worm in the apple that you
mentioned...).
Regards,
Omar Herrera
------------------------------------------------------------------------------
This List Sponsored by: Cenzic
Concerned about Web Application Security?
Why not go with the #1 solution - Cenzic, the only one to win the Analyst's
Choice Award from eWeek. As attacks through web applications continue to rise,
you need to proactively protect your applications from hackers. Cenzic has the
most comprehensive solutions to meet your application security penetration
testing and vulnerability management needs. You have an option to go with a
managed service (Cenzic ClickToSecure) or an enterprise software
(Cenzic Hailstorm). Download FREE whitepaper on how a managed service can
help you: http://www.cenzic.com/news_events/wpappsec.php
And, now for a limited time we can do a FREE audit for you to confirm your
results from other product. Contact us at request@cenzic.com for details.
------------------------------------------------------------------------------
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:55:59 EDT