From: Craig Wright (cwright@bdosyd.com.au)
Date: Thu Aug 10 2006 - 01:40:47 EDT
True, similar to an inspection that only covers the past an audit is an
evaluation of the present. However as you have noted a present and
existing policy is only valid for the present and may have no
correlation to a future state.
The problem is that we have no way to determine what will occur in the
future and thus have to correlate the past and existing results to give
a good guess at how things will progress in the future.
Even audit processes [such as a SAS 70 (part 2)] that require an ongoing
evaluation over time [rather than a point in time model] only give an
estimate and at best, we can derive a probabilistic risk model.
We can hope. Sometimes people do what is right and use an audit as a
means to improve, other it is just an exercise in futility to tick a
compliance box...
Regards,
Craig
-----Original Message-----
From: David M. Zendzian [mailto:dmz@dmzs.com]
Sent: Thursday, 10 August 2006 10:12 AM
To: Craig Wright; stylewar@cox.net; Christine Kronberg; Arkem Paul
Cc: pen-test@securityfocus.com
Subject: RE: Vulnerability Assessment vs. PenTest
One thought on this, btw great writeup.
An audit makes no assumptions about the future. It is a view at the
moment of the audit which can and will most likeley change the moment
after the auditor leaves.
David
-----Original Message-----
From: "Craig Wright" <cwright@bdosyd.com.au>
To: stylewar@cox.net; "Christine Kronberg" <seeker@shalla.de>; "Arkem
Paul" <bob@mornmist.com>
Cc: pen-test@securityfocus.com
Sent: 8/8/06 9:35 PM
Subject: RE: Vulnerability Assessment vs. PenTest
To add a little to this debate.
First, there are two types of Audit, internal and external. An audit,
consisting of an evaluation of an organisation's systems processes and
controls, is performed against the set standard or documented process.
Audits are designed to provide an independent assessment through a
qualified independent assessment of representations about the system or
process. An audit may also provide a gap analysis of the operating
effectiveness of the internal controls.
An audit differs from an inspection in that an audit makes
representations about likely future results. An inspection evaluates
past results. Or an audit to be valid it must be conducted according to
accepted principles. In this, the audit team and individual auditors
must be certified and qualified for the engagement. Numerous "audits"
are provided without certification, these however are qualified reviews.
A penetration test is an attempt to bypass controls and gain access to a
one system. The goal of the penetration test is to prove the that the
system may be compromised. A penetration test does not assess the
relative control strength nor the system or processes deployed, rather,
it is a "red teaming" styled exercise designed to prove illicit access.
The real strength of a penetration test is marketing the need to improve
controls to internal management. A penetration test is of limited value
in the greater scheme of a systems information security due to the
restricted nature of the test and the lack of inclusion of many key
controls.
A vulnerability assessment is an assessment and gap analysis of a site's
or a system's control strengths. A vulnerability assessment is a risk
based process. The process involves the identification and
classification of the primary vulnerabilities which may impact the
system. Often, methodologies such as fault tree analysis end cause
consequence analysis are employed in this review.
Both vulnerability assessments and penetration tests may be conducted as
a white box or black box analysis. A black box analysis is instigated
with little or no knowledge of the system being tested. A white box
analysis is conducted for knowledge of the system.
A vulnerability assessment is a critical component of any threat risk
assessment. Following the vulnerability assessment and impact analysis
is conducted and used in conjunction with a threat report to provide for
an estimation of the organisation's risk to selected attack vectors.
External audits are conducted (or at least should be) by independent
parties no rights or ability to alter or update the system. Internal
audits involve a feedback process where the auditor may not only audit
the system but also potentially provide advice in a limited fashion. And
external auditor is precluded from advising their client. They are
limited to reporting any control gaps and leading the client to a source
of accepted principles.
The common perception that running an automated scanner such as Nessus
or one of its commercial cohorts is in itself a vulnerability or
penetration test is false.
Most of the so-called penetration tests that are provided are no more
than a system scan using tools. A penetration test it correctly provided
will attempt the use of various methodologies to bypass controls. In
some instances this may involve the creation of new or novel
scripts/programs.
The issue is not that many people commonly use the words interchangeably
but that so-called professionals fail to differentiate the terms. Of
particular concern is the use of audit and the designation auditor. This
is as these terms are often restricted in code. This is that most
jurisdictions have statutory requirements surrounding their use and
application.
Information security systems provide many of the functions that
construct a control system. Of particular concern are controls that
limit access to accounting and financial records. This includes records
held by systems that provide an e-commerce transaction path. In many
jurisdictions it is an offence to sign off an audit report when you are
not a certified auditor. Traditionally the path around this has been not
to call the process of testing the system and audit, but rather to call
it an agreed procedures review.
An agreed procedures review or simply a review is an analysis of
controls performed against an agreed process.
Some example's of an audit include SAS 70 (part 1 or 2) audits, ISO
9001,17799:2/27001 certification audits, HIPPA audits. There are many
different types of audits and many standards that an audit may be
applied against.
There are various processes and procedures used to provide vulnerability
assessments and threat risk analysis. Standards such as AS/NZS 4360:2006
I commonly mandated by government organisations.
Penetration testing, if done correctly, may provide some value in its
free-form approach. When correctly implemented, a penetration test adds
a level of uncertainty to the testing. The benefit of this uncertainty
is that it might uncover potential flaws in the system or controls that
had not been taken into account when designing the control system. To be
of value, a penetration test must needs to do more than scan a system.
It needs to do something novel and unexpected.
There is little similarity between a penetration test, vulnerability
assessment, risk assessment or audit. The lack of understanding of these
differences impedes the implementation of effective security controls.
Bus to finish, 'Stylewar' is correct in stating that "an audit must
follow a rigorous program...". Christine's appraisal of a vulnerability
assessment would more correctly be termed as a controls assessment. A
controls assessment may also be known as a security controls review.
As for the need to develop a structured taxonomy (naming system), there
is already one in existence. None of these terms or services is new. All
these services have been provided for as long as computers have been
used by business and government. They were definitely employed as far
back as the 70s.
Regards,
Craig
-----Original Message-----
From: StyleWar [mailto:stylewar@cox.net] Sent: Wednesday, 9 August 2006
3:19 AM
To: 'Christine Kronberg'; 'Arkem Paul'
Cc: pen-test@securityfocus.com
Subject: RE: Vulnerability Assessment vs. PenTest
Point of fact that an audit must follow a rigorous program, and has a
set of
documentation and traceability requirements with it that an 'assessment'
does not. They are 'approximate' in the hands of a well disciplined
assessment team - but I would stop a hair short of calling them equal..
-
StyleWar
"Ancora Imparo"
> -----Original Message-----
> From: Christine Kronberg [mailto:seeker@shalla.de] > Sent: Sunday,
August 06, 2006 11:54 AM
> To: Arkem Paul
> Cc: pen-test@securityfocus.com
> Subject: Re: Vulnerability Assessment vs. PenTest
> > On Sun, 6 Aug 2006, Arkem Paul wrote:
> > >
> > A Vulnerability Assessment should be a comprehensive look > from
policy > > and procedures to implementation of security in the network >
and should > > include such things as patch management, virus
protection, user > > education, SOE hardening, infrastructure
configuration, etc.
> > So basicly an assessment is equal to an audit? The > description
above
> is what I usually expect from someone doing an audit.
> A vulnerability assessment I tend to understand in terms > of
investigating
> a specific application (in far more detail than a > penetration
test).
> > There are a couple of term mixed every now and again (like
someone
> else just stated: funny that we professionals don't come > up with
_one_
> definition):
> > Audit
> Security Scan
> Security Assessment
> Vulnerability Assessment
> Penetration Test
> > Did I miss one?
> > Cheers,
> > Christine Kronberg.
> >
Liability limited by a scheme approved under Professional Standards
Legislation in respect of matters arising within those States and
Territories of Australia where such legislation exists.
DISCLAIMER
The information contained in this email and any attachments is
confidential If you are not the intended recipient, you must not use or
disclose the information. If you have received this email in error,
please inform us promptly by reply email or by telephoning +61 2 9286
5555. Please delete the email and destroy any printed copy.
Any views expressed in this message are those of the individual sender.
You may not rely on this message as advice unless it has been
electronically signed by a Partner of BDO or it is subsequently
confirmed by letter or fax signed by a Partner of BDO.
BDO accepts no liability for any damage caused by this email or its
attachments due to viruses, interference, interception, corruption or
unauthorised access.
------------------------------------------------------------------------
------
This List Sponsored by: Cenzic
Concerned about Web Application Security?
Why not go with the #1 solution - Cenzic, the only one to win the
Analyst's
Choice Award from eWeek. As attacks through web applications continue to
rise,
you need to proactively protect your applications from hackers. Cenzic
has the
most comprehensive solutions to meet your application security
penetration
testing and vulnerability management needs. You have an option to go
with a
managed service (Cenzic ClickToSecure) or an enterprise software
(Cenzic Hailstorm). Download FREE whitepaper on how a managed service
can
help you: http://www.cenzic.com/news_events/wpappsec.php
And, now for a limited time we can do a FREE audit for you to confirm
your
results from other product. Contact us at request@cenzic.com for
details.
------------------------------------------------------------------------
------
------------------------------------------------------------------------
------
This List Sponsored by: Cenzic
Concerned about Web Application Security?
Why not go with the #1 solution - Cenzic, the only one to win the
Analyst's
Choice Award from eWeek. As attacks through web applications continue to
rise,
you need to proactively protect your applications from hackers. Cenzic
has the
most comprehensive solutions to meet your application security
penetration
testing and vulnerability management needs. You have an option to go
with a
managed service (Cenzic ClickToSecure) or an enterprise software
(Cenzic Hailstorm). Download FREE whitepaper on how a managed service
can
help you: http://www.cenzic.com/news_events/wpappsec.php
And, now for a limited time we can do a FREE audit for you to confirm
your
results from other product. Contact us at request@cenzic.com for
details.
------------------------------------------------------------------------
------
Liability limited by a scheme approved under Professional Standards Legislation in respect of matters arising within those States and Territories of Australia where such legislation exists.
DISCLAIMER
The information contained in this email and any attachments is confidential. If you are not the intended recipient, you must not use or disclose the information. If you have received this email in error, please inform us promptly by reply email or by telephoning +61 2 9286 5555. Please delete the email and destroy any printed copy.
Any views expressed in this message are those of the individual sender. You may not rely on this message as advice unless it has been electronically signed by a Partner of BDO or it is subsequently confirmed by letter or fax signed by a Partner of BDO.
BDO accepts no liability for any damage caused by this email or its attachments due to viruses, interference, interception, corruption or unauthorised access.
------------------------------------------------------------------------------
This List Sponsored by: Cenzic
Concerned about Web Application Security?
Why not go with the #1 solution - Cenzic, the only one to win the Analyst's
Choice Award from eWeek. As attacks through web applications continue to rise,
you need to proactively protect your applications from hackers. Cenzic has the
most comprehensive solutions to meet your application security penetration
testing and vulnerability management needs. You have an option to go with a
managed service (Cenzic ClickToSecure) or an enterprise software
(Cenzic Hailstorm). Download FREE whitepaper on how a managed service can
help you: http://www.cenzic.com/news_events/wpappsec.php
And, now for a limited time we can do a FREE audit for you to confirm your
results from other product. Contact us at request@cenzic.com for details.
------------------------------------------------------------------------------
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:56:38 EDT