Re: Pen-test pricing (long)

From: Volker Tanger (vtlists@wyae.de)
Date: Fri Feb 04 2005 - 19:09:51 EST


Greetings!

On Fri, 04 Feb 2005 12:29:41 +0100
Christoph Puppe <puppe@hisolutions.com> wrote:
>
> most companies charge per day. Only if it is a emergency-response,
> then by the hour.

...because emergency (and -response) usually does not care about
office-hours. Off-hours and overtime are charged extra - and rightfully
so!

> The number of servers, locations, firewalls, DMZs
> and other stuff that is to be tested should help you to calculate a
> number of days, that you will need to do a good job (hrs /system *
> systems / 8) and meet the targets.

This equation first strook me odd - first I read (hrs/systems^2)/8 which
made no sense to me. But now I think you said:

        total_hours = (hrs per average_system) * number of systems
        total days = total_hours / 8
                        
This only works if you have roughly similar systems. For checking
average clerk workstations and standard file+print servers I'm with you.
But I often experienced huge differences between systems when speaking
of firewalls or (exposed) servers.

E.g. within the same company I've seen a tight CheckPoint Firewall on
minimized, current Solaris with ~10 well documented rules, 2 NAT rules
and two interfaces - and another "legagcy" CkeckPoint system on
unpatched WinNT with 6 interfaces, 300 rules and 450 NAT rules (just
wonder, don't ask). Checking the Solaris box was fast - the NT system
took days just to understand it's function within the network.

> PT-Targets the easy way is first to establish what the customer want's
> to protect against:
>
> Class 1 Attacker (governmental or organized crime funded, very
> knowledgable, cappable of impressive stunts)
> Class 2 Attacker (corp. Espionage or knowledgable person with some
> funds) Class 3 Attacker (Skript-Kid, Scanner-Swinging, persons who do
> not target your customer, but just look for low hanging fruits)
>
> For Class 1, multiply the number of days you would need for a good job
> by two. Class 3, divide by 2 ;)

I usually recommend running a risk analysis first - so security needs
and vulnerable systems are identified, giving the customer an idea on
where protection is needed. "Vulnerable" here means "*Ouch*"-factor, not
the "easy-to-hack" one. An isolated webserver with static content
(replacable within some ten minutes) can be handled much more lax than
the single ERP/SAP system - still without even coming near the impact of
a failure or hack of the latter system...

When the systems and their potential (impact gravity) are defined (or
set by the customer), then I recommend continuing with a whitebox
analysis
(i.e. having admin access to the system and checking all settings). This
usually is much faster than a backbox access ("Here's the IP range, find
out the rest for yourself..."), saving the customer expensive time.

The whitebox analysis usually is followed an automated scan ("for free")
just to make sure I have not overseen anything.

Risk analysis can run from a few minutes to weeks, depending on
preparedness of the company - or they simply point to "those systems,
please".

Then there are 1-2 days of defining scope and target (= overall goal,
not victim systems) of the audit. With the risk analysis and a *very*
brief look onto the system specs (what is it, what does it) the
thoroughness of the whitebox analysis can be agreed upon by auditor(s)
and customer on a per-system(-class) basis. An that time is to be spent
on each system digging into the gory details - unless something
"strange" crops up. Then the customer is notified so he can agree on an
analysis extension for that system(-class). Ah yes, nearly forgot: here
"system" can be access to buildings or infrastructure (down to
dumpster-diving), employees (social engineering), too - not just
computers.

All followed by roll-up, documentation (-polishing) and presentation
(1 to a few days).

In contrast a blackbox analysis goes the other way 'round: after a brief
scope definition (IP range or network connection - and hotline number in
case of "oops, it broke") an automated scan follows in parallel to
research to identify vulnerable systems. After that the auditor tries to
break (down or into the) systems found vulnerable or "seeming to be
interesting". Lots of guesswork and a respectable risk of disrupting
something (and if only performance and/or NOC where the IDS is running
wild). The resulting report tries to identify the risky systems - with
respect to breakability.

Compared to a whitebox risk analysis the blackbox results and risk are
usually not worth the ammount of insight won by the days spent mainly to
identify "interesting" systems and tries to break them.

Did I forget to mention that I am heavily biased towards targeted
drilling vs. "aimless" prowling around in the mud? ;-)

With a whitebox approach the customer knows which systems are checked
and how thoroughly - knows what he gets for his money. When he is kept
in the loop, it is transparent to all where extensions to the original
schedule estimations are made. No arguing about "time wasted" on
low-impact systems (old SAP development server where only the printer
server is used - but the ERP with dummy data still was running,
forgotten), no system broken during tests to be furious about, etc.

OTOH a blackbox pen-test can be a much more impressive eye-opener to
non-tech-savy suits. And it usually takes longer, which is good for our
consultant's purse... ;-)

Back to the original question:

So with the whitebox-approach we identify the systems, their importance
and calculate from this the time needed for an analysis. Add preparation
and roll-up to get the number of days the audit will take/be billed.

For a blackbox analysis I usually suggest the approach of "playing" the
attacker the customer wants:
        - target of the attacker (bot install, break/deface, data access)
        - level/skill of attack (from script kiddie to expert)
        - stealthiness of attack, closely correlated to
        - timeframe of attack - hours total and their spread over X days
        - means of attack (network, dumpster-diving, social-engineering)

Advantage of a blackbox attack is that customer AND auditoris that a
customer can order a type of attack for a price X without having to
invest into (brief) consulting first (risk analysis and/or scope
definition). Such a "COTS"-Blackbox-Audit for fixed price usually is
much easier for sales, too.

Some simple example blackbox packages too often found as "Qualified
PenTest" offers - especially when offered in a service package where
check automatically are run weekly/monthly:

"Script-Kiddie's cursory fly-by"
        NMAP-Scan, resulting in interesting/not classification.
        5 Minutes per system plus documentation.

"Script-Kiddie's automated attack"
        Nessus-Scan, resulting in full report and an abstract.
        15 Minutes per system including documentation plus setup.

"Professional criminals targeting bank security"
        See e.g. the "Sneakers" movie, teaser sequence:
        Team working 1-2 weeks on research plus 1 day execution
        plus 2-3 days of documentation.

But of course the official product names differ highly from the ones
listed here. ;-)

Short summary on the original question:

Only bill fully automated scans per system, any human interaction by
hour or day (= 8h during regular working hours).

Mixed calculations are not transparent enough, leaving customers
wondering about wether the price they are paying are fair - and maybe
lead you into a calculation calculation/assumption error.

Good luck!

Volker

PS: Greetings from Berlin to Berlin! ;-)

-- 
Volker Tanger    http://www.wyae.de/volker.tanger/
--------------------------------------------------
vtlists@wyae.de                    PGP Fingerprint
378A 7DA7 4F20 C2F3 5BCC  8340 7424 6122 BB83 B8CB


This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:54:15 EDT