RE: Spidering

From: Thor (Hammer of God) (thor@hammerofgod.com)
Date: Fri Jan 18 2008 - 14:58:45 EST


This doesn't answer your question, but in regard to the validity of the technique, understand that an easy way to thwart these efforts (as far as IIS is concerned) is to edit your ISAPI extensions and have the ASP.DLL parse your .HTM(L) pages as well and just rename your .asp pages to .htm -- that way they will provide dynamic content the same way, but will have .htm extensions (in IIS7, you'll use a handler mappings) so that people will not (immediately) know they are dynamic pages by extention...

t

> -----Original Message-----
> From: listbounce@securityfocus.com
> [mailto:listbounce@securityfocus.com] On Behalf Of me me
> Sent: Thursday, January 17, 2008 9:59 AM
> To: pen-test@securityfocus.com
> Subject: Spidering
>
> Hi All
>
> One of the most important aspects of assessing a web application is
> separating the dynamic code from the static HTML. I tend to do this
> though
> spidering, and use a number of tools including Paros and a commercial
> tool.
>
> Basically, I like to know:
>
> The names and places of the scripts
> The variables they take
> The Comments etc in HTML
> Hidden form values
>
> All the usual.
>
> Whilst I don't expect it to get everything (JavaScript etc is going to
> take
> manual intervention, so is a number of other possible technologies), I
> have
> never really found a tool that I consider to be the defacto spidering
> tool
> from this perspective. One of the biggest problems is a lot of the
> spiders
> seem to choke on really big sites, or go into infinite loops etc etc.
>
> Has anyone got a reliable spider that they've used time and again (even
> on
> big sites) that they could recommend? Or are most people WGETing the
> site
> and then regex'ing the local mirror?
>
> Thanks in advance
>
> -----------------------------------------------------------------------
> -
> This list is sponsored by: Cenzic
>
> Need to secure your web apps NOW?
> Cenzic finds more, "real" vulnerabilities fast.
> Click to try it, buy it or download a solution FREE today!
>
> http://www.cenzic.com/downloads
> -----------------------------------------------------------------------
> -



This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:58:20 EDT