RE: Determining the encryption used

From: Sahir Hidayatullah (sahirh@mielesecurity.com)
Date: Fri May 12 2006 - 02:42:46 EDT


If you have access to the application that is doing the encryption (not
necessarily on the target system, maybe you can download a demo), you can
try a chosen plaintext attack. One way that I've had success with when
attacking weak poly-alphabetic ciphers is to get the program to encrypt a
Vigenère table like so:

ABCDEFGHIJKLMNOPQRSTUVWXYZ
BCDEFGHIJKLMNOPQRSTUVWXYZA
CDEFGHIJKLMNOPQRSTUVWXYZAB
DEFGHIJKLMNOPQRSTUVWXYZABC
EFGHIJKLMNOPQRSTUVWXYZABCD
FGHIJKLMNOPQRSTUVWXYZABCDE
GHIJKLMNOPQRSTUVWXYZABCDEF
HIJKLMNOPQRSTUVWXYZABCDEFG
IJKLMNOPQRSTUVWXYZABCDEFGH
...
...
...
TUVWXYZABCDEFGHIJKLMNOPQRS
UVWXYZABCDEFGHIJKLMNOPQRST
VWXYZABCDEFGHIJKLMNOPQRSTU
WXYZABCDEFGHIJKLMNOPQRSTUV
XYZABCDEFGHIJKLMNOPQRSTUVW
YZABCDEFGHIJKLMNOPQRSTUVWX
ZABCDEFGHIJKLMNOPQRSTUVWXY

Now when you look at the ciphertext, you might see distinct patterns that
give way to decrypting the text. Alternatively, if you know some of the data
that has been encrypted (say a company name, or an order category), then you
can mount known-plaintext attacks.

You have some other options as well:

1. Try and determine if it's not a salted hash by encrypting known plaintext
and comparing it to the ciphertext (for example, encrypt ABC, then generate
ABC's ciphertext with known algorithms like MD5, SHA-1 etc and compare the
two ciphertexts. If it works (wow lucky), then you can just run these
through a standard cracker.

2. Reverse engineer the binaries - obviously this is time / skill /
motivation based, but you'll be grinning if you find yourself something
stupid hardcoded in the strings!

3. As you said, counting the length and character set of the ciphertext
might be a decent clue, for example, 32 hexadecimal characters is likely to
be MD5, if you get variable length strings of all uppercase characters,
you're probably dealing with something home-made. If the character set goes
into the non-ascii range, its might be a XOR routine without a modulus
operation etc.

4. Throw the ciphertext through a number of conversion routines (yes, ROT13
as well ;)) and see if you get any plaintext back. To save you time coding
this, I remember there was a tool called napkin that did multiple
conversions.

5. If it's a COTS application, you might just want to look up the product /
contact the vendor. They will either tell you the algorithm (if it's well
known) or they'll tell you it's proprietary (at which point your eyes should
light up, because it's much more likely to be breakable with a little work).

Obviously you can't post the original sample data, but maybe if you encrypt
a few of your own records, you could give those up to a crypto list and see
what they have to say. You might also want to have a quick read:
http://en.wikipedia.org/wiki/Topics_in_cryptography
To see what you might be up against.

I'll make way for the mathematicians now.

Cheers,

Sahir Hidayatullah.

-----Original Message-----
From: John Madden [mailto:chiwawa999@yahoo.com]
Sent: Thursday, May 11, 2006 10:50 PM
To: pen-test@securityfocus.com
Subject: Determining the encryption used

Hi,

While doing a pen-test I came across a database with
encrypted fields and I was curious to try and see what
I would do with it.

Is it possible to determine the encryption used by
"looking" at the encrypted results or lenght ?

I know that with Base64 it's pretty easy because of
the "==" at the end.

I would like to learn more about the subject and
considering the amount of documentation out there, I
would like your comments on what you have used and
found usefull.

Also if there are any tools besides openssl that you
found usefull to try all the cipher, consecutively
with a passphrase/keyword against a file/string

Thank you for your time.

------------------------------------------------------------------------------
This List Sponsored by: Cenzic

Concerned about Web Application Security?
Why not go with the #1 solution - Cenzic, the only one to win the Analyst's
Choice Award from eWeek. As attacks through web applications continue to rise,
you need to proactively protect your applications from hackers. Cenzic has the
most comprehensive solutions to meet your application security penetration
testing and vulnerability management needs. You have an option to go with a
managed service (Cenzic ClickToSecure) or an enterprise software
(Cenzic Hailstorm). Download FREE whitepaper on how a managed service can
help you: http://www.cenzic.com/news_events/wpappsec.php
And, now for a limited time we can do a FREE audit for you to confirm your
results from other product. Contact us at request@cenzic.com for details.
------------------------------------------------------------------------------



This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:55:56 EDT