From bram at GAWTH.COM  Thu May  2 20:11:11 2002
From: bram at GAWTH.COM (Bram Cohen)
Date: Thu, 2 May 2002 11:11:11 -0700
Subject: [PYTHON-CRYPTO] New release of BitTorrent,
              with very crypto-laden protocol, is out
Message-ID: <Pine.LNX.4.21.0205021108580.26711-100000@ultra.gawth.com>

I recently pushed out a new release of BitTorrent, which now has a very
secure protocol. The tracker can't get downloaders confused about what
they're downloading, and an attacker can't connect to a downloader and
find out what they're downloading without being able to download the
things themselves.

You can get it here -

http://bitconjurer.org/BitTorrent/download.html

-Bram Cohen

"Markets can remain irrational longer than you can remain solvent"
                                        -- John Maynard Keynes




From thenault at nerim.net  Sat May  4 14:30:55 2002
From: thenault at nerim.net (Sylvain =?iso-8859-1?Q?Th=E9nault?=)
Date: Sat, 4 May 2002 14:30:55 +0200
Subject: [PYTHON-CRYPTO] python cryptography toolkit
In-Reply-To: <000a01c1f364$31cbbf00$3fc13318@pbc.adelphia.net>
References: <000a01c1f364$31cbbf00$3fc13318@pbc.adelphia.net>
Message-ID: <20020504123055.GB1384@ntw23.fr>

On Saturday 04 May ? 08:06, Burt Leavenworth wrote:
> When I run: python setup.py build
> I get:
>
> running build_ext
> building 'Crypto.Hash.MD2' extension
> ...
> error: command 'cl.exe' failed: No such file or directory
>
> Any idea why this doesn't work?

I'm a newbie with the pct, but I guess you don't have cl.exe (which is
needed to build C extensions) installed or in your PATH.

Try to ask your question to PYTHON-CRYPTO at NIC.SURFNET.NL or c.l.py to
have an answer from someone more involved in the pct.

> Burt Leavenworth

cheers

--
Sylvain Th?nault




From rien at yeepa.org  Sat May  4 16:01:44 2002
From: rien at yeepa.org (Adrien Plisson)
Date: Sat, 4 May 2002 16:01:44 +0200
Subject: [PYTHON-CRYPTO] python cryptography toolkit
References: <000a01c1f364$31cbbf00$3fc13318@pbc.adelphia.net> 
            <20020504123055.GB1384@ntw23.fr>
Message-ID: <000701c1f374$39f68e20$2cee4cd4@RANDOM>

From: "Sylvain Th?nault" <thenault at NERIM.NET>
> On Saturday 04 May ? 08:06, Burt Leavenworth wrote:
> > When I run: python setup.py build
> > I get:
> >
> > running build_ext
> > building 'Crypto.Hash.MD2' extension
> > ...
> > error: command 'cl.exe' failed: No such file or directory
> >
> > Any idea why this doesn't work?
>
> I'm a newbie with the pct, but I guess you don't have cl.exe (which is
> needed to build C extensions) installed or in your PATH.

cl.exe is just the "Microsoft (R) 32-bit C/C++ Optimizing Compiler Version
12.00.8168 for 80x86" (as said when typing cl.exe /?)
it's a compiler though... it could be helpful to build some program...

- try opening a MS-DOS command prompt and type cl.exe.
if it complains about not finding cl.exe, then it is not on your path.
- try finding it on your computer.
if you can't find it on your computer, it means you just don't have a
microsoft compiler installed...
else add the patyh to cl.exe in your PATH environment variable. btw, you can
do it in at the command prompt, but it will be only for this console until
you close it.

if cl.exe is on your computer and in the PATH, then try sending a more
complete log of nything that happens when you try setup.py.

regards,

--
rien




From phr-pycrypt at nightsong.com  Tue May 21 02:55:53 2002
From: phr-pycrypt at nightsong.com (Paul Rubin)
Date: Tue, 21 May 2002 00:55:53 -0000
Subject: [PYTHON-CRYPTO] AES CFB mode
Message-ID: <20020521005553.20107.qmail@brouhaha.com>

    Forgive my previous pedantic question: I just didn't realize how
    CFB worked.  I assume the slowdown is due to the fact that in
    8-bit CFB, an entire BLOCK_SIZE is encrypted for each byte of
    output. I am thinking of reducing BLOCK_SIZE for my purposes ( CFB
    only ). Any recommendations?

If you change the block size, it's not AES any more.  You can change
the feedback size--for example you can set it to be the same as
the block size, so you'll get one encryption per block, if your
application can work that way.  If it can, though, you probably
don't need CFB mode in the first place.




From bryan at EEVOLVED.COM  Mon May 20 21:20:07 2002
From: bryan at EEVOLVED.COM (Bryan Mongeau)
Date: Mon, 20 May 2002 15:20:07 -0400
Subject: [PYTHON-CRYPTO] Pycrypto AES CFB implementation
Message-ID: <PYTHON-CRYPTO%2002052021225840@NIC.SURFNET.NL>

Hello,

I've been playing around with the Pycrypto AES implementation and am quite
impressed. One question however, regarding CFB mode: how come it's so slow?
It's 12 times slower than any of the other modes according to by benchmarks.
Looking at the source, I can see that the implementation is 8-bit CFB but
still, according to AC, p.209, CFB's speed should be the same as the block
cipher. What's going on?

Thanks.
--
Bryan Mongeau
eEvolved Inc. - IT Consulting & Custom Software
http://eevolved.com/
--
"The secret to creativity is knowing how to hide your sources."-- Einstein




From bryan at EEVOLVED.COM  Tue May 21 02:48:39 2002
From: bryan at EEVOLVED.COM (Bryan Mongeau)
Date: Mon, 20 May 2002 20:48:39 -0400
Subject: [PYTHON-CRYPTO] Pycrypto AES CFB implementation
Message-ID: <PYTHON-CRYPTO%2002052102512486@NIC.SURFNET.NL>

Forgive my previous pedantic question: I just didn't realize how CFB worked.
I assume the slowdown is due to the fact that in 8-bit CFB, an entire
BLOCK_SIZE is encrypted for each byte of output. I am thinking of reducing
BLOCK_SIZE for my purposes ( CFB only ). Any recommendations?
--
Bryan Mongeau
eEvolved Inc. - IT Consulting & Custom Software
http://eevolved.com/
--
"God does not care about our mathematical difficulties. He integrates
empirically."-- Einstein




From jason-list-python-crypto at MASTALER.COM  Wed May 29 00:10:07 2002
From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler)
Date: Wed, 29 May 2002 02:40:07 +0430
Subject: [PYTHON-CRYPTO] high-performance HMAC module
Message-ID: <1022623807.40170.TMDA@nightshade.la.mastaler.com>

I'm currently using a pure Python HMAC implementation in one of my
applications based on the one in Paul Rubin's p2.py[1].

This application does lots of HMAC manipulation, and I wonder if it
would be worth writing an HMAC module in C.  On the other hand, I
wonder how much this would really speed things up since the underlying
hash function modules (sha, md5) are already written in C.

Any ideas?

Footnotes:
1.  http://www.nightsong.com/phr/crypto/p2.py

--
(http://tmda.sourceforge.net/)




From PaulLambert at WOODSIDENET.COM  Wed May 29 21:42:08 2002
From: PaulLambert at WOODSIDENET.COM (Paul Lambert)
Date: Wed, 29 May 2002 12:42:08 -0700
Subject: [PYTHON-CRYPTO] high-performance HMAC module
Message-ID: <3FFBC907DD03A34CA4410C5C745DEB126AD209@wnimail.WoodsideNet.Com>

> would be worth writing an HMAC module in C.

No, not for this purpose.  It would be better to move to a real encryption algorithm (like AES) written in C than to use p2.


Paul

> -----Original Message-----
> From: Jason R. Mastaler [mailto:jason-list-python-crypto at MASTALER.COM]
> Sent: Tuesday, May 28, 2002 3:10 PM
> To: PYTHON-CRYPTO at NIC.SURFNET.NL
> Subject: high-performance HMAC module
> 
> 
> I'm currently using a pure Python HMAC implementation in one of my
> applications based on the one in Paul Rubin's p2.py[1].
> 
> This application does lots of HMAC manipulation, and I wonder if it
> would be worth writing an HMAC module in C.  On the other hand, I
> wonder how much this would really speed things up since the underlying
> hash function modules (sha, md5) are already written in C.
> 
> Any ideas?
> 
> Footnotes:
> 1.  http://www.nightsong.com/phr/crypto/p2.py
> 
> --
> (http://tmda.sourceforge.net/)
> 




From jason-list-python-crypto at MASTALER.COM  Wed May 29 22:02:26 2002
From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler)
Date: Wed, 29 May 2002 23:02:26 +0300
Subject: [PYTHON-CRYPTO] high-performance HMAC module
In-Reply-To: <3FFBC907DD03A34CA4410C5C745DEB126AD209@wnimail.WoodsideNet.Com>
              (Paul Lambert's message of "Wed, 29 May 2002 12:42:08 -0700")
References: <3FFBC907DD03A34CA4410C5C745DEB126AD209@wnimail.WoodsideNet.Com>
Message-ID: <1022702546.42563.TMDA@nightshade.la.mastaler.com>

Paul Lambert <PaulLambert at WOODSIDENET.COM> writes:

>> would be worth writing an HMAC module in C.
>
> No, not for this purpose.  It would be better to move to a real
> encryption algorithm (like AES) written in C than to use p2.

I think you misunderstood my question.  I'm not using p2.  But p2
contains a pure-python HMAC module similar to the one that I'm using
(which I've attached below).

What I'm asking is whether I will experience a considerable speedup by
using an HMAC module written in C instead.  Perhaps this isn't worth
the effort to write such a module since the underlying modules (sha,
md5) that HMAC uses are already written in C.

Thanks.

---- cut here ----

# -*- python -*-
#
# Copyright (C) 2001,2002 Jason R. Mastaler <jason at mastaler.com>
#
# This file is part of TMDA.
#
# TMDA is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.  A copy of this license should
# be included in the file COPYING.
#
# TMDA is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
# FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License
# along with TMDA; if not, write to the Free Software Foundation, Inc.,
# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

"""
HMAC (Keyed-Hashing for Message Authentication) in Python.

Implements the HMAC algorithm for SHA-1 as described by
RFC 2104 (http://www.faqs.org/rfcs/rfc2104.html).

Python >= 2.2 includes a more general purpose HMAC module
(http://www.python.org/doc/current/lib/module-hmac.html).
"""

from array import array
import sha


_ipad="\x36"*64
_opad="\x5C"*64

_itrans = array('B', [0]*256)
_otrans = array('B', [0]*256)
for i in xrange(256):
    _itrans[i] = i ^ 0x36
    _otrans[i] = i ^ 0x5c
_itrans = _itrans.tostring()
_otrans = _otrans.tostring()

newh = sha.new

class hmac:
    def __init__(self, k, m=None):
        if isinstance(k, hmac):
            self.inner = k.inner.copy()
            self.outer = k.outer.copy()
        else:
            self.outer = newh()
            self.inner = newh()
            if len(k) > 64:
                k=newh(k).digest()
            k = k + chr(0)*(64-len(k))
            self.inner.update((k.translate(_itrans)+_ipad)[:64])
            self.outer.update((k.translate(_otrans)+_opad)[:64])
            if (m):
                self.update(m)
    def update(self, m):
        self.inner.update(m)
    def digest(self):
        h=self.outer.copy()
        h.update(self.inner.digest())
        return h.digest()

def new(k, m=None):
    return hmac(k, m)




From PaulLambert at WOODSIDENET.COM  Wed May 29 23:48:10 2002
From: PaulLambert at WOODSIDENET.COM (Paul Lambert)
Date: Wed, 29 May 2002 14:48:10 -0700
Subject: [PYTHON-CRYPTO] high-performance HMAC module
Message-ID: <3FFBC907DD03A34CA4410C5C745DEB12A3EA78@wnimail.WoodsideNet.Com>

> -----Original Message-----
> From: Jason R. Mastaler [mailto:jason-list-python-crypto at MASTALER.COM]

> >> would be worth writing an HMAC module in C.
> >
> > No, not for this purpose.  It would be better to move to a real
> > encryption algorithm (like AES) written in C than to use p2.
> 
> I think you misunderstood my question.  I'm not using p2.  But p2
> contains a pure-python HMAC module similar to the one that I'm using
> (which I've attached below).

Oops, yes.  

> 
> What I'm asking is whether I will experience a considerable speedup by
> using an HMAC module written in C instead.  Perhaps this isn't worth
> the effort to write such a module since the underlying modules (sha,
> md5) that HMAC uses are already written in C.

Not sure .... it might help for help for very long strings.  For long files you'd still be making be HMACing in chunks/lines so it would not help much.

Paul

> 
> Thanks.
> 
> ---- cut here ----
> 
> # -*- python -*-
> #
> # Copyright (C) 2001,2002 Jason R. Mastaler <jason at mastaler.com>
> #
> # This file is part of TMDA.
> #
> # TMDA is free software; you can redistribute it and/or modify it
> # under the terms of the GNU General Public License as published by
> # the Free Software Foundation; either version 2 of the License, or
> # (at your option) any later version.  A copy of this license should
> # be included in the file COPYING.
> #
> # TMDA is distributed in the hope that it will be useful, but WITHOUT
> # ANY WARRANTY; without even the implied warranty of 
> MERCHANTABILITY or
> # FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General 
> Public License
> # for more details.
> #
> # You should have received a copy of the GNU General Public License
> # along with TMDA; if not, write to the Free Software 
> Foundation, Inc.,
> # 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
> 
> """
> HMAC (Keyed-Hashing for Message Authentication) in Python.
> 
> Implements the HMAC algorithm for SHA-1 as described by
> RFC 2104 (http://www.faqs.org/rfcs/rfc2104.html).
> 
> Python >= 2.2 includes a more general purpose HMAC module
> (http://www.python.org/doc/current/lib/module-hmac.html).
> """
> 
> from array import array
> import sha
> 
> 
> _ipad="\x36"*64
> _opad="\x5C"*64
> 
> _itrans = array('B', [0]*256)
> _otrans = array('B', [0]*256)
> for i in xrange(256):
>     _itrans[i] = i ^ 0x36
>     _otrans[i] = i ^ 0x5c
> _itrans = _itrans.tostring()
> _otrans = _otrans.tostring()
> 
> newh = sha.new
> 
> class hmac:
>     def __init__(self, k, m=None):
>         if isinstance(k, hmac):
>             self.inner = k.inner.copy()
>             self.outer = k.outer.copy()
>         else:
>             self.outer = newh()
>             self.inner = newh()
>             if len(k) > 64:
>                 k=newh(k).digest()
>             k = k + chr(0)*(64-len(k))
>             self.inner.update((k.translate(_itrans)+_ipad)[:64])
>             self.outer.update((k.translate(_otrans)+_opad)[:64])
>             if (m):
>                 self.update(m)
>     def update(self, m):
>         self.inner.update(m)
>     def digest(self):
>         h=self.outer.copy()
>         h.update(self.inner.digest())
>         return h.digest()
> 
> def new(k, m=None):
>     return hmac(k, m)
> 




From bram at GAWTH.COM  Wed May 29 23:54:43 2002
From: bram at GAWTH.COM (Bram Cohen)
Date: Wed, 29 May 2002 14:54:43 -0700
Subject: [PYTHON-CRYPTO] high-performance HMAC module
In-Reply-To: <3FFBC907DD03A34CA4410C5C745DEB12A3EA78@wnimail.WoodsideNet.Com>
Message-ID: <Pine.LNX.4.21.0205291454270.11230-100000@ultra.gawth.com>

By the way, the sha1 included in Python is very slow - around 2 megs/sec.

-Bram Cohen

"Markets can remain irrational longer than you can remain solvent"
                                        -- John Maynard Keynes




From jason-list-python-crypto at MASTALER.COM  Thu May 30 01:08:52 2002
From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler)
Date: Wed, 29 May 2002 18:08:52 -0500
Subject: [PYTHON-CRYPTO] high-performance HMAC module
In-Reply-To: <3FFBC907DD03A34CA4410C5C745DEB12A3EA78@wnimail.WoodsideNet.Com>
              (Paul Lambert's message of "Wed, 29 May 2002 14:48:10 -0700")
References: <3FFBC907DD03A34CA4410C5C745DEB12A3EA78@wnimail.WoodsideNet.Com>
Message-ID: <1022713732.42783.TMDA@nightshade.la.mastaler.com>

Paul Lambert <PaulLambert at WOODSIDENET.COM> writes:

> Not sure .... it might help for help for very long strings.

In this particular app (TMDA), it's almost all short strings, so it
probably wouldn't help much.

Thanks.

--
(http://tmda.sourceforge.net/)




From jason-list-python-crypto at MASTALER.COM  Thu May 30 01:10:02 2002
From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler)
Date: Wed, 29 May 2002 19:10:02 -0400
Subject: [PYTHON-CRYPTO] high-performance HMAC module
In-Reply-To: <Pine.LNX.4.21.0205291454270.11230-100000@ultra.gawth.com> (Bram
              Cohen's message of "Wed, 29 May 2002 14:54:43 -0700")
References: <3FFBC907DD03A34CA4410C5C745DEB12A3EA78@wnimail.WoodsideNet.Com>
            <Pine.LNX.4.21.0205291454270.11230-100000@ultra.gawth.com>
Message-ID: <1022713802.42802.TMDA@nightshade.la.mastaler.com>

Bram Cohen <bram at GAWTH.COM> writes:

> By the way, the sha1 included in Python is very slow - around 2
> megs/sec.

I didn't know this.  Any reason then why someone (you?) haven't
contributed a faster implementation for the Python standard lib?

--
(http://tmda.sourceforge.net/)




From bryan at EEVOLVED.COM  Thu May 30 02:03:50 2002
From: bryan at EEVOLVED.COM (Bryan Mongeau)
Date: Wed, 29 May 2002 20:03:50 -0400
Subject: [PYTHON-CRYPTO] high-performance HMAC module
In-Reply-To: <1022713802.42802.TMDA@nightshade.la.mastaler.com>
References: <3FFBC907DD03A34CA4410C5C745DEB12A3EA78@wnimail.WoodsideNet.Com>
            <Pine.LNX.4.21.0205291454270.11230-100000@ultra.gawth.com>
            <1022713802.42802.TMDA@nightshade.la.mastaler.com>
Message-ID: <PYTHON-CRYPTO%2002053002035255@NIC.SURFNET.NL>

On Wednesday 29 May 2002 07:10 pm, Jason R. Mastaler wrote:
> Bram Cohen <bram at GAWTH.COM> writes:
> > By the way, the sha1 included in Python is very slow - around 2
> > megs/sec.
>
> I didn't know this.  Any reason then why someone (you?) haven't
> contributed a faster implementation for the Python standard lib?


As a reference point, my SHA 256 implementation hits 12 MB/s and SHA 384/512
hits 6 MB/s on my 800 Mhz.  Strange that the standard lib SHA 160 scores so
low. Should be straight-forward to code properly.
--
Bryan Mongeau
eEvolved Inc. - IT Consulting & Custom Software
http://eevolved.com/
--
"C keeps your hands tied. PERL gives you enough rope to hang yourself." --
Anonymous




From jason-list-python-crypto at MASTALER.COM  Thu May 30 19:33:34 2002
From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler)
Date: Thu, 30 May 2002 14:33:34 -0300
Subject: [PYTHON-CRYPTO] high-performance HMAC module
In-Reply-To: <PYTHON-CRYPTO%2002053002035255@NIC.SURFNET.NL> (Bryan Mongeau's
              message of "Wed, 29 May 2002 20:03:50 -0400")
References: <3FFBC907DD03A34CA4410C5C745DEB12A3EA78@wnimail.WoodsideNet.Com>
            <Pine.LNX.4.21.0205291454270.11230-100000@ultra.gawth.com>
            <1022713802.42802.TMDA@nightshade.la.mastaler.com>
            <PYTHON-CRYPTO%2002053002035255@NIC.SURFNET.NL>
Message-ID: <1022780014.44978.TMDA@nightshade.la.mastaler.com>

Bryan Mongeau <bryan at EEVOLVED.COM> writes:

> As a reference point, my SHA 256 implementation hits 12 MB/s and SHA
> 384/512 hits 6 MB/s on my 800 Mhz.  Strange that the standard lib
> SHA 160 scores so low. Should be straight-forward to code properly.

Yup.  Would you be interested in taking look at optimizing the Python
lib's shamodule.c?  Lots of folks whose applications use this module
would be grateful I'm sure.  I'm not a C programmer, or else I'd have
already done it.

BTW, I also noticed an outstanding feature request for SHA 256/384/512
in the Python standard lib.  See http://python.org/sf/480967
Looks like the Python team is agreeable, provided someone can fork up
the code.

Still plenty of time for both of these to make it into Python 2.3.

--
(http://tmda.sourceforge.net/)




From phr-pycrypt at nightsong.com  Thu May 30 19:39:46 2002
From: phr-pycrypt at nightsong.com (Paul Rubin)
Date: Thu, 30 May 2002 17:39:46 -0000
Subject: [PYTHON-CRYPTO] HMAC module
Message-ID: <20020530173946.2957.qmail@brouhaha.com>

For long strings, the speed of HMAC will be limited by the speed of
the SHA implementation.  For short strings, the hmac from p2.py should
be quite a bit faster than the Python 2.2 hmac.py, but one in C will
be even faster.

There's a very fast SHA implementation included with OpenSSL, that
you could compare with Python's implementation.  Type "openssl speed
sha" to run a benchmark.  Again, there's lots of overhead for short
strings.  "2M/sec" is uninformative unless the string length is
specified.

My guess is that Python's SHA implementation isn't super-efficient but
it's probably not too bad.

p2.py is intended to supply an encryption function in pure-Python
environments where you can't load a C module.  Of course it's better
to use an AES module if one is available.  I'm still travelling but
will return next week and should be able to do something about the AES
module when I get back.

I don't mind if you adapt p2.py's HMAC implementation for your own
module, but I'd appreciate a credit in your source file if you do
that.  FWIW, I also have a replacement hmac.py (i.e. one that supplies
the same OO interface as Python 2.2's HMAC module) that I'll put on my
web site when I get back, if anyone wants it.  I was thinking of
submitting it to Guido since it should be somewhat faster than the
original (though not as fast as a C function).

Note that the OO stuff probably slows things down considerably for
short strings.  You may be better off using a string-function
interface like the one in p2.py rather than the OO interface in your
TMDA file.

Finally, if your application needs a very fast MAC but doesn't have to
be HMAC-SHA compatible, you might look at Dan Bernstein's hash127
function, <http://cr.yp.to/hash127.html>.  It's considerably faster
than MD5 or SHA, and has a security proof that's better than either.

Paul




From akuchlin at mems-exchange.org  Thu May 30 20:49:00 2002
From: akuchlin at mems-exchange.org (Andrew Kuchling)
Date: Thu, 30 May 2002 14:49:00 -0400
Subject: [PYTHON-CRYPTO] high-performance HMAC module
In-Reply-To: <Pine.LNX.4.21.0205291454270.11230-100000@ultra.gawth.com>
References: <3FFBC907DD03A34CA4410C5C745DEB12A3EA78@wnimail.WoodsideNet.Com>
            <Pine.LNX.4.21.0205291454270.11230-100000@ultra.gawth.com>
Message-ID: <20020530184900.GA27918@ute.mems-exchange.org>

On Wed, May 29, 2002 at 02:54:43PM -0700, Bram Cohen wrote:
>By the way, the sha1 included in Python is very slow - around 2 megs/sec.

I can think of a few possible reasons for this.  First is that the
main transformation is in the sha_transform() function in the C code,
so there's a C function call for every block of input text.  This
might be difficult to fix, though, as sha_transform() is called from
several places and it's pretty long.

One silly thing that should be easily fixable is that sha_transform()
also does byte-swapping on every call, and it really shouldn't.
That's probably the low-hanging fruit here.

The R32() macro and the f()-functions could also be translated to
assembly code, but I haven't looked at the output of 'gcc -S' to see
what the generated code looks like.

--amk




From bram at GAWTH.COM  Thu May 30 21:06:09 2002
From: bram at GAWTH.COM (Bram Cohen)
Date: Thu, 30 May 2002 12:06:09 -0700
Subject: [PYTHON-CRYPTO] HMAC module
In-Reply-To: <20020530173946.2957.qmail@brouhaha.com>
Message-ID: <Pine.LNX.4.21.0205301200270.31646-100000@ultra.gawth.com>

Paul Rubin wrote:

> There's a very fast SHA implementation included with OpenSSL, that
> you could compare with Python's implementation.  Type "openssl speed
> sha" to run a benchmark.  Again, there's lots of overhead for short
> strings.  "2M/sec" is uninformative unless the string length is
> specified.

Python 2.1.3 (#1, Apr 20 2002, 10:14:34)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "copyright", "credits" or "license" for more information.
>>> from time import time
>>> from sha import sha
>>> x = 'a' * 2 ** 20
>>> def test():
...     t = time()
...     for i in xrange(100):
...         sha(x).digest()
...     print 100.0/(time() - t)
...
>>> test()
1.34924164806

so that's 1.35 megs/sec on megabyte size blocks.

openssl reports 3.63 megs/sec on blocks of 8k.

that's a factor of 2.7 difference. Python's sha1 is, in fact, slow.

-Bram Cohen

"Markets can remain irrational longer than you can remain solvent"
                                        -- John Maynard Keynes




From jason-list-python-crypto at MASTALER.COM  Fri May 31 00:38:11 2002
From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler)
Date: Thu, 30 May 2002 16:38:11 -0600
Subject: [PYTHON-CRYPTO] HMAC module
In-Reply-To: <20020530173946.2957.qmail@brouhaha.com> (Paul Rubin's message of
              "Thu, 30 May 2002 17:39:46 -0000")
References: <20020530173946.2957.qmail@brouhaha.com>
Message-ID: <1022798291.45457.TMDA@nightshade.la.mastaler.com>

Paul Rubin <phr-pycrypt at NIGHTSONG.COM> writes:

> For long strings, the speed of HMAC will be limited by the speed of
> the SHA implementation.  For short strings, the hmac from p2.py
> should be quite a bit faster than the Python 2.2 hmac.py, but one in
> C will be even faster.

For clarification, what are the length of "short" and "long" strings
in this context?

> I don't mind if you adapt p2.py's HMAC implementation for your own
> module, but I'd appreciate a credit in your source file if you do
> that.

I thought I had done that, sorry.  It's done now.

> FWIW, I also have a replacement hmac.py (i.e. one that supplies the
> same OO interface as Python 2.2's HMAC module) that I'll put on my
> web site when I get back, if anyone wants it.

Yes, I'd like to take a look at it.

> I was thinking of submitting it to Guido since it should be somewhat
> faster than the original (though not as fast as a C function).

Why not just submit a C-based HMAC module instead?

> Finally, if your application needs a very fast MAC but doesn't have
> to be HMAC-SHA compatible, you might look at Dan Bernstein's hash127
> function

TMDA doesn't necessarily need HMAC-SHA, so hash127 would work, except
for that fact that I want to keep TMDA 100% Python, and don't want to
have to ship any C code.

Thanks.

--
(http://tmda.sourceforge.net/)




From zooko at zooko.com  Fri May 31 13:22:24 2002
From: zooko at zooko.com (Zooko)
Date: Fri, 31 May 2002 04:22:24 -0700
Subject: [PYTHON-CRYPTO] SHA-1 benchmarks (was: HMAC module)
In-Reply-To: Message from Bram Cohen <bram@GAWTH.COM> of "Thu, 30 May 2002
              12:06:09 PDT."
              <Pine.LNX.4.21.0205301200270.31646-100000@ultra.gawth.com>
References: <Pine.LNX.4.21.0205301200270.31646-100000@ultra.gawth.com>
Message-ID: <E17DkUD-0004Bp-00@imp>

These are all on the same machine just now.  I'm not sure if they are
really comparable.


First I copied Bram's Python script to measure the `sha' module.  I got a
vastly different result than he did:

MAIN imp:~$ python2.2
Python 2.2.1 (#1, May  3 2002, 23:19:03)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from time import time
>>> from sha import sha
>>> x = 'a' * 2 ** 20
>>> def test():
...     t = time()
...     for i in xrange(100):
...         sha(x).digest()
...     print 100.0/(time() - t)
...
>>> test()
18.208912899
>>>

(Bram got 1.3 as the result from the identical script.)


Then I ran `openssl speed sha':

MAIL imp:~/playground/mnet/extsrc/tst$ openssl speed sha
Doing sha1 for 3s on 8 size blocks: 902056 sha1's in 3.00s
Doing sha1 for 3s on 64 size blocks: 454294 sha1's in 3.00s
Doing sha1 for 3s on 256 size blocks: 195575 sha1's in 3.00s
Doing sha1 for 3s on 1024 size blocks: 59617 sha1's in 3.00s
Doing sha1 for 3s on 8192 size blocks: 7966 sha1's in 3.00s
OpenSSL 0.9.6c 21 dec 2001
built on: Sat Jan  5 19:53:45 CET 2002
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) blowfish(idx)
compiler: gcc -fPIC -DTHREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DNO_IDEA -DNO_MDC2 -DNO_RC5 -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -Wall
The 'numbers' are in 1000s of bytes per second processed.
type              8 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
sha1              2405.48k     9691.61k    16689.07k    20349.27k    21752.49k


Then I ran `cryptest.exe b' (from Crypto++ 4.2), and it reported
this:

Algorithm       Bytes Processed Time Taken      Megabytes(2^20 bytes)/Second
SHA-1           4194304         0.720           5.556


I doubt that these are comparable numbers.  For one thing, Bram's machine and
mine are probably about the same speed, so him getting 1.3 and me getting 18.2
is pretty suspicious.


Regards,

Zooko

-------
Secure Distributed Systems Consulting -- http://zooko.com/
-------