From greg at ELECTRICRAIN.COM Mon Apr 2 10:24:25 2001 From: greg at ELECTRICRAIN.COM (Gregory P. Smith) Date: Mon, 2 Apr 2001 01:24:25 -0700 Subject: [PYTHON-CRYPTO] Needed: public CVS repository for amkCrypto In-Reply-To: ; from akuchlin@MEMS-EXCHANGE.ORG on Wed, Mar 28, 2001 at 04:00:51PM -0500 References: Message-ID: <20010402012425.A19133@zot.electricrain.com> On Wed, Mar 28, 2001 at 04:00:51PM -0500, Andrew Kuchling wrote: > I've noted the recent threads about bugs in amkCrypto, and feel bad > about not being able to spend time issuing a new release to fix them > (lack of time, and partially lack of interest). An obvious solution > would be to set up a publicly accessible CVS tree somewhere, but > where? SourceForge is obviously available, but it's inside the US; is > there a SourceForge-like site available in Europe? Just go with sourceforge, the US crypto laws are reasonable these days. -G -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From ngps at POST1.COM Mon Apr 2 17:08:02 2001 From: ngps at POST1.COM (Ng Pheng Siong) Date: Mon, 2 Apr 2001 23:08:02 +0800 Subject: [PYTHON-CRYPTO] [ANN] HOWTO: Programming S/MIME in Python with M2Crypto Message-ID: <20010402230802.A1288@madcap.dyndns.org> Hi, I'm pleased to announce a HOWTO on programming S/MIME in Python with M2Crypto. See it here: http://www.post1.com/home/ngps/m2 Also new is a HOWTO on creating your own CA with OpenSSL. As usual, feedback is welcome. -- Ng Pheng Siong * http://www.post1.com/home/ngps From aarchiba at YAHOO.COM Mon Apr 2 20:41:51 2001 From: aarchiba at YAHOO.COM (Andrew Archibald) Date: Mon, 2 Apr 2001 14:41:51 -0400 Subject: [PYTHON-CRYPTO] Needed: public CVS repository for amkCrypto In-Reply-To: <20010402012425.A19133@zot.electricrain.com>; from greg@ELECTRICRAIN.COM on Mon, Apr 02, 2001 at 01:24:25AM -0700 References: <20010402012425.A19133@zot.electricrain.com> Message-ID: <20010402144151.B20680@hedgehog> On Mon, Apr 02, 2001 at 01:24:25AM -0700, Gregory P. Smith wrote: > On Wed, Mar 28, 2001 at 04:00:51PM -0500, Andrew Kuchling wrote: > > I've noted the recent threads about bugs in amkCrypto, and feel bad > > about not being able to spend time issuing a new release to fix them > > (lack of time, and partially lack of interest). An obvious solution > > would be to set up a publicly accessible CVS tree somewhere, but > > where? SourceForge is obviously available, but it's inside the US; is > > there a SourceForge-like site available in Europe? > > Just go with sourceforge, the US crypto laws are reasonable these days. So, this seems to be the common wisdom. But it's not clear to me that the laws really are OK. As I understand it, the laws haven't changed, but the policy has, and it could change back at any moment. In any case, you should send some mail to exports at crypto.com to announce the availability of your source code. Who gets in trouble? If you're not an American citizen, it seems like it should be SourceForge, in which case they're already at risk. It would be good to have an international place to put amkCrypto. Andrew From rsalz at ZOLERA.COM Mon Apr 2 20:57:10 2001 From: rsalz at ZOLERA.COM (Rich Salz) Date: Mon, 2 Apr 2001 14:57:10 -0400 Subject: [PYTHON-CRYPTO] Needed: public CVS repository for amkCrypto References: <20010402012425.A19133@zot.electricrain.com> <20010402144151.B20680@hedgehog> Message-ID: <3AC8CB86.27E06AF8@zolera.com> > So, this seems to be the common wisdom. But it's not clear to me that the > laws really are OK. As I understand it, the laws haven't changed, but the > policy has, and it could change back at any moment. US export of crypto has *always* been controlled by regulation, not law. The regulations might change, but they can't make it retroactively illegal. It is possible that tomorrow the US will say "export of crypto source code requires written permission from Bureau of Export Control, effective immediately." Is that a realistic concern? In my opinion (and I've been peripherally involved in these issues since the 1980's, when I posted DES code in the Usenet newsgroup mod.sources), the answer is strongly NO. There will either be time to move things, or an immediate court challenge that will get a delay. I think there are far more important things to worry about (like SF going bankrupt :) > In any case, you should send some mail to exports at crypto.com to announce > the availability of your source code. Yes, that's a good idea. (That's why I asked Matt to create it. :) > Who gets in trouble? If you're not an American citizen, it seems like it > should be SourceForge, in which case they're already at risk. And according to one of their staffers (Tim Purdue), they're fine with crypto projects. > It would be good to have an international place to put amkCrypto. I don't feel it's necessary. Your mileage may vary. /r$ From greg at ELECTRICRAIN.COM Mon Apr 2 21:39:31 2001 From: greg at ELECTRICRAIN.COM (Gregory P. Smith) Date: Mon, 2 Apr 2001 12:39:31 -0700 Subject: [PYTHON-CRYPTO] Needed: public CVS repository for amkCrypto In-Reply-To: <20010402144151.B20680@hedgehog>; from aarchiba@yahoo.com on Mon, Apr 02, 2001 at 02:41:51PM -0400 References: <20010402012425.A19133@zot.electricrain.com> <20010402144151.B20680@hedgehog> Message-ID: <20010402123931.A7437@zot.electricrain.com> On Mon, Apr 02, 2001 at 02:41:51PM -0400, Andrew Archibald wrote: > On Mon, Apr 02, 2001 at 01:24:25AM -0700, Gregory P. Smith wrote: > > On Wed, Mar 28, 2001 at 04:00:51PM -0500, Andrew Kuchling wrote: > > > (lack of time, and partially lack of interest). An obvious solution > > > would be to set up a publicly accessible CVS tree somewhere, but > > > where? SourceForge is obviously available, but it's inside the US; is > > > there a SourceForge-like site available in Europe? > > > > Just go with sourceforge, the US crypto laws are reasonable these days. > > So, this seems to be the common wisdom. But it's not clear to me that the > laws really are OK. As I understand it, the laws haven't changed, but the > policy has, and it could change back at any moment. > > In any case, you should send some mail to exports at crypto.com to announce > the availability of your source code. > > Who gets in trouble? If you're not an American citizen, it seems like it > should be SourceForge, in which case they're already at risk. > > It would be good to have an international place to put amkCrypto. > > Andrew If sourceforge accepts the project then it is their responsability. Several other crypto related projects are already hosted there. If the us export rules suddenly change for the worse (highly unlikely) your code will not vanish, you'll just need to find a new server. -G -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From jason-list-python-crypto at MASTALER.COM Tue Apr 10 17:53:39 2001 From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler) Date: Tue, 10 Apr 2001 15:53:39 -0000 Subject: [PYTHON-CRYPTO] amkCrypto-0.1.3 build fails on BSD/OS Message-ID: <20010410155339.7278.qmail@nightshade.la.mastaler.com> Any ideas why amkCrypto won't compile for me? This is BSD/OS 4.0, gcc 2.7.2.1 and Python 2.0. (python setup.py build) building 'Crypto.mxCrypto' extension creating build/temp.bsd creating build/temp.bsd/os-4.0-i386-2.0 creating build/temp.bsd/os-4.0-i386-2.0/mxCrypto gcc -D_HAVE_BSDI -g -O2 -Wall -Wstrict-prototypes -fpic -I/usr/local/ssl/include -I/usr/local/python2.0/include/python2.0 -c mxCrypto/mxCrypto.cc -o build/temp.bsd/os-4.0-i386-2.0/mxCrypto/mxCrypto.o mxCrypto/mxCrypto.cc: In method `StreamCipher::StreamCipher(PyStringObject *, int)': mxCrypto/mxCrypto.cc:125: warning: `catch', `throw', and `try' are all C++ reserved words mxCrypto/mxCrypto.cc:125: exception handling disabled, use -fhandle-exceptions to enable. mxCrypto/mxCrypto.cc:110: warning: unused parameter `struct PyStringObject * key' mxCrypto/mxCrypto.cc:124: warning: label `onError' defined but not used mxCrypto/mxCrypto.cc: In method `struct _object * StreamCipher::encrypt(struct PyStringObject *)': mxCrypto/mxCrypto.cc:134: warning: unused parameter `struct PyStringObject * input' mxCrypto/mxCrypto.cc: In method `struct _object * StreamCipher::decrypt(struct PyStringObject *)': mxCrypto/mxCrypto.cc:142: warning: unused parameter `struct PyStringObject * input' mxCrypto/mxCrypto.cc: In method `RC2Cipher::RC2Cipher(PyStringObject *, int, PyStringObject *)': mxCrypto/mxCrypto.cc:201: warning: label `onError' defined but not used mxCrypto/mxCrypto.cc: In method `RC5Cipher::RC5Cipher(PyStringObject *, int, PyStringObject *, int)': mxCrypto/mxCrypto.cc:332: warning: label `onError' defined but not used mxCrypto/mxCrypto.cc: In method `BlowfishCipher::BlowfishCipher(PyStringObject *, int, PyStringObject *)': mxCrypto/mxCrypto.cc:404: warning: label `onError' defined but not used mxCrypto/mxCrypto.cc: In method `Hash::Hash(PyStringObject *)': mxCrypto/mxCrypto.cc:812: warning: unused parameter `struct PyStringObject * data' mxCrypto/mxCrypto.cc: In method `struct _object * Hash::setstate(struct PyStringObject *)': mxCrypto/mxCrypto.cc:828: warning: unused parameter `struct PyStringObject * state' mxCrypto/mxCrypto.cc: In method `struct _object * Hash::update(struct PyStringObject *)': mxCrypto/mxCrypto.cc:859: warning: unused parameter `struct PyStringObject * data' mxCrypto/mxCrypto.cc: In method `struct _object * MD2Hash::update(struct PyStringObject *)': mxCrypto/mxCrypto.cc:925: warning: label `onError' defined but not used mxCrypto/mxCrypto.cc: In method `struct _object * MD5Hash::update(struct PyStringObject *)': mxCrypto/mxCrypto.cc:988: warning: label `onError' defined but not used mxCrypto/mxCrypto.cc: In method `struct _object * SHAHash::update(struct PyStringObject *)': mxCrypto/mxCrypto.cc:1051: warning: label `onError' defined but not used mxCrypto/mxCrypto.cc: In method `struct _object * RIPEMDHash::update(struct PyStringObject *)': mxCrypto/mxCrypto.cc:1114: warning: label `onError' defined but not used error: command 'gcc' failed with exit status 1 From itamarst at YAHOO.COM Tue Apr 17 15:51:34 2001 From: itamarst at YAHOO.COM (Itamar S.-T.) Date: Tue, 17 Apr 2001 06:51:34 -0700 Subject: [PYTHON-CRYPTO] Compiling amkCrypto in Windows Message-ID: <20010417135134.91860.qmail@web13001.mail.yahoo.com> I'm attempting to compile amkCrypto's mxCrypto extension module in Windows (win2000, vc6, openssl 0.9.6a, Python 2.0). When I run "setup.py build", I get the following error. Any ideas? C:\Program Files\Microsoft Visual Studio\VC98\BIN\cl.exe /c /nologo /Ox /MD /W3 /GX -Ic:\openssl\include -IC:\Python20\Include /TpmxCrypto/mxCrypto.cc /Fobuild\temp.win32-2.0\Release\mxCrypto/mxCrypto.obj mxCrypto.cc mxCrypto/mxCrypto.h(45) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(45) : error C2252: 'blocksize' : pure specifier can only bespecified for functions mxCrypto/mxCrypto.h(46) : error C2252: 'keysize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(62) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(62) : error C2252: 'blocksize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(125) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(125) : error C2252: 'keysize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(140) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(140) : error C2252: 'keysize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(158) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(158) : error C2252: 'keysize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(184) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(184) : error C2252: 'digestsize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(201) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(201) : error C2252: 'digestsize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(216) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(216) : error C2252: 'digestsize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(231) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(231) : error C2252: 'digestsize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.h(246) : error C2258: illegal pure syntax, must be '= 0' mxCrypto/mxCrypto.h(246) : error C2252: 'digestsize' : pure specifier can only be specified for functions mxCrypto/mxCrypto.cc(165) : error C2065: 'blocksize' : undeclared identifier mxCrypto/mxCrypto.cc(473) : error C2065: 'keysize' : undeclared identifier mxCrypto/mxCrypto.cc(904) : error C2065: 'digestsize' : undeclared identifier error: command '"C:\Program Files\Microsoft Visual Studio\VC98\BIN\cl.exe"' failed with exit status 2 ===== Itamar Shtull-Trauring, itamar(at)shtull-trauring.org __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From jason-list-python-crypto at MASTALER.COM Tue Apr 17 17:16:03 2001 From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler) Date: Tue, 17 Apr 2001 15:16:03 -0000 Subject: [PYTHON-CRYPTO] Compiling amkCrypto in Windows In-Reply-To: <20010417135134.91860.qmail@web13001.mail.yahoo.com> ("Itamar S.-T."'s message of "Tue, 17 Apr 2001 06:51:34 -0700") References: <20010417135134.91860.qmail@web13001.mail.yahoo.com> Message-ID: <20010417151603.2588.qmail@nightshade.la.mastaler.com> "Itamar S.-T." writes: > I'm attempting to compile amkCrypto's mxCrypto extension module in > Windows (win2000, vc6, openssl 0.9.6a, Python 2.0). Do you have any better luck if you compile with cygwin/gcc instead? From itamarst at YAHOO.COM Tue Apr 17 17:42:46 2001 From: itamarst at YAHOO.COM (Itamar S.-T.) Date: Tue, 17 Apr 2001 08:42:46 -0700 Subject: [PYTHON-CRYPTO] Compiling amkCrypto in Windows In-Reply-To: <20010417151603.2588.qmail@nightshade.la.mastaler.com> Message-ID: <20010417154246.28830.qmail@web13007.mail.yahoo.com> --- "Jason R. Mastaler" wrote: > "Itamar S.-T." writes: > > > I'm attempting to compile amkCrypto's mxCrypto > extension module in > > Windows (win2000, vc6, openssl 0.9.6a, Python > 2.0). > > Do you have any better luck if you compile with > cygwin/gcc instead? I don't think compiling with cygwin will yield binaries that are compatible with Python (since its compiled with Visual C)++. mingw32 *can* compile binaries that are compatible with VC, but I've only seen instructions for using it with Python 1.5.2 (and changing it to 2.0 is beyond me). It does in fact work, so if someone would help update the instructions I could do it: http://starship.python.net/crew/kernr/mingw32/Notes.html ===== Itamar Shtull-Trauring, itamar(at)shtull-trauring.org __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From itamarst at YAHOO.COM Wed Apr 18 11:53:21 2001 From: itamarst at YAHOO.COM (Itamar S.-T.) Date: Wed, 18 Apr 2001 02:53:21 -0700 Subject: [PYTHON-CRYPTO] ANN: amkCrypto for Win32 In-Reply-To: <20010417135134.91860.qmail@web13001.mail.yahoo.com> Message-ID: <20010418095321.95302.qmail@web13004.mail.yahoo.com> I've managed to compile amkCrypto for Windows, with a few minor changes to the source code. I've created a win32 distribution of amkCrypto that includes the necessary changes to the source, plus precompiled binaries so you can use it out of the box. You can download it at http://itamarst.org/downloads/ ===== Itamar Shtull-Trauring, itamar(at)shtull-trauring.org __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From jason-list-python-crypto at MASTALER.COM Wed Apr 18 19:49:04 2001 From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler) Date: Wed, 18 Apr 2001 17:49:04 -0000 Subject: [PYTHON-CRYPTO] ANN: amkCrypto for Win32 In-Reply-To: <20010418095321.95302.qmail@web13004.mail.yahoo.com> ("Itamar S.-T."'s message of "Wed, 18 Apr 2001 02:53:21 -0700") References: <20010418095321.95302.qmail@web13004.mail.yahoo.com> Message-ID: <20010418174904.11359.qmail@nightshade.la.mastaler.com> "Itamar S.-T." writes: > I've managed to compile amkCrypto for Windows, with a few minor > changes to the source code. I've created a win32 distribution of > amkCrypto that includes the necessary changes to the source, plus > precompiled binaries so you can use it out of the box. Good deal. It would be nice to get these changes integrated into the main amkCrypto distribution. From jason-list-python-crypto at MASTALER.COM Wed Apr 18 19:50:31 2001 From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler) Date: Wed, 18 Apr 2001 17:50:31 -0000 Subject: [PYTHON-CRYPTO] Needed: public CVS repository for amkCrypto In-Reply-To: <20010402123931.A7437@zot.electricrain.com> ("Gregory P. Smith"'s message of "Mon, 2 Apr 2001 12:39:31 -0700") References: <20010402012425.A19133@zot.electricrain.com> <20010402144151.B20680@hedgehog> <20010402123931.A7437@zot.electricrain.com> Message-ID: <20010418175031.11369.qmail@nightshade.la.mastaler.com> "Gregory P. Smith" writes: > If sourceforge accepts the project then it is their responsability. > Several other crypto related projects are already hosted there. If > the us export rules suddenly change for the worse (highly unlikely) > your code will not vanish, you'll just need to find a new server. Is this reasoning good enough for you Andrew, or are you still looking for a CVS repository outside the US? From akuchlin at mems-exchange.org Wed Apr 18 20:06:11 2001 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Wed, 18 Apr 2001 14:06:11 -0400 Subject: [PYTHON-CRYPTO] Needed: public CVS repository for amkCrypto In-Reply-To: <20010418175031.11369.qmail@nightshade.la.mastaler.com>; from jason-list-python-crypto@MASTALER.COM on Wed, Apr 18, 2001 at 05:50:31PM -0000 References: <20010402012425.A19133@zot.electricrain.com> <20010402144151.B20680@hedgehog> <20010402123931.A7437@zot.electricrain.com> <20010418175031.11369.qmail@nightshade.la.mastaler.com> Message-ID: <20010418140611.C24396@ute.cnri.reston.va.us> On Wed, Apr 18, 2001 at 05:50:31PM -0000, Jason R. Mastaler wrote: >Is this reasoning good enough for you Andrew, or are you still looking >for a CVS repository outside the US? No, I'm waiting for SourceForge to actually import the CVS tree into the existing PyCrypto project (which PC Drew registered previously). The request was submitted on April 7; still waiting... --amk From jason-list-python-crypto at MASTALER.COM Fri Apr 20 18:22:03 2001 From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler) Date: Fri, 20 Apr 2001 10:22:03 -0600 Subject: [PYTHON-CRYPTO] Needed: public CVS repository for amkCrypto In-Reply-To: <20010418140611.C24396@ute.cnri.reston.va.us> (Andrew Kuchling's message of "Wed, 18 Apr 2001 14:06:11 -0400") References: <20010402012425.A19133@zot.electricrain.com> <20010402144151.B20680@hedgehog> <20010402123931.A7437@zot.electricrain.com> <20010418175031.11369.qmail@nightshade.la.mastaler.com> <20010418140611.C24396@ute.cnri.reston.va.us> Message-ID: Andrew Kuchling writes: > No, I'm waiting for SourceForge to actually import the CVS tree into > the existing PyCrypto project (which PC Drew registered previously). > The request was submitted on April 7; still waiting... Did you file this request again project alexandria (code name for sourceforge?) They will likely never see it otherwise. Go to I see some CVS import requests that have been submitted and then closed during the past few days, so they should get to yours fairly quickly if you submit the request again. From itamarst at YAHOO.COM Mon Apr 23 13:39:10 2001 From: itamarst at YAHOO.COM (Itamar S.-T.) Date: Mon, 23 Apr 2001 04:39:10 -0700 Subject: [PYTHON-CRYPTO] Another amkCrypto/win32 issue Message-ID: <20010423113910.83574.qmail@web13003.mail.yahoo.com> In Windows, the resolution in time.time() is not as high as in Unix. As a result the use of the delta between time.time() calls as entropy in Crypto.Utils.randpool fails (the delta is always 0.0). The solution I found was to use time.clock() instead of time.time(). ===== Itamar Shtull-Trauring, itamar(at)shtull-trauring.org __________________________________________________ Do You Yahoo!? Yahoo! Auctions - buy the things you want at great prices http://auctions.yahoo.com/ From bram at GAWTH.COM Mon Apr 23 21:31:08 2001 From: bram at GAWTH.COM (Bram Cohen) Date: Mon, 23 Apr 2001 12:31:08 -0700 Subject: [PYTHON-CRYPTO] Python rijndael implementation Message-ID: I needed an implementation of rijndael which supported 256-bit block and key size, and it turned out the easiest way to do with was to port the java reference code to python, here it is - http://gawth.com/bram/rijndael.py This one also has a decent API. It would be nice if someone would make a crijndael as well which implements the same API. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From bram at GAWTH.COM Tue Apr 24 01:08:29 2001 From: bram at GAWTH.COM (Bram Cohen) Date: Mon, 23 Apr 2001 16:08:29 -0700 Subject: [PYTHON-CRYPTO] Python rijndael implementation In-Reply-To: Message-ID: The rijndael implementation I gave earlier has only a very minimal API. I did this because designing APIs for actual encryption modes is a bottomless tar pit likely to drown any attempt to make python crypto bindings. Fortunately, if the appropriate primitives are implemented in C, implementing modes in Python isn't a performance problem. To support this theory, I've created the following files - http://gawth.com/bram/binaryint.py http://gawth.com/bram/xor.py http://gawth.com/bram/rijndael.py Which are reasonably efficient (for Python) implementations of the basic primitives needed to create all standard encryption modes. My thesis is that if these libraries were implemented in C, then the overhead of the Python calls to them to implement a mode will be low enough to not matter for the vast majority of applications. To demonstrate this, I've created the following benchmark program - http://gawth.com/bram/times.py Which on my machine gives the following output - call benchmark 0.0503439903259 xor benchmark 3.68108105659 binaryint benchmark 9.96818590164 encrypt benchmark 17.5898689032 Which demonstrates that these operations are the bottlenecks behind encryption mode implementations. It appears that a Python mode implementation using C versions of these would spend most of it's time in the underlying C calls. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From rsalz at ZOLERA.COM Tue Apr 24 03:53:27 2001 From: rsalz at ZOLERA.COM (Rich Salz) Date: Mon, 23 Apr 2001 21:53:27 -0400 Subject: [PYTHON-CRYPTO] Python rijndael implementation References: Message-ID: <3AE4DC97.944A2D57@zolera.com> > It appears that a Python mode implementation using C versions > of these would spend most of it's time in the underlying C calls. The thing that kills you is (needless) buffer copies. In C it's pretty easy to declare a couple of buffers (often, on the stack) and use them repeatedly for the various chaining codes, e.g. It's hard to do that in python, at least in a way that's pythonic. :) As a result, there's PyString_FromStringAndSize calls, which are lots of malloc/memcpy's that you'd really rather avoid. I'd love to be proven wrong. /r$ From bram at GAWTH.COM Tue Apr 24 06:30:00 2001 From: bram at GAWTH.COM (Bram Cohen) Date: Mon, 23 Apr 2001 21:30:00 -0700 Subject: [PYTHON-CRYPTO] Python rijndael implementation In-Reply-To: <3AE4DC97.944A2D57@zolera.com> Message-ID: On Mon, 23 Apr 2001, Rich Salz wrote: > > It appears that a Python mode implementation using C versions > > of these would spend most of it's time in the underlying C calls. > > The thing that kills you is (needless) buffer copies. In C it's pretty > easy to declare a couple of buffers (often, on the stack) and use them > repeatedly for the various chaining codes, e.g. It's hard to do that in > python, at least in a way that's pythonic. :) Python's string immutability does introduce some expense, but I think not all that much - it forces an extra malloc/memcpy for each call to xor and encrypt, which means two extra malloc/memcpy calls per block, which I don't *think* is all that bad compared to the fourteen rounds of thorough bit twiddling done inside of the rijndael encrypt function. > > I'd love to be proven wrong. The only way to tell for sure is to benchmark both ways - I'll go write a CBC mode implementation, and if someone else does a C implementation of rijndael and xor we can benchmark it... At worst, doing modes in Python is 'not as fast as an optimized all C version' slow, rather than 'hope you've got a good book, it's about to start bit-twiddling in Python' slow. I'm hoping it incurs only a 25% or so extra hit, which I think would warrant doing it standard, and letting applications which *really* depend on crypto speed to do their own pure-C mode implementation. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From aarchiba at YAHOO.COM Tue Apr 24 16:11:31 2001 From: aarchiba at YAHOO.COM (Andrew Archibald) Date: Tue, 24 Apr 2001 10:11:31 -0400 Subject: [PYTHON-CRYPTO] Python rijndael implementation In-Reply-To: ; from bram@GAWTH.COM on Mon, Apr 23, 2001 at 09:30:00PM -0700 References: <3AE4DC97.944A2D57@zolera.com> Message-ID: <20010424101131.C7204@hedgehog> On Mon, Apr 23, 2001 at 09:30:00PM -0700, Bram Cohen wrote: > On Mon, 23 Apr 2001, Rich Salz wrote: > > > > It appears that a Python mode implementation using C versions > > > of these would spend most of it's time in the underlying C calls. > > > > The thing that kills you is (needless) buffer copies. In C it's pretty > > easy to declare a couple of buffers (often, on the stack) and use them > > repeatedly for the various chaining codes, e.g. It's hard to do that in > > python, at least in a way that's pythonic. :) > > Python's string immutability does introduce some expense, but I think not > all that much - it forces an extra malloc/memcpy for each call to xor and > encrypt, which means two extra malloc/memcpy calls per block, which I > don't *think* is all that bad compared to the fourteen rounds of thorough > bit twiddling done inside of the rijndael encrypt function. There's a security issue here, too: if you strew copies of plaintext and keys all over memory, it's almost certain to get swapped to disk. Now, secure memory is hard, even in C, but at least if you can burn buffers you have a little more hope. There's something to be said for an extremely heavyweight crypto API (like cryptix) where keys and encryption are all opaque blocks, perhaps in a hardware device, perhaps in another process (which has used mlock) which protects them from needless fiddling. That said, I think it's better for now to just implement this stuff in python. > > > > I'd love to be proven wrong. > > The only way to tell for sure is to benchmark both ways - I'll go write a > CBC mode implementation, and if someone else does a C implementation of > rijndael and xor we can benchmark it... xor you can do surprisingly fast with bytestolong and longtobytes or array: def xor(a,b): aa = array.array('i',a) bb = array.array('i',b) for i in range(len(aa)): bb[i] = bb[i] ^ aa[i] return bb.tostring() > At worst, doing modes in Python is 'not as fast as an optimized all C > version' slow, rather than 'hope you've got a good book, it's about to > start bit-twiddling in Python' slow. > > I'm hoping it incurs only a 25% or so extra hit, which I think would > warrant doing it standard, and letting applications which *really* depend > on crypto speed to do their own pure-C mode implementation. The crypto API should provide convenience functions for common modes, if only because most people (myself included) are likely to mess up implementing them. So rather than letting people write their own, it's probably easier to provide an implementation of that interface in C... when we get around to it. Andrew From bram at GAWTH.COM Thu Apr 26 04:20:45 2001 From: bram at GAWTH.COM (Bram Cohen) Date: Wed, 25 Apr 2001 19:20:45 -0700 Subject: [PYTHON-CRYPTO] Python rijndael implementation In-Reply-To: <20010424101131.C7204@hedgehog> Message-ID: On Tue, 24 Apr 2001, Andrew Archibald wrote: > xor you can do surprisingly fast with bytestolong and longtobytes or array: > > def xor(a,b): > aa = array.array('i',a) > bb = array.array('i',b) > for i in range(len(aa)): > bb[i] = bb[i] ^ aa[i] > return bb.tostring() I have now modified my xor.py to work that way - http://gawth.com/bram/xor.py I had to use 'B' instead of 'i' to make it work on arbitrary length strings, which made it a little bit slower. The overall speedup was about a factor of 2, but (I think) it's still vastly slower than a native C implementation would be. > The crypto API should provide convenience functions for common modes, if > only because most people (myself included) are likely to mess up > implementing them. It's very, very difficult to write crypto code for which you can honestly say 'here, use this, it will solve your problem'. I'll give cbc mode as an example when I'm done with it. We should attempt to have decent mode support, but realistically, even slightly novel crypto applications often need their own custom ones. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From jason-list-python-crypto at MASTALER.COM Thu Apr 26 17:44:34 2001 From: jason-list-python-crypto at MASTALER.COM (Jason R. Mastaler) Date: Thu, 26 Apr 2001 09:44:34 -0600 Subject: [PYTHON-CRYPTO] RFC: verifying e-mail addresses using HMACs Message-ID: Greetings, I'm working on an application that generates and verifies one-time "expirable" e-mail addresses using HMACs. I thought I'd run my methodology by the list and would appreciate any comments/suggestions on it. A 'dated' address is one that is valid only for a certain interval of time. Its format is: name-dated-$date.$datemac at domain.dom (For example, jason-dated-988298746.9d619c at mastaler.com) An incoming message with such a 'dated' address is accepted if: $date < currentdate AND $datemac == a new HMAC generated using $date as input. A 'sender' address is one that only a particular sender can use. Its format is: name-sender-$sendermac at domain.dom (For example, jason-sender-c12d9f6630f00645 at mastaler.com) An incoming message with such a 'sender' address is accepted if: $sendermac == a new HMAC generated using the sender's e-mail address as input. The following code illustrates how these addresses are generated using amkCrypto. ======================================================================= #!/usr/bin/env python from Crypto.Hash import HMAC from Crypto.Hash import SHA import binascii import time now = '%d' % time.time() hexkey = '0a7ba002d968c2c6a87c91c54ed68a15987cc546' key = binascii.unhexlify(hexkey) def make_datemac(time): datemac = HMAC.HMAC(SHA).hash(key,[time])[0] return binascii.hexlify(datemac[:3]) dated_address = 'jason-dated-' + now + '.' + make_datemac(now) print dated_address def make_sendermac(address): sendermac = HMAC.HMAC(SHA).hash(key,[address])[0] return binascii.hexlify(sendermac[:8]) sender_address = 'jason-sender-' + make_sendermac('jason at mastaler.com') print sender_address ======================================================================= Thanks. From bram at GAWTH.COM Fri Apr 27 03:07:12 2001 From: bram at GAWTH.COM (Bram Cohen) Date: Thu, 26 Apr 2001 18:07:12 -0700 Subject: [PYTHON-CRYPTO] Doing modes in Python In-Reply-To: Message-ID: To test my theory about times (and also because it's useful) I created the following implementation of Cipher Block Chaining with CipherText Stealing mode - http://gawth.com/bram/cbccts.py I further created the following timing code - http://gawth.com/bram/times2.py Which, when using Andrew Archibald's C rijndael implementation, gives the following info - full time 0.524374961853 time for xors 0.411889910698 time for encrypts and decrypts 0.0720390081406 time for buffer writes 0.0136380195618 In other words, the amount of time taken is dominated by the (still in Python) xor function, and of the remainder most of it is spent actually doing rijndael. I believe this supports my idea that once rijndael, xor, and binaryint are implemented in C, there isn't much pressing performance need to implement the modes in C as well. Also, I'd like to point you all to the documentation of cbccts.py, which documents some of the significant caveats that even this simple API to one of the most straightforward modes has. It demonstrates what a mess trying to finalize the API for modes now would be. CBC is only ankle-deep in tar, some of the others are into it up to their belly button. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From aarchiba at YAHOO.COM Fri Apr 27 07:30:39 2001 From: aarchiba at YAHOO.COM (Andrew Archibald) Date: Fri, 27 Apr 2001 01:30:39 -0400 Subject: [PYTHON-CRYPTO] Doing modes in Python In-Reply-To: ; from bram@GAWTH.COM on Thu, Apr 26, 2001 at 06:07:12PM -0700 References: Message-ID: <20010427013039.D9120@hedgehog> On Thu, Apr 26, 2001 at 06:07:12PM -0700, Bram Cohen wrote: > Also, I'd like to point you all to the documentation of cbccts.py, which > documents some of the significant caveats that even this simple API to one > of the most straightforward modes has. It demonstrates what a mess trying > to finalize the API for modes now would be. CBC is only ankle-deep in tar, > some of the others are into it up to their belly button. Ah, to illustrate your point, I think that the way you're encrypting short packets is not the only way to do it. A more consistent way would be to steal bits from the IV (although this requires changing the IV). I would say, though, that the standard modes are just that: standard. ECB, CBC (without stealing), n-bit CFB, OFB and Counter mode are all pretty standard, and they cover enough of the bases to be worth implementing in C. (You can also, for example, implement ciphertext stealing efficiently given only CBC and ECB mode). Andrew From bram at GAWTH.COM Sat Apr 28 04:13:10 2001 From: bram at GAWTH.COM (Bram Cohen) Date: Fri, 27 Apr 2001 19:13:10 -0700 Subject: [PYTHON-CRYPTO] Doing modes in Python In-Reply-To: <20010427013039.D9120@hedgehog> Message-ID: On Fri, 27 Apr 2001, Andrew Archibald wrote: > I would say, though, that the standard modes are just that: standard. > ECB, CBC (without stealing), n-bit CFB, OFB and Counter mode are all > pretty standard, and they cover enough of the bases to be worth > implementing in C. While I agree that having pure C versions of everything done eventually is a good idea, it's very tricky to get right, for example - How do you pad ECB? Do you reject strings of the wrong length? Do you pad with all zeros? If so, what do you do about losing information about the length of the file? How do you pad CBC? With counter mode, do you allow it to start at a counter other than 0? Do you make it big- or little-ending, or allow either? I don't know OFB and CFB very well, so I can't comment on them, but I believe they're quite tricky as well. There's also the matter of support stream-style CBC, so a whole file doesn't have to be pulled into memory at once, but I don't think that's required very often. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From aarchiba at YAHOO.COM Sat Apr 28 09:57:47 2001 From: aarchiba at YAHOO.COM (Andrew Archibald) Date: Sat, 28 Apr 2001 03:57:47 -0400 Subject: [PYTHON-CRYPTO] Doing modes in Python In-Reply-To: ; from bram@GAWTH.COM on Fri, Apr 27, 2001 at 07:13:10PM -0700 References: <20010427013039.D9120@hedgehog> Message-ID: <20010428035746.G9120@hedgehog> On Fri, Apr 27, 2001 at 07:13:10PM -0700, Bram Cohen wrote: > On Fri, 27 Apr 2001, Andrew Archibald wrote: > > > I would say, though, that the standard modes are just that: standard. > > ECB, CBC (without stealing), n-bit CFB, OFB and Counter mode are all > > pretty standard, and they cover enough of the bases to be worth > > implementing in C. > > While I agree that having pure C versions of everything done eventually is > a good idea, it's very tricky to get right, for example - > > How do you pad ECB? Do you reject strings of the wrong length? Do you pad > with all zeros? If so, what do you do about losing information about the > length of the file? > > How do you pad CBC? You don't pad. Users can add padding in python --- in fact, a good policy is leave in python anything that's O(1) in the length of the message. This includes ciphertext stealing, padding, arrangement of MAC/IV. > With counter mode, do you allow it to start at a counter other than 0? Do > you make it big- or little-ending, or allow either? Of course counter mode starts at any IV you provide; endianness can be selectable, but big-endian seems to be the standard for cyptographic "long" numbers. > I don't know OFB and CFB very well, so I can't comment on them, but I > believe they're quite tricky as well. CFB is about as standard as CBC, and it has the advantage that it handles arbitrary-length messages without any hackery. Representing the state is a little bit awkward (if you want to be able to stop and restart the C code in the middle of a block). OFB is simple and standard and nobody uses it. > There's also the matter of support stream-style CBC, so a whole file > doesn't have to be pulled into memory at once, but I don't think that's > required very often. Stream-style CBC is easy: just take the last block as your IV next time. Buffer one block (or two, if you're planning on stealing). But yes, a preliminary python implementation is worth doing and may well be fast enough for most people most of the time. Get one at http://www.math.mcgill.ca/archibal/crypto/modes.py Andrew From bram at GAWTH.COM Sat Apr 28 23:02:34 2001 From: bram at GAWTH.COM (Bram Cohen) Date: Sat, 28 Apr 2001 14:02:34 -0700 Subject: [PYTHON-CRYPTO] Doing modes in Python In-Reply-To: <20010428035746.G9120@hedgehog> Message-ID: On Sat, 28 Apr 2001, Andrew Archibald wrote: > You don't pad. Users can add padding in python --- in fact, a good > policy is leave in python anything that's O(1) in the length of the > message. This includes ciphertext stealing, padding, arrangement of > MAC/IV. That's a good idea, it simplifies things a lot. > But yes, a preliminary python implementation is worth doing and may > well be fast enough for most people most of the time. Get one at > http://www.math.mcgill.ca/archibal/crypto/modes.py I took a stab at a streaming counter mode implementation, it's at - http://gawth.com/bram/countermode.py I'm not done testing it yet, but I've gotta catch a plane, I'll be back with more in a week. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes