From phr-pycrypt at nightsong.com Wed Apr 3 07:32:01 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Wed, 3 Apr 2002 05:32:01 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020403053201.15125.qmail@brouhaha.com> [Bram] The clear winners in terms of popularity are CBC and counter, and ECB needs to be included for completeness. In search results, there are lots of mentions of the problems with CFB and suggestions for all sorts of funky variants of it. Also, while the API I suggested for using CFB is clearly the only reasonable one, it hardly gets any mention in the press, as people seem to greatly favor the moronic practice of doing a reduced block size for *every* encryption, as opposed to the reasonable but still funky practice of adjusting the block size dynamically to suit how the data is coming in. You have to understand, CFB (especially with reduced block sizes) generally is used for communications systems and not for something like files. Adjusting the block size dynamically wouldn't make sense, partly because the data usually comes at a constant rate, partly because you'd need some kind of synchronization mechanism to let the other end know how many encryptions you'd just done. Remember that doing eight 1-bit CFB operations is NOT the same as doing one 8-bit CFB operation and if you don't tell the receiver what you've done, it won't be able to decode the ciphertext. Setting the feedback size in the cipher object constructor is the right way to do it. If you're going to support all the other modes, omitting CFB is weird. A typical use of CFB might be something like an encrypted walkie talkie using a CVSD speech codec (i.e. a 1-bit speech coding scheme). You'd use 1-bit CFB to encrypt the voice stream and drive the radio modulator with the ciphertext at maybe 16 kbps. Because it's CVSD you don't have to worry about speech frames, and because of the 1-bit CFB, if you get data errors or the radio signal drops out for a second, you don't have to worry about resynchronization--CFB automatically resynchronizes itself after 1 encryption block (< 10 msec at 16 kbps). You do a lot more encryption operations than your data seems to need, but you get simplicity that isn't provided by any of the other modes. So there really isn't any substitute for CFB in these situations. I also think CBC MAC's should be supported, as mentioned before. Note that you want to encrypt and authenticate a message, you must NOT use the same key for the encryption and the MAC. > I think CTR mode should use the same encrypt/decrypt operation names > as the other modes, even if encrypt and decrypt in CTR mode do the > same thing, instead of process/processor. How about crypt/crypter? I don't understand why you want to call it something different. Encrypt/decrypt is fine even if they both do the same thing. Suppose you're writing a modular arithmetic package that supports addition and subtraction in base 5, base 7, base 10, etc. As it happens, addition and subtraction in base 2 are the same thing--does that mean your package shouldn't support both operations in base 2? It's not a big deal but I'd rather use encrypt/decrypt consistently. Even in CTR mode, in a variant with an integrity check, encrypt and decrypt won't do the same thing. [Eric] I would agree with this entirely. It seems to me that the algorithm should just implement ECB mode and then the modes of operation should be handled elsewhere. We could then use mix-ins or Factory classes to generate the actual cipher instance we want. What's "elsewhere"? I think it's better to submit just one extension module rather than a mess of them. The modes of operation should be included in the extension module because implementing them in Python is slow. I know you are only talking of implementing AES but we should try to be forward looking in doing this ... and I want to be able to easliy plug-in different algorithms. What algorithms do you want to use, other than AES? I'm not asking for a list of all algorithms that sound interesting, I'm asking which ones you ACTUALLY want to use. The same logic applies to modes of operation--I'm still having trouble understanding why we want to support all these modes. The same applies for the padding methods. If the algorithm forces the padding method it will almost certainly never support the one I need in a hurry. Which padding methods do you want to use, other than PKCS5? [Phr] I'd like to propose adding a cryptographic PRNG interface to the module, or alternatively making a separate module with a CPRNG. Either way, it's needed. It's nice if it's in the same module with AES since then the AES functions can use it to generate random keys and IV's. Implentation: - On Linux/xBSD, read random data from /dev/urandom - On Windows, call CAPI CryptGenRandom - On Solaris and similar, call the OpenSSL EGD (entropy gathering daemon) interface. The user must install this separately. - On other systems, punt. Don't try to provide a portable entropy source. That's too tricky. From bram at GAWTH.COM Wed Apr 3 08:55:02 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Tue, 2 Apr 2002 22:55:02 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020403053201.15125.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > A typical use of CFB might be something like an encrypted walkie > talkie using a CVSD speech codec (i.e. a 1-bit speech coding scheme). > You'd use 1-bit CFB to encrypt the voice stream and drive the radio > modulator with the ciphertext at maybe 16 kbps. Because it's CVSD you > don't have to worry about speech frames, and because of the 1-bit CFB, > if you get data errors or the radio signal drops out for a second, you > don't have to worry about resynchronization--CFB automatically > resynchronizes itself after 1 encryption block (< 10 msec at 16 kbps). Ah, I see. That ain't never gonna perform well implemented in python. I see your point about completeness though. How about I drop OFB as well, so we just have ECB, CBC, and counter? > I also think CBC MAC's should be supported, as mentioned before. Isn't CBC MAC trivial to build when you have CBC? > How about crypt/crypter? > > I don't understand why you want to call it something different. > Encrypt/decrypt is fine even if they both do the same thing. Okay, okay. Encrypt/decrypt it is. > [Eric] > > I would agree with this entirely. It seems to me that the > algorithm should just implement ECB mode and then the modes of > operation should be handled elsewhere. We could then use mix-ins > or Factory classes to generate the actual cipher instance we want. > > What's "elsewhere"? I think it's better to submit just one extension > module rather than a mess of them. The modes of operation should be > included in the extension module because implementing them in Python > is slow. Yes, that's my reasoning as well. > I know you are only talking of implementing AES but we should try to be > forward looking in doing this ... and I want to be able to easliy plug-in > different algorithms. > > What algorithms do you want to use, other than AES? I'm not asking > for a list of all algorithms that sound interesting, I'm asking which > ones you ACTUALLY want to use. There may be a call for 3DES at some point in the future. I think we could just make a 3DES module with the same API as aes though. > I'd like to propose adding a cryptographic PRNG interface to the > module, or alternatively making a separate module with a CPRNG. > Either way, it's needed. It's nice if it's in the same module with > AES since then the AES functions can use it to generate random keys > and IV's. Implentation: > > - On Linux/xBSD, read random data from /dev/urandom > - On Windows, call CAPI CryptGenRandom > - On Solaris and similar, call the OpenSSL EGD (entropy gathering > daemon) interface. The user must install this separately. > - On other systems, punt. Don't try to provide a portable entropy > source. That's too tricky. I think this is very warranted, but should be in a separate module. I'm considering making the API have actual processing objects, rather than functions, which have the following methods - write(s) - adds the string s to the stream read(amount = None) - reads amount bytes of plaintext/ciphertext. reads however much is possible if no amount is set. amount_read() - the number of bytes which have been read so far amount_ready() - the number of bytes which are ready to be read amount_buffered() - the difference between the number of bytes written and the number of bytes read so far. In counter mode this will always be the same as amount_ready() My reasoning is that this will get rid of a lot of unnecessary data copies, and also simplify using it quite a bit, since there won't have to be a lot of taking the output of the processor and shoving it into a cStringIO. Note that with this API, PKCS#5 is ridiculously easy to implement - d = enc.amount_buffered() - enc.amount_ready() if d == 0: d = 16 enc.write(chr(d) * d) and then you can read whatever's left out of enc. What does everyone think? -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From phr-pycrypt at nightsong.com Wed Apr 3 11:13:29 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Wed, 3 Apr 2002 09:13:29 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020403091329.18760.qmail@brouhaha.com> Ah, I see. That ain't never gonna perform well implemented in python. I see your point about completeness though. How about I drop OFB as well, so we just have ECB, CBC, and counter? I won't miss OFB. CBC should stay since it's what most systems use for encrypting multiple messages without needing to re-key between them. ECB should stay in case the application wants to synthesize other stuff with it. Do you really need CTR? Is there a good reason BitTorrent doesn't use CBC? Isn't CBC MAC trivial to build when you have CBC? Sort of, but if the plaintext is big, you end up allocating a lot of memory for the ciphertext just to discard it. Also, you want separate calls to compute and verify MAC's, to have an interface that can still work in systems with more key management (e.g. key objects that are allowed to check signatures but not to sign things). I guess this isn't a big deal though and maybe it just clutters up the module, so we can leave it out. > What algorithms do you want to use, other than AES? I'm not asking > for a list of all algorithms that sound interesting, I'm asking which > ones you ACTUALLY want to use. There may be a call for 3DES at some point in the future. I think we could just make a 3DES module with the same API as aes though. If we want 3DES, I'm leaning towards wanting in the same module as AES. For some reason I'd thought of a module as being compiled from a single .c file, but of course it isn't. So unless we really want to support a wide variety of ciphers for interoperability purposes, we can implement 3DES and AES and call it quits. (If we want a lot of ciphers we probably also want all the operation modes, etc.) Internally, of course, there should be some consistent interface to block ciphers, so that the operation modes would be implemented just once instead of multiple times. > I'd like to propose adding a cryptographic PRNG interface to the > module, or alternatively making a separate module with a CPRNG. I think this is very warranted, but should be in a separate module. I now tend to prefer having it in the same module as AES, for the same reason as above, but I can go either way. I'm considering making the API have actual processing objects, rather than functions, which have the following methods - So if you wanted to encrypt a string, would you say ciphertext = enc.read(enc.write(plaintext)) This all seems a little too abstract for me. However, if such an abstraction is really wanted, it should implement the Python i/o stream interface so you can print to it and stuff like that. From zooko at zooko.com Wed Apr 3 13:37:07 2002 From: zooko at zooko.com (Zooko) Date: Wed, 3 Apr 2002 03:37:07 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: Message from Bram Cohen of "Tue, 02 Apr 2002 22:55:02 PST." References: Message-ID: Bram Cohen > ... > What does everyone think? It sounds okay to me. In particular the processing objects is better than functions (although I suggest optional string -> string functions as well). I'm always wary of APIs that are designed "top down" without an implementation, of course. My personal plan for new crypto in Python remains to wrap Crypto++ with Boost and following the Crypto++ API as much as possible. After I've seen what that looks like I will have a better idea of what kind of AES API I would want to have in the standard library. Regards, Zooko --- zooko.com Security and Distributed Systems Engineering --- From zooko at zooko.com Wed Apr 3 13:41:27 2002 From: zooko at zooko.com (Zooko) Date: Wed, 3 Apr 2002 03:41:27 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: Message from Paul Rubin of "Wed, 03 Apr 2002 09:13:29 GMT." <20020403091329.18760.qmail@brouhaha.com> References: <20020403091329.18760.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > > Do you really need CTR? Is there a good reason BitTorrent > doesn't use CBC? I need CTR for a project I'm working on, not only for performance but for the technical reason that encryption and decryption are the same operation. Regards, Zooko --- zooko.com Security and Distributed Systems Engineering --- From sholden at HOLDENWEB.COM Wed Apr 3 14:29:23 2002 From: sholden at HOLDENWEB.COM (Steve Holden) Date: Wed, 3 Apr 2002 07:29:23 -0500 Subject: [PYTHON-CRYPTO] aes library References: Message-ID: <027001c1db0b$30bd9aa0$d200000a@COMPUTER> ----- Original Message ----- From: "Zooko" To: Sent: Wednesday, April 03, 2002 6:37 AM Subject: Re: aes library > Bram Cohen > > > ... > > What does everyone think? > > It sounds okay to me. In particular the processing objects is better than > functions (although I suggest optional string -> string functions as well). > > I'm always wary of APIs that are designed "top down" without an implementation, > of course. My personal plan for new crypto in Python remains to wrap Crypto++ > with Boost and following the Crypto++ API as much as possible. > > After I've seen what that looks like I will have a better idea of what kind of > AES API I would want to have in the standard library. > Ultimately, whatever code gets implemented will only find wider distribution if it's accompanied by (comprehensive and) comprehensible documentation, which is what I find is usually missing with crypto software. regards Steve From bram at GAWTH.COM Wed Apr 3 20:12:00 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Wed, 3 Apr 2002 10:12:00 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020403091329.18760.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > Do you really need CTR? Is there a good reason BitTorrent > doesn't use CBC? Yes. CBC has the property that you need an even block size to do an encryption/decryption step, which makes it *awful* for operating on streams. As a general rule, if you're encrypting files, you use CBC, if you're encrypting streams, you use counter. > Isn't CBC MAC trivial to build when you have CBC? > > Sort of, but if the plaintext is big, you end up allocating a lot of > memory for the ciphertext just to discard it. Also, you want separate > calls to compute and verify MAC's, to have an interface that can still > work in systems with more key management (e.g. key objects that are > allowed to check signatures but not to sign things). I guess this > isn't a big deal though and maybe it just clutters up the module, so > we can leave it out. We can have a discard() method in addition to the read() method to avoid unnecessary allocs when doing CBC MAC. > If we want 3DES, I'm leaning towards wanting in the same module as > AES. For some reason I'd thought of a module as being compiled from a > single .c file, but of course it isn't. So unless we really want to > support a wide variety of ciphers for interoperability purposes, we > can implement 3DES and AES and call it quits. We can do that by naming the classes AES_whatever and 3DES_whatever. I can see the advantage of being able to say 'all the cryptographic stuff goes here'. Although sha and sha256 are already in their own modules... I could go either way. > Internally, of course, there should be some consistent interface to > block ciphers, so that the operation modes would be implemented just > once instead of multiple times. It makes sense for that to be done in C. > I'm considering making the API have actual processing objects, rather than > functions, which have the following methods - > > So if you wanted to encrypt a string, would you say > > ciphertext = enc.read(enc.write(plaintext)) Not really. If you were just encrypting a single thing you'd use the single encrypt() call. If you're operating on a stream you might do that, but you'd probably do multiple write()s before doing the read(), to avoid having to do some equally obscure-looking operations on a cStringIO. Also, write() has no return value, so it looks like this - enc.write(part1) enc.write(part2) ciphertext = enc.read() Aside from being generally nicer to use, there will be very big performance improvements from this approach under some circumstances - many network applications spend most of their time doing string copies, and this approach reduces the number of those you have to do by 1. > However, if such an abstraction is really wanted, it should implement > the Python i/o stream interface so you can print to it and stuff like > that. Where is that documented? -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From bram at GAWTH.COM Wed Apr 3 20:15:15 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Wed, 3 Apr 2002 10:15:15 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: Message-ID: Zooko wrote: > I'm always wary of APIs that are designed "top down" without an implementation, > of course. I've written block cipher APIs at least three times. > My personal plan for new crypto in Python remains to wrap Crypto++ > with Boost and following the Crypto++ API as much as possible. I've wrapped part of Crypto++ before. If my memory serves, it has a very similar approach to the one I'm suggesting. My main objection to using Crypto++ at all is that it's a bloated pig of a C++ library. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From bram at GAWTH.COM Wed Apr 3 20:24:03 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Wed, 3 Apr 2002 10:24:03 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <027001c1db0b$30bd9aa0$d200000a@COMPUTER> Message-ID: Steve Holden wrote: > Ultimately, whatever code gets implemented will only find wider distribution > if it's accompanied by (comprehensive and) comprehensible documentation, > which is what I find is usually missing with crypto software. I'll happily provide usable sample code for all the common cases. That should be easy to do with CBC and counter modes, since their motivating cases are very common. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From phr-pycrypt at nightsong.com Thu Apr 4 02:08:37 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Thu, 4 Apr 2002 00:08:37 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020404000837.29897.qmail@brouhaha.com> [Note: if you reply to the list, please don't cc me your reply--I don't need to get two copies. Thanks.] [Zooko] > Do you really need CTR? Is there a good reason BitTorrent > doesn't use CBC? I need CTR for a project I'm working on, not only for performance but for the technical reason that encryption and decryption are the same operation. This is also true of OFB and sort of true of CFB. [Bram] Yes. CBC has the property that you need an even block size to do an encryption/decryption step, which makes it *awful* for operating on streams. As a general rule, if you're encrypting files, you use CBC, if you're encrypting streams, you use counter. Traditionally for streams, you'd use CFB, which is self-resynchronizing and doesn't need to be re-keyed so much. CTR wasn't used much with DES. It may be better with AES because of the larger block length. Anyway, it sounds like there's desire to keep CTR, so fine, let's keep it. We can have a discard() method in addition to the read() method to avoid unnecessary allocs when doing CBC MAC. This fancy OO interface doesn't seem like the right thing for a C extension module. Something like the SHA module interface (as we discussed before) could be ok. Each update operation would produce ciphertext output (possibly 0 bytes) and keep up to one partial plaintext block buffered, and the final() call would flush any buffered partial block. That way there's no magic buffer in the crypto object getting dynamically resized on each operation. enc.write(part1) enc.write(part2) ciphertext = enc.read() Aside from being generally nicer to use, there will be very big performance improvements from this approach under some circumstances - many network applications spend most of their time doing string copies, and this approach reduces the number of those you have to do by 1. I think the encryption and Python function calls will be much slower than any string copies. We could support Python buffer objects, but we're probably better off keeping things simple. We can do that by naming the classes AES_whatever and 3DES_whatever. I can see the advantage of being able to say 'all the cryptographic stuff goes here'. Although sha and sha256 are already in their own modules... I could go either way. I don't think there's a standard sha256 module, but SHA and MD5 do currently have separate modules. So I could go either way too (see below, though). The advantage of putting AES and DES in the same module is that the same code can implement the modes of operation without creating dependencies between modules. The other possible organization is three modules, a modes-of-operation module callable from python, and AES and DES modules that implement the raw block ciphers that the modes-of-operation module can use. > Ultimately, whatever code gets implemented will only find wider > distribution if it's accompanied by (comprehensive and) > comprehensible documentation, which is what I find is usually > missing with crypto software. I'll happily provide usable sample code for all the common cases. That should be easy to do with CBC and counter modes, since their motivating cases are very common. We get wide distribution if we can get the Python maintainers to put AES into the standard Python release, which is one reason to put everything in one module--it may be easier to get one module accepted than several. As for documentation, rather than try to turn the poor app developer into a cryptographer by explaining CBC mode and so forth, I think the most important thing is to have an extremely simple interface for the most obvious case, e.g. ciphertext = aes.encrypt(plaintext, key) plaintext = aes.decrypt(ciphertext, key) where key and plaintext are arbitrary strings. This however means the "encrypt" function should take care of all padding and IV generation, which means it needs good random numbers, which makes the encryption module depend on a CPRNG whether the CPRNG is in the same module or a separate one. Anyway, I think we're somewhat going around in circles at this point and we should implement something and see how we like it. Anyone want to meet up with me (San Francisco area) and start coding? From bram at GAWTH.COM Thu Apr 4 06:58:53 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Wed, 3 Apr 2002 20:58:53 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020404000837.29897.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > Traditionally for streams, you'd use CFB, which is self-resynchronizing > and doesn't need to be re-keyed so much. For net protocols self-resynching is irrelevant. I think the larger block length is very helpful with AES, since it means you don't have to re-key at all, although mostly there's more focus on counter now because it allows random access, which OFB and CBC do not (CBC allows it for reading, but not writing.) > CTR wasn't used much with DES. I'm not sure what 3DES modes we might want to support - is there even a canonical variant of it with canonical key format? The only mode I know of being used for it widely was CBC. If there isn't a good, standard least common denominator feature set for 3des then I say we drop it - it's just there for backwards compatibility, not something we want to encourage people to use. > Anyway, it sounds like there's desire to keep CTR, so fine, let's keep it. So we seem to have general agreement on CBC, ECB, and CTR modes, at least for aes. > This fancy OO interface doesn't seem like the right thing for a C > extension module. Something like the SHA module interface (as we > discussed before) could be ok. Each update operation would produce > ciphertext output (possibly 0 bytes) and keep up to one partial > plaintext block buffered, and the final() call would flush any > buffered partial block. That way there's no magic buffer in the > crypto object getting dynamically resized on each operation. The simplicity of that approach certainly does have much appeal, especially after I've spent some time thinking about what is involved in implementing the more sophisticated one. There should be a call to find out how much it's got buffered but unreturned regardless though, since it has to have that information anyway. > I think the encryption and Python function calls will be much slower > than any string copies. We could support Python buffer objects, but > we're probably better off keeping things simple. Okay then, simple it is. > The advantage of putting AES and DES in the same module is that the > same code can implement the modes of operation without creating > dependencies between modules. The other possible organization is > three modules, a modes-of-operation module callable from python, and > AES and DES modules that implement the raw block ciphers that the > modes-of-operation module can use. That makes some sense. DES can be added later though. > As for documentation, rather than try to turn the poor app developer > into a cryptographer by explaining CBC mode and so forth, I think the > most important thing is to have an extremely simple interface for the > most obvious case, e.g. > > ciphertext = aes.encrypt(plaintext, key) > plaintext = aes.decrypt(ciphertext, key) > > where key and plaintext are arbitrary strings. This however means the > "encrypt" function should take care of all padding and IV generation, > which means it needs good random numbers, which makes the encryption > module depend on a CPRNG whether the CPRNG is in the same module or a > separate one. That can be included as a simpleencrypt and simpledecryt functions written in pure Pyhton. Note that simpledecrypt will have to raise ValueError though. > Anyway, I think we're somewhat going around in circles at this point > and we should implement something and see how we like it. Anyone want > to meet up with me (San Francisco area) and start coding? I'm up for that. I think we actually are making some progress, albeit slowly. It occurs to me that the counter mode API should really use position rather than counter - counter of x is the same as position 16 * x. If the API is based on counter you just wind up making something based on position on top of it. Also, since one might want to do a few different operations using the same key, I think it makes sense to have an AES class with all the methods hanging off of it, to avoid unnecessary re-keying (and duplicate code in the implementation.) Here's my latest stab at the API (hey, it can't hurt to have this as worked out as possible before we sit down and hack) - AES.__init__(key) key can be of length 16, 24, or 32. AES.ECB_encryt(str) AES.ECB_decryt(str) Those do what you'd expect. AES.CTR_encrypt(str, pos = 0, bigendian = 1) AES.CTR_decrypt(str, pos = 0, bigendian = 1) These are the same function. pos is a counter of bytes, not the counter tally. In all cases, CTR_encrypt(s1, a) + CTR_encrypt(s2, len(s1)+a) = CTR_encrypt(s1 + s2, a) AES.CBC_encrypt(str, iv, padding = 1) AES.CBC_decrypt(str, iv, padding = 1) CBC_encrypt gets pissy if you give it padding = 0 and a str whose length isn't a multiple of 16. CBC_decrypt automatically strips padding. padding = 1 refers to pkcs5. padding values other than 0 and 1 are reserved for possible later use. AES.CBC_encrypter(pos = 0, bigendian = 1) AES.CBC_decrypter(pos = 0, bigendian = 1) These are the same function. They return an object with the following methods - setpos(pos) getpos() encrypt(str) decrypt(str) encrypt and decrypt above are the same function. AES.CBC_encrypter(iv) this returns an object with the following methods - encrypt(str) finish(padding = 1) amount_buffered() Note that it's possible to implement finish using the other two. AES.CBC_decrypter(iv) this returns an object with the following methods - decrypt(str) finish(padding = 1) amount_buffered() Note that, again, finish can be implemented using the other two, and also that finish might raise a ValueError. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From bram at GAWTH.COM Thu Apr 4 07:03:22 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Wed, 3 Apr 2002 21:03:22 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020404032210.1208.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > You guys know about this, right? > > http://sourceforge.net/projects/cryptkit/ I don't like it's API, although it's got some working Python->C hooks for a what I believe is a fast AES implementation (unlike the one optimized for ease of dealing with which I'm using.) > It's a successor to PyAES, which I had fooled around a while ago and > had forgotten about. It adds several modules: > > CryptKit is a small, fast cryptographic toolkit for python. It > implements Rijndael(AES), SHA 256, Elliptic Curve PKI, Diffie-Hellman > key exchange and Nyberg-Ruppel signature/verification. Comprehensive > enough to provide a secure socket alternative to SSL. I don't think there's a need for DH and the like to be in the standard libs, since we have built-in modular exponentiation, which is all you need to build them in pure Python. Those modules probably are handy though. > Is Brian Mongeau (the author) on this list? I dunno, he seems hard to reach - I spoke to him on IRC once, and have mailed him since, but haven't heard back... -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From bryan at EEVOLVED.COM Thu Apr 4 08:33:00 2002 From: bryan at EEVOLVED.COM (Bryan Mongeau) Date: Thu, 4 Apr 2002 01:33:00 -0500 Subject: [PYTHON-CRYPTO] aes library Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Is Brian Mongeau (the author) on this list? Yes, I'm here and I've been loosely following the recent thread about an AES API. I'm sorry to say that I have to also operate a business and as such cannot devote as much time as I'd like to improving my crypto lib. I haven't yet participated in the conversation because I have my doubts about strong encryption ever being included in the standard library. I also doubt that a single API could ever please everyone. As for my AES API, I initially designed it according to the policy of least surprise for the un-initiated, so I recognize that its current incarnation probably isn't as flexible as it ought to be for the power-user and as such I am willing to change it. Migrating from SWIG to native python is on my TODO list :) Some of you seem to be discussing the particular details of the primitives in the kit, so I will elaborate further: SHA 256/384/512 - - Swigged Aaron D. Gifford's ANSI C implementation. - - PEP 247-compliant API, with the exception of the .new() method, which I abhor. AES - - Swigged optimized ANSI C implementation by Paulo Barreto. - - ECB and CBC modes ECC ( still being developped, 80% complete ) - - Currently being revamped to support 8 different keysizes - - NIST curves up to 571 bits - - Based on Mike Rosing's ECC code, in polynomial basis. - - Optimized polynomial inversion routines. - - Uses internal C bignum lib, freely licensed. - - Key exchange is going to be configurable between Diffie-Hellman, Massey-Omura, ElGamal, Menezes-Qu-Vanstone(default) - - Signature/verification will switch to ECDSA, despite slower speed. Nyberg-Ruppel will still be available. As stated many times, the primary goal of my kit is to provide a smaller, easier, faster and more secure replacement for SSL network connections. Resultingly, the API I designed was intentionally simplistic. I would of course welcome developpers to help improve it and would grant CVS access if so desired. > I dunno, he seems hard to reach - I spoke to him on IRC once, and have > mailed him since, but haven't heard back... Are you Bram Cohen of Mojo Nation fame ? As for the unanswered email... umm... it must be my mail server that dropped it :-) Sorry. - -- Bryan Mongeau http://eevolved.com/bryan - -- "The facts will eventually test all our theories, and they form, after all, the only impartial jury to which we can appeal" -- Louis Agassiz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjyr85wACgkQ3SCd0lDF8naUigCfWMhruynOvEY4nFxoJG3bm4T3 VHMAmQGkZ3qaWER0azD7H85X30VZvOmB =77WA -----END PGP SIGNATURE----- From phr-pycrypt at nightsong.com Thu Apr 4 05:22:10 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Thu, 4 Apr 2002 03:22:10 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020404032210.1208.qmail@brouhaha.com> You guys know about this, right? http://sourceforge.net/projects/cryptkit/ It's a successor to PyAES, which I had fooled around a while ago and had forgotten about. It adds several modules: CryptKit is a small, fast cryptographic toolkit for python. It implements Rijndael(AES), SHA 256, Elliptic Curve PKI, Diffie-Hellman key exchange and Nyberg-Ruppel signature/verification. Comprehensive enough to provide a secure socket alternative to SSL. Is Brian Mongeau (the author) on this list? From phr-pycrypt at nightsong.com Thu Apr 4 07:36:43 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Thu, 4 Apr 2002 05:36:43 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020404053643.3735.qmail@brouhaha.com> [bram] It occurs to me that the counter mode API should really use position rather than counter - counter of x is the same as position 16 * x. If the API is based on counter you just wind up making something based on position on top of it. Hmm, this makes more sense in an interface that encrypts byte streams rather than strings. It becomes somewhat removed from the AES CTR definition, but I guess that's ok. Here's my latest stab at the API (hey, it can't hurt to have this as worked out as possible before we sit down and hack) - AES.__init__(key) key can be of length 16, 24, or 32. AES.ECB_encryt(str) AES.ECB_decryt(str) Those do what you'd expect. I thought we weren't going to try to mix modes for a single key, i.e. we'd have: a = AES.ECB(key) a.encrypt(str) a.decrypt(str) And for CTR mode you'd use a separate constructor: a = AES.CTR(key, bigendian = 1, pos = 0) initializes CTR key with internal position counter set to 0. a.encrypt(str) # advances position counter by len(str) a.decrypt(str) # likewise a.setpos(pos=0) # reset internal counter, for random access a.getpos() # return value of internal counter a.streamout(n) # same as a.encrypt('\0'*n) [Bram] AES.CBC_encrypt(str, iv, padding = 1) AES.CBC_decrypt(str, iv, padding = 1) CBC_encrypt gets pissy if you give it padding = 0 and a str whose length isn't a multiple of 16. CBC_decrypt automatically strips padding. padding = 1 refers to pkcs5. padding values other than 0 and 1 are reserved for possible later use. If padding==0 and len(str) is a multiple of 16, what happens to the IV? Here's an alternate interface: a = AES.CBC(key, iv, direction='e') # saves IV in aes object a.set_iv(iv) # change internal IV a.get_iv() # return old iv, don't change it a.update(str) # encrypts str and returns ciphertext, buffering # final partial input block and updating iv a.final(str) # encrypts any buffered partial block with str, # applying PKCS #5 padding The terms 'update' and 'final' are consistent with the hashing modules. For decryption you'd use direction='d' in the constructor. Should the IV be part of the ciphertext? [Bram] AES.CBC_encrypter(pos = 0, bigendian = 1) AES.CBC_decrypter(pos = 0, bigendian = 1) These are the same function. They return an object with the following methods - setpos(pos) getpos() encrypt(str) decrypt(str) I don't understand what setpos and getpos are supposed to do for CBC mode. I also don't see how the IV is supposed to fit into this. > You guys know about this, right? > > http://sourceforge.net/projects/cryptkit/ I don't like it's API, although it's got some working Python->C hooks for a what I believe is a fast AES implementation (unlike the one optimized for ease of dealing with which I'm using.) I've been looking at it a little more. It uses SWIG and we should fix it sometime to use the standard Python C API, but it's a reasonable starting point. I'd forgotten about the SWIG thing because last time I did anything with it, I didn't have SWIG and had to hack the SWIG-generated wrapper file directly :-P. > CryptKit is a small, fast cryptographic toolkit for > python. It implements Rijndael(AES), SHA 256, Elliptic Curve > PKI, Diffie-Hellman key exchange and Nyberg-Ruppel > signature/verification. Comprehensive enough to provide a > secure socket alternative to SSL. I don't think there's a need for DH and the like to be in the standard libs, since we have built-in modular exponentiation, which is all you need to build them in pure Python. Those modules probably are handy though. Bleah, it looks like it has its own bignum library. It uses Mike Rosing's ECC code and I'd want to make sure that stuff was free to copy before distributing it further. I like having ECC but would prefer characteristic p for some large p, rather than characteristic 2 even if it's slower. I think there may be patents on optimal normal base arithmetic as well :(. I agree that for regular DH, pure Python is enough--I wrote a library a while back that does it. For characteristic p elliptic curves, pure python might be ok. For characteristic 2, C code is needed. I don't understand what Nyberg-Ruppel signatures are. I'd rather use DSA and/or ECDSA. Paul From phr-pycrypt at nightsong.com Thu Apr 4 11:31:55 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Thu, 4 Apr 2002 09:31:55 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020404093155.6604.qmail@brouhaha.com> [Brian Mongeau] Yes, I'm here and I've been loosely following the recent thread about an AES API. I'm sorry to say that I have to also operate a business and as such cannot devote as much time as I'd like to improving my crypto lib. Ok, that's understandable--I'm glad you're here regardless. I haven't yet participated in the conversation because I have my doubts about strong encryption ever being included in the standard library. I'm fairly optimistic about getting an AES-based encryption module accepted, since they already have that silly rotor module. I worry more about getting a platform-dependent CPRNG accepted, and we need both. I also doubt that a single API could ever please everyone. As for my AES API, I initially designed it according to the policy of least surprise for the un-initiated, so I recognize that its current incarnation probably isn't as flexible as it ought to be for the power-user and as such I am willing to change it. Fair enough. If you're ok with it being the basis of a more general purpose AES module, I guess we can start hacking it. ECC ( still being developped, 80% complete ) Do you think that implementation is better (faster?) than a straightforward implementation in Python using Python long ints or gmpy, in characteristic p, maybe with projective coordinates? As stated many times, the primary goal of my kit is to provide a smaller, easier, faster and more secure replacement for SSL network connections. I just looked at the cryptsock.py code and immediately notice: 1) it uses cPickle to deserialize messages, which has a security hole (see current pickle docs) if the peer is hostile. 2) It appears to do an ECC signature on every message--why do that, if there's a shared secret key for the session? Just put a checksum underneath the secret-key encryption. However, this gets off the topic of AES. From akuchlin at mems-exchange.org Thu Apr 4 15:19:22 2002 From: akuchlin at mems-exchange.org (A.M. Kuchling) Date: Thu, 4 Apr 2002 08:19:22 -0500 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: ; from bryan@EEVOLVED.COM on Thu, Apr 04, 2002 at 01:33:00AM -0500 References: Message-ID: <20020404081922.A17791@mozart.mems-exchange.org> On Thu, Apr 04, 2002 at 01:33:00AM -0500, Bryan Mongeau wrote: >cannot devote as much time as I'd like to improving my crypto lib. I haven't >yet participated in the conversation because I have my doubts about strong >encryption ever being included in the standard library. I also doubt that a There's been some discussion on the Python Software Foundation's mailing list about the legalities required for export because of the rotor module, and there don't seem to be any legal problems with adding an AES module. (Whether GvR will care, I can't say.) --amk (www.amk.ca) The universe should be big enough for the both of us ... just. -- The Doctor, in "The Two Doctors" From bryan at EEVOLVED.COM Thu Apr 4 18:53:36 2002 From: bryan at EEVOLVED.COM (Bryan Mongeau) Date: Thu, 4 Apr 2002 11:53:36 -0500 Subject: [PYTHON-CRYPTO] aes library Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 [ Perhaps the Reply-To headers of this list ought to direct replies to the list rather than the sender? ] On Thursday 04 April 2002 04:31 am, you wrote: > I'm fairly optimistic about getting an AES-based encryption module > accepted, since they already have that silly rotor module. I worry > more about getting a platform-dependent CPRNG accepted, and we need both. What about platform-independent CPRNG's we could implement in pure python? Like high-resolution timers and thread races that get their entropy from system load? > Fair enough. If you're ok with it being the basis of a more general > purpose AES module, I guess we can start hacking it. Great. Do you think CVS would be needed? Who will be collaborating? > Do you think that implementation is better (faster?) than a > straightforward implementation in Python using Python long ints or > gmpy, in characteristic p, maybe with projective coordinates? To be honest with you Paul, I couldn't tell you if a pure python implementation would be faster or not. I can guess that a GF2^N polynomial basis implementation in python would be slower and an ONB or GF(p) implementation might be faster, depending mostly on the speed of python's long int code. When I started this Cryptkit thing a year ago, I really didn't have the time, nor the mathematics needed to implement in pure python, although I have thought about it. However, now it seems like an interesting course of action, especially with projective coordinates :-) > I just looked at the cryptsock.py code and immediately notice: > > 1) it uses cPickle to deserialize messages, which has a security > hole (see current pickle docs) if the peer is hostile. Yes, this is a problem I am aware of. There are other problems as well if you keep looking :-) > 2) It appears to do an ECC signature on every message--why do that, > if there's a shared secret key for the session? Just put a > checksum underneath the secret-key encryption. Very good point, thank you Paul. Looking forward to working out an AES API... - - -- Bryan Mongeau http://eevolved.com/bryan - - -- "The only thing that interferes with my learning is my education."-- Einstein -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iEUEARECAAYFAjyshRAACgkQ3SCd0lDF8nY/HACY1gcy1b4Qq0WmP8psWdE44OfK /gCbBUnGGsCr7hrgf7h/Qok1QfN+buE= =ZtWU -----END PGP SIGNATURE----- From akuchlin at mems-exchange.org Thu Apr 4 20:09:33 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Thu, 4 Apr 2002 13:09:33 -0500 Subject: [PYTHON-CRYPTO] pycrypto revisions Message-ID: Just a notice for anyone who's interested: I've resumed updating the code for the pycrypto SF project, intending to make a 1.9alpha release some time in the next month. (People on the pycrypto-checkins mailing list will have noticed a bunch of messages in the last few days.) So far I've updated and simplified the build process, dropped a bunch of obscure algorithms, and added AES. I still need to read more of the C code carefully and clean it up; after that, the Python code in Crypto.Util and the documentation will be my next concern, and then an alpha1 release can be made. Feel free to grab the code from the CVS repository if you want to try it out or offer suggestions for other things that should be cleaned up. --amk (www.amk.ca) "There can be no question, my dear Watson, of the value of exercise before breakfast." -- Sherlock Holmes, in "The Adventure of Black Peter" From bryan at EEVOLVED.COM Thu Apr 4 21:14:42 2002 From: bryan at EEVOLVED.COM (Bryan Mongeau) Date: Thu, 4 Apr 2002 14:14:42 -0500 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020404081922.A17791@mozart.mems-exchange.org> References: <20020404081922.A17791@mozart.mems-exchange.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 04 April 2002 08:19 am, A.M. Kuchling wrote: > and there don't seem to be any legal problems with > adding an AES module. What about including other modules in the standard lib, such as RSA or ECC? Will it be possible to distribute Python with full, strong crypto out-of-the-box? - -- Bryan Mongeau http://eevolved.com/bryan - -- "People are stunned to hear that one company has data files on 185 million Americans." -- Ralph Nader -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjyspiIACgkQ3SCd0lDF8nbxaACfZS0/XcQAdDlIpuMAKFceXEZW VsQAn1jF8aQiuWR1ez5JfsJLTyQDkfMS =/yiU -----END PGP SIGNATURE----- From akuchlin at mems-exchange.org Thu Apr 4 21:31:05 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Thu, 4 Apr 2002 14:31:05 -0500 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: References: <20020404081922.A17791@mozart.mems-exchange.org> Message-ID: <20020404193105.GA1620@crystal.mems-exchange.org> On Thu, Apr 04, 2002 at 02:14:42PM -0500, Bryan Mongeau wrote: >-----BEGIN PGP SIGNED MESSAGE----- >What about including other modules in the standard lib, such as RSA or ECC? >Will it be possible to distribute Python with full, strong crypto >out-of-the-box? It seems legally possible to add crypto code. The only thing to worry about is whether the python-dev community will think that it's worth doing; send an e-mail to the mailing list and raise the issue. --amk (www.amk.ca) Mediocrity knows nothing higher than itself; but talent instantly recognizes genius ... -- Watson, in "The Valley of Fear" From bram at GAWTH.COM Thu Apr 4 21:54:24 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Thu, 4 Apr 2002 11:54:24 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020404053643.3735.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > I thought we weren't going to try to mix modes for a single key, i.e. > we'd have: > > a = AES.ECB(key) > a.encrypt(str) > a.decrypt(str) That's the approach I gave before, I figured I'd float this other one as a possibility - I don't have a strong preference either way, although if you're going to do both CBC and ECB operations, for example, the approach I just gave avoids multiple key setupds. > And for CTR mode you'd use a separate constructor: > > a = AES.CTR(key, bigendian = 1, pos = 0) > > initializes CTR key with internal position counter set to 0. > > a.encrypt(str) # advances position counter by len(str) > a.decrypt(str) # likewise > a.setpos(pos=0) # reset internal counter, for random access > a.getpos() # return value of internal counter > a.streamout(n) # same as a.encrypt('\0'*n) Argh, I cut and pasted wrong, so I mixed up the API for CBC and counter, which of course made them gibberish. The above is almost identical to the methods I meant to post for counter objects, except that I didn't include a streamout (although I can see it's utility, I don't have a strong opinion about it). > AES.CBC_encrypt(str, iv, padding = 1) > AES.CBC_decrypt(str, iv, padding = 1) > > CBC_encrypt gets pissy if you give it padding = 0 and a str whose length > isn't a multiple of 16. CBC_decrypt automatically strips padding. padding > = 1 refers to pkcs5. padding values other than 0 and 1 are reserved for > possible later use. > > If padding==0 and len(str) is a multiple of 16, what happens to the IV? The same thing that always happens to the IV - padding just has to do with what's at the ending. In PKCS5 it's done by essentially the following trick - p = len(str) % 16 if p == 0: p = 16 str += chr(p) * p With 'no' padding it just raises a ValueError if len(str) % 16 isn't 0. > Here's an alternate interface: > > a = AES.CBC(key, iv, direction='e') # saves IV in aes object > a.set_iv(iv) # change internal IV > a.get_iv() # return old iv, don't change it > > a.update(str) # encrypts str and returns ciphertext, buffering > # final partial input block and updating iv > > a.final(str) # encrypts any buffered partial block with str, > # applying PKCS #5 padding reusing iv's is such a bad idea that I think having set_iv and get_iv is inadvisable, since it doesn't have any performance benefit when you're not reusing iv's, and encourages very bad behavior. 'update' and 'final' I can go with, although 'final' should take an optional padding = 1. I don't like the direction call at all - encryption and decryption are two completely different operations, likely not using any shared code, and using direction='d' is vastly more obscure than calling the constructor of CBC_decrypter. > Should the IV be part of the ciphertext? No, if the user wants to prepend the IV they can do that themselves. > [Bram] > > AES.CBC_encrypter(pos = 0, bigendian = 1) > AES.CBC_decrypter(pos = 0, bigendian = 1) That should have been CTR_encrypter and CTR_decrypter, which makes the following make a lot more sense - > These are the same function. They return an object with the following > methods - > > setpos(pos) > getpos() > encrypt(str) > decrypt(str) (see comments above) > I agree that for regular DH, pure Python is enough--I wrote a library > a while back that does it. For characteristic p elliptic curves, > pure python might be ok. For characteristic 2, C code is needed. > > I don't understand what Nyberg-Ruppel signatures are. I'd rather use > DSA and/or ECDSA. Is it currently possible to implement DSA in pure Python? I'd like the minimum base functions put into the standard libs which make basic strong crypto possible without resorting to C code. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From tdi01fhu at SYD.KTH.SE Fri Apr 5 02:13:30 2002 From: tdi01fhu at SYD.KTH.SE (FREDRIK HULDTGREN) Date: Fri, 5 Apr 2002 02:13:30 +0200 Subject: [PYTHON-CRYPTO] aes library Message-ID: Pardon my ignorance as I am rather new to this list, and even though my interest in cryptology is quite large, I don't really have a grip on AES in the way the way that I would want to have(One of the major reasons for me joining this list). Recent mails have included a whole bunch of abbreviations which I have absolutely no clue about, anyone feel like posting a link or a post explaining some of these to me? (CBC, CTR, etc...) :=) /Fredrik From phr-pycrypt at nightsong.com Fri Apr 5 03:52:06 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Fri, 5 Apr 2002 01:52:06 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020405015206.18425.qmail@brouhaha.com> Pardon my ignorance as I am rather new to this list, and even though my interest in cryptology is quite large, I don't really have a grip on AES in the way the way that I would want to have(One of the major reasons for me joining this list). Recent mails have included a whole bunch of abbreviations which I have absolutely no clue about, anyone feel like posting a link or a post explaining some of these to me? (CBC, CTR, etc...) :=) ECB, CBC, CFB and OFB are described here: http://www.iks-jena.de/mitarb/lutz/security/cryptfaq/q82.html http://www.iks-jena.de/mitarb/lutz/security/cryptfaq/q83.html CTR is done by just encrypting the blocks 0001, 0002, 0003, ... and xoring the resulting stream against the plaintext stream. It has the advantage of being very simple, and not needing special padding if you want to encrypt a plaintext of 23 bytes or something like that. Its disadvantages are that you must never re-use a key, and it is somewhat more subject to ciphertext modification attacks than other modes. Flipping a single bit in the ciphertext results in flipping the same bit in the plaintext without disturbing the surrounding bits. From phr-pycrypt at nightsong.com Fri Apr 5 03:57:16 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Fri, 5 Apr 2002 01:57:16 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020405015716.18532.qmail@brouhaha.com> [Brian] > I'm fairly optimistic about getting an AES-based encryption > module accepted, since they already have that silly rotor > module. I worry more about getting a platform-dependent CPRNG > accepted, and we need both. What about platform-independent CPRNG's we could implement in pure python? Like high-resolution timers and thread races that get their entropy from system load? I don't know what to think of these. They're slow and I'm skeptical of their security. I think it's important to provide an interface to system RNG's (almost all boxes running Python have system RNG's) I wouldn't want a "portable" one to lead someone into thinking the system one was redundant. Have you tested your thread racing implementation much? How do the results look? > Fair enough. If you're ok with it being the basis of a more general > purpose AES module, I guess we can start hacking it. Great. Do you think CVS would be needed? Who will be collaborating? Sounds like we're all collaborating ;-). I don't think CVS access to the cryptkit repository is needed, if that's what you're asking. This would be a different module than cryptkit--it would just share some code. For the initial implementation I think someone (maybe Bram and me) should just meet up and bang out the code, and circulate it for testing informally via the list. After that, we can put it up on sourceforge with its own CVS. > and there don't seem to be any legal problems with > adding an AES module. What about including other modules in the standard lib, such as RSA or ECC? Will it be possible to distribute Python with full, strong crypto out-of-the-box? I think we'll have to ask the Python maintainers what they'd accept. For me, the main thing is to get AES and a CPRNG in. RSA or DH can easily be done in pure Python. ECC doesn't seem that important, and in GF(p) it's probably ok to use pure Python. No offense intended but I'm opposed to submitting cryptsock in anything like its present form. [Please don't cc: me responses, I get them thru the list and don't want 2 copies --thanks]. From phr-pycrypt at nightsong.com Fri Apr 5 05:08:50 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Fri, 5 Apr 2002 03:08:50 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020405030850.19718.qmail@brouhaha.com> [Bram] > I thought we weren't going to try to mix modes for a single key, i.e. > we'd have: > > a = AES.ECB(key) > a.encrypt(str) > a.decrypt(str) That's the approach I gave before, I figured I'd float this other one as a possibility - I don't have a strong preference either way, although if you're going to do both CBC and ECB operations, for example, the approach I just gave avoids multiple key setupds. I strongly prefer separate objects per mode. Mixing modes probably leads to security failures. > a.streamout(n) # same as a.encrypt('\0'*n) The above is almost identical to the methods I meant to post for counter objects, except that I didn't include a streamout (although I can see it's utility, I don't have a strong opinion about it). I think it's useful, but don't like the name streamout, and wonder if anyone can suggest an alternative. reusing iv's is such a bad idea that I think having set_iv and get_iv is inadvisable, since it doesn't have any performance benefit when you're not reusing iv's, and encourages very bad behavior. The idea isn't to re-use iv's, but rather, to be able to encrypt a single message in multiple pieces if for some reason you don't want to keep the cipher object open between pieces. Normally you wouldn't use these. I don't like the direction call at all - encryption and decryption are two completely different operations, likely not using any shared code, and using direction='d' is vastly more obscure than calling the constructor of CBC_decrypter. They'd share code. A direction flag is how most other systems do it, and no one has complained. I don't see any big philosophical difference, and fewer different constructors makes the implementation simpler, so I'd rather stay with the direction flag. Is it currently possible to implement DSA in pure Python? I'd like the minimum base functions put into the standard libs which make basic strong crypto possible without resorting to C code. The math part of DSA is easy to implement in Python, but DSA requires secure random numbers. So once again, the CPRNG is the obstacle, not the algorithm. Btw, is anyone here good at Windows C programming and have dev tools? Also, is anyone here on Python-dev? We should send them an email asking what kind of module they'd accept. I propose something along these lines: 1) We think the "Cryptographic Services" part of the library should include a good encryption algorithm to go along with MD5 and SHA. The rotor module is of questionable security and should be deprecated. So we'd like to submit a module that does encryption based on AES, for inclusion in future Python releases. It would have an easy-to-use interface like the rotor module, and also provide an API that allowed more fine-grained control of the algorithm (modes of operation, etc.) for users who want that. This module would be written in portable C. 2) We'd also like to submit a CPRNG module. Secure random numbers are central to many security algorithms and protocols, and CPRNG's are included in most current OS instances. The lack of a CPRNG is a serious deficiency in the Python library. Unfortunately, it's not possible to write a good CPRNG portably. So we'd like to submit a CPRNG module with platform-dependent code that calls the OS-provided CPRNG's for Linux/BSD (apparently including Mac OS 10) and for Windows. This should cover most of the boxes currently running Python. For other platforms we may be able to supply a sort-of-useable portable CPRNG that uses thread racing, but it should be used only as a fallback measure. (This is sort of like the portable implementation of java.security.secureRandom, and anyway not all platforms provide threads). However, even if we can't support all platforms, we still urge shipping a CPRNG for those platforms that can support it. 3) For interoperability with older systems, we'd like to also support DES/3DES (DES is the predecessor of AES). However this is lower priority for us than #1 and #2. 4) We'd like to know the opinion of the Python maintainers about this stuff: how big an issue is OS dependence for #3? Should we make separate modules for AES, DES, and CPRNG, or combine them all into one? Is it ok if the AES/DES modules have a number of modes and entry points that will interest only advanced users? From bryan at EEVOLVED.COM Fri Apr 5 18:01:55 2002 From: bryan at EEVOLVED.COM (Bryan Mongeau) Date: Fri, 5 Apr 2002 11:01:55 -0500 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020405015716.18532.qmail@brouhaha.com> References: <20020405015716.18532.qmail@brouhaha.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 04 April 2002 08:57 pm, Paul Rubin wrote: > Have you tested your thread racing implementation much? How do the > results look? Well, it tops out at ~300 bytes/sec on my 800 and is solidly indeterministic. One time I ran it for 3 days and passed the output through DIEHARD. As I understood the p-values, the more even the distribution ( from 0.0 to 1.0 ) the better. An average of all 234 p-values collected yielded 0.531609. - -- Bryan Mongeau http://eevolved.com/bryan - -- "The most incomprehensible thing about the world is that it is comprehensible."-- Einstein -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjytynMACgkQ3SCd0lDF8naIbQCeJXiEYgQP2dMq4+/pWAaCs2jy hXoAnR0EZ5PZhNfPiXQ44uo47EVkogt6 =XHON -----END PGP SIGNATURE----- From Eric.Johnson at GDAI.COM Sat Apr 6 02:50:29 2002 From: Eric.Johnson at GDAI.COM (Eric Johnson) Date: Fri, 5 Apr 2002 19:50:29 -0500 Subject: [PYTHON-CRYPTO] aes library Message-ID: Some comments on the recent discussions... [Paul] > The rotor module is of questionable security and should be deprecated. I think "questionable" is somewhat understating things! Does anyone actually use it? [Paul (in different emails)] > What algorithms do you want to use, other than AES? I'm not asking > for a list of all algorithms that sound interesting, I'm asking which > ones you ACTUALLY want to use. The same logic applies to modes of > Which padding methods do you want to use, other than PKCS5? > 3) For interoperability with older systems, we'd like to also support > DES/3DES (DES is the predecessor of AES). However this is lower > priority for us than #1 and #2. [Bram] > If there isn't a good, standard least common denominator feature set for > 3des then I say we drop it - it's just there for backwards compatibility, > not something we want to encourage people to use. I have yet to use AES. I use DES and DES3 daily. The banking industry uses DES/DES3. I need to test and generate data used in smartcards for the telecom, government and banking industries. A python crypto module MUST support DES/DES3 to be of any use to me. I have had to use and implement PKCS#5 padding once (for a project 5 years ago). The padding methods I use daily are from ISO/IEC 9797-1. I have never used any modes of operation other than ECB and CBC. [Paul] > What's "elsewhere"? I think it's better to submit just one extension > module rather than a mess of them. The modes of operation should be > included in the extension module because implementing them in Python > is slow. > This fancy OO interface doesn't seem like the right thing for a C > extension module. > The advantage of putting AES and DES in the same module is that the > same code can implement the modes of operation without creating > dependencies between modules. The other possible organization is > three modules, a modes-of-operation module callable from python, and > AES and DES modules that implement the raw block ciphers that the > modes-of-operation module can use. [Bram] > That makes some sense. DES can be added later though. I agree that we should have the algorithms (ie just the actual block cipher) implemented as a C extension. I personally don't encipher/decipher megabytes of data but I'd have thought Python was efficient enough to do the the modes of operation but if not then that should be an extension module as well. I think secure randoms, hash functions (RIPEMD160 etc) and PK stuff should be in different modules. MACcing should be in the block cipher module. I think the C extension should have a simple interface and the wrapper code in Python (similar to the _tkinter / Tkinter modules) should present a nice high level OO interface. It could also expose a low level one if needed. I believe that factory methods are the way to go so that new extension modules with new ciphers can be added without changing the external interface. Similarly for modes of operation. That is I personally think something like aes = MakeBlockCipher(AES, CBC).setKey(key) aes.setIV(iv) aes.encipher(x) is better than either AES.CBC_encrypt(str, iv, padding = 1) or aes = AES.CBC_encrypt(key) Of course, the MakeBlockCipher method would just invoke something like aes = AES.CBC() or aes = AES.CBC_encrypt(key) so people don't need to use the abstraction layer if they are philosophically opposed to OO. I want Factory methods so I can easily use algorithms (mainly for authentication) that cannot be made public due to NDAs and don't have to rewrite my code all the time. Finally, I think we should have pure Python implementations as a fallback in case the C Extension is not available. I also need this because I sometimes need intermediate data for testing implementations or for cryptanalytic purposes and this is much easier in Python than a C implementation. I am sure we can agree on an interface or combination of interfaces that will be agreeable to most people. Sorry this went on so long. Eric From bram at GAWTH.COM Sun Apr 7 04:01:57 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Sat, 6 Apr 2002 18:01:57 -0800 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020405030850.19718.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > I strongly prefer separate objects per mode. Mixing modes probably > leads to security failures. True enough. Separate modes it is. > The idea isn't to re-use iv's, but rather, to be able to encrypt a > single message in multiple pieces if for some reason you don't want > to keep the cipher object open between pieces. Normally you wouldn't > use these. One can simply create a new cipher object in that case. > Also, is anyone here on Python-dev? We should send them an email > asking what kind of module they'd accept. I propose something along > these lines: > > 1) We think the "Cryptographic Services" part of the library should > include a good encryption algorithm to go along with MD5 and > SHA. The rotor module is of questionable security and should be > deprecated. So we'd like to submit a module that does > encryption based on AES, for inclusion in future Python > releases. > > It would have an easy-to-use interface like the rotor module, > and also provide an API that allowed more fine-grained control > of the algorithm (modes of operation, etc.) for users who want > that. This module would be written in portable C. That's emphasizing it's replacing of rotor rather that it's utility as a library, which I think is more important. I have no sense of which is more likely to sell. > 4) We'd like to know the opinion of the Python maintainers about > this stuff: how big an issue is OS dependence for #3? Should we > make separate modules for AES, DES, and CPRNG, or combine them all > into one? Is it ok if the AES/DES modules have a number of modes > and entry points that will interest only advanced users? The AES calls could easily be used as part of libraries called by non-'advanced' users. They make it so that crypto stuff in general can be in pure Python. Bryan Mongeau wrote: > > Have you tested your thread racing implementation much? How do the > > results look? > > Well, it tops out at ~300 bytes/sec on my 800 and is solidly > indeterministic. That makes it a good punt, but it's still a punt - a system could easily be rebooted and give the same 'random' values twice in a row. Best to use the system-specific source of 'real' random numbers whenever possible. Eric Johnson wrote: > I have yet to use AES. I use DES and DES3 daily. The banking industry > uses DES/DES3. Ah, we have reason to include that then. > I have had to use and implement PKCS#5 padding once (for a project 5 > years ago). The padding methods I use daily are from ISO/IEC 9797-1. Where is that documented? We can easily add it as another padding number. > I have never used any modes of operation other than ECB and CBC. Okay then, no counter mode for the DES module. I'm not sure what to do with the DES/3DES distinction in the API. There are a few possibilities - have all calls have a flag to indicate des/3des have parallel calls for des/3des have different modules for des/3des > I think secure randoms, hash functions (RIPEMD160 etc) and PK stuff > should be in different modules. MACcing should be in the block cipher > module. That's my feeling as well, based on how other modules are organised, specifically sha and md5 being in their own modules. > I want Factory methods so I can easily use algorithms (mainly for > authentication) that cannot be made public due to NDAs and don't have to > rewrite my code all the time. That can easily be implemented as a higher layer of abstraction in pure Python. I sincerely hope that NDA's aren't the banking industry's way of not admitting that they're using 1DES. Okay, here's the latest version of the API, I think it's non-asymptotically approaching being finished - ECB(key) encrypt(str) decrypt(str) ECB is simple, straightforward, and non-controversial. I think we have consensus that this is the right way to do it. CTR(key, pos = 0, bigendian = 1) encrypt(str) decrypt(str) setpos(pos) getpos() scan(amount) I realized that having non-incremental calls to CTR is a bit silly, so this is the simplified version. The only call here I'm a bit unsure about is scan() - it's the same as encrypt(chr(0) * amount) - I'm not sure if it's needed, and if it is, I don't know if I like the name. CBC(key) encrypt(str, iv, padding = 1) decrypt(str, iv, padding = 1) encrypter(iv) decrypter(iv) The tricky bit here is padding, it can have one of these values - 0 - none 1 - PKCS#5 2 - ISO 9797-1 I'm not sure which is the default value and whether there should be any more padding modes in the first version. I'm also not sure which one should be the default. The objects returned by encrypter() and decrypter() have the following mehtods - update(str) finish(padding = 1) amount_buffered() I prefer finish() instead of final() unless there's something else which uses final. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From akuchlin at mems-exchange.org Mon Apr 8 02:57:12 2002 From: akuchlin at mems-exchange.org (A.M. Kuchling) Date: Sun, 7 Apr 2002 20:57:12 -0400 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020405015716.18532.qmail@brouhaha.com>; from phr-pycrypt@NIGHTSONG.COM on Fri, Apr 05, 2002 at 01:57:16AM -0000 References: <20020405015716.18532.qmail@brouhaha.com> Message-ID: <20020407205711.B22142@mozart.mems-exchange.org> On Fri, Apr 05, 2002 at 01:57:16AM -0000, Paul Rubin wrote: >Sounds like we're all collaborating ;-). I don't think CVS access to >the cryptkit repository is needed, if that's what you're asking. ... >After that, we can put it up on >sourceforge with its own CVS. It's possible to create multiple modules within a CVS repository; for example, Python has the Distutils as a separate module, and PyXML has xml, www, test, and sandbox modules. IMHO it's better to have multiple modules within one repository, rather than scatter code among several repositories. In particular, having a single checkins list means that changes to module A get proofread by more people, because the developers of module B are also on the -checkins list. (This happens a fair bit with Python, where bugs in checkins are caught by people who don't maintain the code in question, but skimmed the patch anyway.) --amk (www.amk.ca) This supports reflection, which is the 90s way of writing self-modifying code. -- John Aycock at IPC7, during his parsing talk From akuchlin at mems-exchange.org Wed Apr 10 17:53:38 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Wed, 10 Apr 2002 11:53:38 -0400 Subject: [PYTHON-CRYPTO] randpool.getBytes(self,N) bug if N < 20 bytes in amkCrypto 0.1.3 . In-Reply-To: References: Message-ID: <20020410155337.GA13160@crystal.mems-exchange.org> On Wed, Apr 10, 2002 at 04:28:02PM +0200, Adam Fleming wrote: > self.entropy=-8*N Thanks for catching this! This is the only change in Adam's code; the original line was: self.entropy=self.entropy-8*num Which is wrong, as you point out. I think your suggested correction should really be to just change 'num' to 'N', and have applied this to CVS. (Anyone want to write a test suite for randpool.py to catch things like this?) --amk From Afleming at ES-NET.CO.UK Wed Apr 10 16:28:02 2002 From: Afleming at ES-NET.CO.UK (Adam Fleming) Date: Wed, 10 Apr 2002 16:28:02 +0200 Subject: [PYTHON-CRYPTO] randpool.getBytes(self,N) bug if N < 20 bytes in amkCrypto 0.1.3 . Message-ID: I was just experimenting with the library when this issue came up, but if you call GetBytes with N < 20, the result increases the value of entropy, which is clearly daft. The problem is that dsize is larger than num when subtracted from it, so when num is subtracted from self.entropy it increases. The following is an alteration to the routine to allow less than 20 byte reads, I expect your flames shortly over this issue. : def getBytes(self,N) : s='' i, pool = self._getPos, self._randpool h=self.hash.new() dsize=self.hash.digestsize num = N while num>0 h.update(self._randpool[i:i+dsize]) s=s+h.digest() num=num-dsize i=(i+dsize) % self.bytes if i hello, i use python 2.1, amkCrypto 0.1.3, and OpenSSL 0.9.6c. i tried to install amkCrypto on windows 2000. i had to correct some headers for them to comply with Visual C++ standards (never the same as others...).now, when i type "setup.py build", it compiles the sources right, but at link it complains about not being able to find crypto.lib. in fact there is no crypto.lib anywhere, even after rebuilding OpenSSL. by the way, i heard about an openssl.dll, and everything i can get is a ssleay.dll. so, how can i get the crypto.lib or build it myself ? thanks. -- rien From itamarst at YAHOO.COM Mon Apr 15 10:57:06 2002 From: itamarst at YAHOO.COM (Itamar S.-T.) Date: Mon, 15 Apr 2002 01:57:06 -0700 Subject: [PYTHON-CRYPTO] amkCrypto and openssl and crypto.lib In-Reply-To: <000901c1e3be$a43e6b10$2cee4cd4@RANDOM> Message-ID: <20020415085706.34462.qmail@web13007.mail.yahoo.com> --- Adrien Plisson wrote: > but at link it complains about not being able to > find crypto.lib. > in fact there is no crypto.lib anywhere, even after > rebuilding OpenSSL. by > the way, i heard about an openssl.dll, and > everything i can get is a > ssleay.dll. > so, how can i get the crypto.lib or build it myself Just comment out the line about crypto.lib - IIRC I managed to link with just ssleay and openssl mentioned. Also, there are precompiled versions of amkCrypto for win32 around the web - e.g. for Python 2.1: http://www.holdenweb.com/Python/ ===== Itamar Shtull-Trauring, itamar(at)shtull-trauring.org __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From ngps at POST1.COM Mon Apr 15 16:55:31 2002 From: ngps at POST1.COM (Ng Pheng Siong) Date: Mon, 15 Apr 2002 22:55:31 +0800 Subject: [PYTHON-CRYPTO] amkCrypto and openssl and crypto.lib Message-ID: <20020415225531.B569@vista.netmemetic.com> Try again... ------------------------ Rejected message (38 lines) -------------------------- On Sun, Apr 14, 2002 at 04:14:06PM +0200, Adrien Plisson wrote: > but at link it complains about not being able to find crypto.lib. > in fact there is no crypto.lib anywhere, even after rebuilding OpenSSL. by > the way, i heard about an openssl.dll, and everything i can get is a > ssleay.dll. > so, how can i get the crypto.lib or build it myself ? By default, building OpenSSL gives you libeay32.[dll|lib], ssleay32.[dll|lib] and openssl.exe. When you build apps, you link against the .lib's. Run-time, the .dll's are needed. I have a zip package of openssl.exe and the .dll's on my website. If you use Borland's free compiler suite, you can generate BC-compatible lib's from the .dll's. I'm sure this can also be done with MSVC and won't be surprised if the Gnu toolchain also does this. -- Ng Pheng Siong * http://www.netmemetic.com ----- End forwarded message ----- From bram at GAWTH.COM Tue Apr 16 19:14:45 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Tue, 16 Apr 2002 10:14:45 -0700 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020416085317.13625.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > What's going on with this--any news? We should try to get something > circulating so we can submit it for Python 2.3 which is coming up > pretty soon. Uh, I think the current plans are for us (meaning me and Paul) to meet up and hack something together. I've been a bit busy hacking on other stuff, but should be free on thursday if you'd like to meet up. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From akuchlin at mems-exchange.org Thu Apr 18 01:10:35 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Wed, 17 Apr 2002 19:10:35 -0400 Subject: [PYTHON-CRYPTO] PEP 272: revision 2 Message-ID: For good or ill, here's another revision of PEP 272 making minor changes to address some of the comments made in the previous round of discussion. I have resisted the temptation to increase the PEP's scope by adding padding, or random number generation, or public-key cryptography, because I think those should be separate PEPs. The 'Changes' section reads as follows: 2002-04: Removed references to stream ciphers; retitled PEP; prefixed feedback mode constants with MODE_; removed PGP feedback mode; added CTR and OFB feedback modes; clarified where numbers are measured in bytes and where in bits. Discussion in comp.lang.python, or on the python-crypto list, please. --amk PEP: 272 Title: API for Block Encryption Algorithms Version: $Revision: 1.7 $ Author: A.M. Kuchling Status: Draft Type: Informational Created: 18-Sep-2001 Post-History: 17-Apr-2002 Abstract This document specifies a standard API for secret-key block encryption algorithms such as DES or Rijndael, making it easier to switch between different algorithms and implementations. Introduction Encryption algorithms transform their input data (called plaintext) in some way that is dependent on a variable key, producing ciphertext. The transformation can easily be reversed if and only if one knows the key. The key is a sequence of bits chosen from some very large space of possible keys. There are two classes of encryption algorithms: block ciphers and stream ciphers. Block ciphers encrypt multibyte inputs of a fixed size (frequently 8 or 16 bytes long), and can be operated in various feedback modes. The feedback modes supported in this specification are: Number Constant Description 1 MODE_ECB Electronic Code Book 2 MODE_CBC Cipher Block Chaining 3 MODE_CFB Cipher Feedback 5 MODE_OFB Output Feedback 6 MODE_CTR Counter These modes are to be implemented as described in NIST publication SP 800-38A [1]. Descriptions of the first three feedback modes can also be found in Bruce Schneier's book _Applied Cryptography_ [2]. (The numeric value 4 is reserved for MODE_PGP, a variant of CFB described in RFC 2440: "OpenPGP Message Format" [3]. This mode isn't considered important enough to make it worth requiring it for all block encryption ciphers, though supporting it is a nice extra feature.) In a strict formal sense, stream ciphers encrypt data bit-by-bit; practically, stream ciphers work on a character-by-character basis. This PEP only aims at specifying an interface for block ciphers, though stream ciphers can support the interface described here by fixing 'block_size' to 1. Feedback modes also don't make sense for stream ciphers, so the only reasonable feedback mode would be ECB mode. Specification Encryption modules can add additional functions, methods, and attributes beyond those described in this PEP, but all of the features described in this PEP must be present for a module to claim compliance with it. Secret-key encryption modules should define one function: new(key, mode, [IV], **kwargs) Returns a ciphering object, using the secret key contained in the string 'key', and using the feedback mode 'mode', which must be one of the constants from the table above. If 'mode' is MODE_CBC or MODE_CFB, 'IV' must be provided and must be a string of the same length as the block size. Not providing a value of 'IV' will result in a ValueError exception being raised. Depending on the algorithm, a module may support additional keyword arguments to this function. Some keyword arguments are specified by this PEP, and modules are free to add additional keyword arguments. If a value isn't provided for a given keyword, a secure default value should be used. For example, if an algorithm has a selectable number of rounds between 1 and 16, and 1-round encryption is insecure and 8-round encryption is believed secure, the default value for 'rounds' should be 8 or more. (Module implementors can choose a very slow but secure value, too, such as 16 in this example. This decision is left up to the implementor.) The following table lists keyword arguments defined by this PEP: Keyword Meaning counter Callable object that returns counter blocks (see below; CTR mode only) rounds Number of rounds of encryption to use segment_size Size of data and ciphertext segments, measured in bits (see below; CFB mode only) The Counter feedback mode requires a sequence of input blocks, called counters, that are used to produce the output. When 'mode' is MODE_CTR, the 'counter' keyword argument must be provided, and its value must be a callable object, such as a function or method. Successive calls to this callable object must return a sequence of strings that are of the length 'block_size' and that never repeats. (Appendix B of the NIST publication gives a way to generate such a sequence, but that's beyond the scope of this PEP.) The CFB mode operates on segments of the plaintext and ciphertext that are 'segment_size' bits long. Therefore, when using this mode, the input and output strings must be a multiple of 'segment_size' bits in length. 'segment_size' must be an integer between 1 and block_size*8, inclusive. (The factor of 8 comes from 'block_size' being measured in bytes and not in bits). The default value for this parameter should be block_size*8. Implementors are allowed to constrain 'segment_size' to be a multiple of 8 for simplicity, but they're encouraged to support arbitrary values for generality. Secret-key encryption modules should define two variables: block_size An integer value; the size of the blocks encrypted by this module, measured in bytes. For all feedback modes, the length of strings passed to the encrypt() and decrypt() must be a multiple of the block size. key_size An integer value; the size of the keys required by this module, measured in bytes. If key_size is None, then the algorithm accepts arbitrary-length keys. You cannot pass a key of length 0 (that is, the null string '') as such a variable-length key. Cipher objects should have two attributes: block_size An integer value equal to the size of the blocks encrypted by this object. For algorithms with a variable block size, this value is equal to the block size selected for this object. IV Contains the initial value which will be used to start a cipher feedback mode; it will always be a string exactly one block in length. After encrypting or decrypting a string, this value is updated to reflect the modified feedback text. It is read-only, and cannot be assigned a new value. Cipher objects require the following methods: decrypt(string) Decrypts 'string', using the key-dependent data in the object and with the appropriate feedback mode. The string's length must be an exact multiple of the algorithm's block size or, in CFB mode, of the segment size. Returns a string containing the plaintext. encrypt(string) Encrypts a non-empty string, using the key-dependent data in the object, and with the appropriate feedback mode. The string's length must be an exact multiple of the algorithm's block size or, in CFB mode, of the segment size. Returns a string containing the ciphertext. Here's an example, using a module named 'DES': >>> import DES >>> obj = DES.new('abcdefgh', DES.MODE_ECB) >>> plaintext = "Guido van Rossum is a space alien." >>> len(plaintext) 34 >>> obj.encrypt(plaintext) Traceback (innermost last): File "", line 1, in ? ValueError: Strings for DES must be a multiple of 8 in length >>> ciphertext = obj.encrypt(plain+'XXXXXX') # Add padding >>> ciphertext '\021,\343Nq\214DY\337T\342pA\372\255\311s\210\363,\300j\330\250\312\347\342I\3215w\03561\303dgb/\006' >>> obj.decrypt(ciphertext) 'Guido van Rossum is a space alien.XXXXXX' References [1] NIST publication SP 800-38A, "Recommendation for Block Cipher Modes of Operation" (http://csrc.nist.gov/encryption/modes/) [2] Applied Cryptography [3] RFC2440: "OpenPGP Message Format" (http://rfc2440.x42.com, http://www.faqs.org/rfcs/rfc2440.html) Changes 2002-04: Remove references to stream ciphers; retitled PEP; prefixed feedback mode constants with MODE_; removed PGP feedback mode; added CTR and OFB feedback modes; clarify where numbers are measured in bytes and where in bits. Acknowledgements Thanks to the readers of the python-crypto list for their comments on this PEP. Copyright This document has been placed in the public domain. From phr-pycrypt at nightsong.com Thu Apr 18 12:44:47 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Thu, 18 Apr 2002 10:44:47 -0000 Subject: [PYTHON-CRYPTO] PEP 272 version 2 Message-ID: <20020418104447.23412.qmail@brouhaha.com> For good or ill, here's another revision of PEP 272 making minor changes to address some of the comments made in the previous round of discussion. I have resisted the temptation to increase the PEP's scope by adding padding, or random number generation, or public-key cryptography, because I think those should be separate PEPs. I don't think any of them should be PEP's--did we have a discussion of that already? Here's the reasoning: I think PEP 272 shouldn't exist, because it aims to solve a non-existent problem, which is providing a standard API for multiple block cipher modules that plug interchangeably into applications. There's just no need for that. PEP 272 seems to be written from a 1980's point of view when we were afraid of the NSA's influence on DES and everyone had their favorite alternative block cipher that they wanted to use instead. But that situation has sorted itself out now. DES/3DES has been a standard for 25 years with no surprising problems. Its paranoia-inspiring design mysteries were unravelled and published in the 90's so no one is afraid of it any more; and the current AES standard has better performance for new applications while supporting all the operations DES supports. So a new application should use AES and stick with it. A program to interoperate with a legacy app (say with old versions of PGP that need the IDEA cipher) has to use whatever cipher the legacy app needs--again, it doesn't need pluggable ciphers since it has no choice what cipher to use. PEP 272 might be a good starting point for the AES/DES cipher module documentation, but it doesn't seem useful as a spec for further such modules. It would be like having a PEP for the API of trig functions in the math library. If someone wants to add a hyperbolic haversine function, they should just implement and document it--a PEP for the API just isn't that useful. So I think PEP 272 should be withdrawn, and its text massaged into documentation for a DES/AES module submitted to the standard library. Note that AES and DES will share code to implement the modes of operation. If someone wants to implement another cipher, they may want to share that same code--it will be obvious how to do that, but it's a matter of low-level C interfaces, which this PEP so far says nothing about. Anyway though, I'll remark on the contents again: The 'Changes' section reads as follows: 2002-04: Removed references to stream ciphers; retitled PEP; prefixed feedback mode constants with MODE_; removed PGP feedback mode; added CTR and OFB feedback modes; clarified where numbers are measured in bytes and where in bits. These are fine, PGP feedback mode can be restored too, but these things should be optional per-implementation. The following table lists keyword arguments defined by this PEP: Keyword Meaning counter Callable object that returns counter blocks (see below; CTR mode only) What's the purpose of this? I thought we were just going to pass in an integer, not a callable object. Let's try to keep things simple. rounds Number of rounds of encryption to use I don't remember this in the last version. IMO it should definitely be removed. There are no ciphers I know of where the number of rounds is a user parameter. In DES, the # of rounds is intimately related to the sequents of rotations in the key schedule and if you mess with it in the slightest way, you weaken the cipher. In AES, the # of rounds is determined by the key length. In IDEA, Skipjack, and GOST, it's fixed. OK, it's sort of a parameter in RC5, but this is a rarity. Anyway, the PEP shouldn't specify this. The Counter feedback mode requires a sequence of input blocks, CTR mode is not a feedback mode. When 'mode' is MODE_CTR, the 'counter' keyword argument must be provided, and its value must be a callable object, such as a function or method. Successive calls to this callable object must return a sequence of strings that are of the length 'block_size' and that never repeats. IMO this is a poor idea since it complicates the implementation and slows it down a lot if the callable function is written in Python. It means the cipher module has to make back-calls into the interpreter. I think we should leave it the way we discussed on the list earlier. The CFB mode operates on segments of the plaintext and ciphertext that are 'segment_size' bits long.... Implementors are allowed to constrain 'segment_size' to be a multiple of 8 for simplicity, but they're encouraged to support arbitrary values for generality. I don't understand why 8-bit CFB should be required if 1-bit CFB isn't required. I say don't require anything. Don't require CFB to be supported at all, but if it's supported, it's ok if the only segment size supported is the same as the block size. Secret-key encryption modules should define two variables: block_size An integer value; the size of the blocks encrypted by this module, measured in bytes. For all feedback modes, the length of strings passed to the encrypt() and decrypt() must be a multiple of the block size. For RC5, this is variable. So if anything, this parameter should be in the cipher object and maybe a keyword parameter to the constructor, not something the module itself defines. key_size An integer value; the size of the keys required by this module, measured in bytes. If key_size is None, then the algorithm accepts arbitrary-length keys. You cannot pass a key of length 0 (that is, the null string '') as such a variable-length key. For AES, this is 128, 192, or 256 bits; it's not a single integer and not arbitrary. Really though, I think this amount of parametrization is excessive. The application should know what cipher it's trying to use, and know the characteristics of that cipher. The cipher object constructor should just throw an exception if you pass in a key length that the cipher doesn't support. IV Contains the initial value which will be used to start a cipher feedback mode; It's used for modes other than CFB as well--different wording is needed. IV - contains the initialization vector used by CBC, CFB, and OFB modes. Cipher objects require the following methods: decrypt(string) Decrypts 'string', using the key-dependent data in the object ... This really needs to understand the padding mode. I don't think it's sensible for a block cipher API spec to go into such detail about feedback modes and say nothing about padding modes. Over the past couple of weeks, we've had long discussions on the list about the API of the AES module that Bram and I are hoping to write tomorrow. That API departed somewhat from PEP 272 but I thought we'd reached consensus about it, at least til we have some code to try out. Now the PEP has made another appearance and it's mostly the same as before--what's the story? I'd like it very much if this PEP was withdrawn and we weren't constrained by it. Let's write an implementation, bang it around for a while, and then if necessary rewrite the PEP based on what we've found the implementation needs to do. This is the internet way--rough consensus, running code, write the RFC afterwards. From bryan at EEVOLVED.COM Thu Apr 18 16:26:07 2002 From: bryan at EEVOLVED.COM (Bryan Mongeau) Date: Thu, 18 Apr 2002 10:26:07 -0400 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: <20020418104447.23412.qmail@brouhaha.com> References: <20020418104447.23412.qmail@brouhaha.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 18 April 2002 06:44 am, Paul Rubin wrote: > I think PEP 272 shouldn't exist, because it aims to solve a > non-existent problem, which is providing a standard API for multiple > block cipher modules that plug interchangeably into applications. > There's just no need for that. I concur wholeheartedly with Paul. Perhaps someone could provide an example of an existing or imagined application that would require pluggable block ciphers? - -- Bryan Mongeau http://eevolved.com/bryan - -- "A man's ethical behavior should be based effectually on sympathy, education, and social ties; no religious basis is necessary. Man would indeed be in a poor way if he had to be restrained by fear of punishment and hope of reward after death."-- Einstein -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iEYEARECAAYFAjy+138ACgkQ3SCd0lDF8natCgCcCkCdmPmGedR5LcJEFXxQMxnc SbYAniTOkWNRXtUJ7uGOrKG8SOYpO1d6 =TFln -----END PGP SIGNATURE----- From p.mayers at IC.AC.UK Thu Apr 18 16:41:04 2002 From: p.mayers at IC.AC.UK (Phil Mayers) Date: Thu, 18 Apr 2002 15:41:04 +0100 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: References: <20020418104447.23412.qmail@brouhaha.com> Message-ID: <1019140864.30219.15.camel@wildfire.net.ic.ac.uk> On Thu, 2002-04-18 at 15:26, Bryan Mongeau wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Thursday 18 April 2002 06:44 am, Paul Rubin wrote: > > > I think PEP 272 shouldn't exist, because it aims to solve a > > non-existent problem, which is providing a standard API for multiple > > block cipher modules that plug interchangeably into applications. > > There's just no need for that. > > I concur wholeheartedly with Paul. Perhaps someone could provide an example > of an existing or imagined application that would require pluggable block > ciphers? I disagree, and for some examples: TLS (I could do this in native python with the code I have), Kerberos (this is one I'm actually doing, in native Python), SPKM (ditto). It's pretty trivial to map an etype in Kerberos to a module with a text config file, so a user could do this: rpm -i crypto-spiffynewalgo I think the point is more - you *are* going to have block ciphers in Python. Why *not* have a standard API for them? Failing to do so means implementors can do what they like. This is inconsistent, difficult to learn, inelegant and wasteful. > > - -- > Bryan Mongeau > http://eevolved.com/bryan > - -- > "A man's ethical behavior should be based effectually on sympathy, education, > and social ties; no religious basis is necessary. Man would indeed be in a > poor way if he had to be restrained by fear of punishment and hope of reward > after death."-- Einstein > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.0.6 (GNU/Linux) > Comment: For info see http://www.gnupg.org > > iEYEARECAAYFAjy+138ACgkQ3SCd0lDF8natCgCcCkCdmPmGedR5LcJEFXxQMxnc > SbYAniTOkWNRXtUJ7uGOrKG8SOYpO1d6 > =TFln > -----END PGP SIGNATURE----- -- Regards, Phil +------------------------------------------+ | Phil Mayers | | Network & Infrastructure Group | | Information & Communication Technologies | | Imperial College | +------------------------------------------+ From alex at BOFH.TORUN.PL Thu Apr 18 16:51:36 2002 From: alex at BOFH.TORUN.PL (Janusz A. Urbanowicz) Date: Thu, 18 Apr 2002 16:51:36 +0200 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: from Bryan Mongeau at "Apr 18, 2002 10:26:07 am" Message-ID: Bryan Mongeau wrote/napisa?[a]/schrieb: -- Start of PGP signed section. > On Thursday 18 April 2002 06:44 am, Paul Rubin wrote: > > > I think PEP 272 shouldn't exist, because it aims to solve a > > non-existent problem, which is providing a standard API for multiple > > block cipher modules that plug interchangeably into applications. > > There's just no need for that. > > I concur wholeheartedly with Paul. Perhaps someone could provide an example > of an existing or imagined application that would require pluggable block > ciphers? GNU Privacy Guard has a pluggable block ciphers. SILC has pluggable block ciphers. In my opinion any program that supports more block cipher algorithms than 3DES shoud support pluggable block ciphers on some level. The world of cryptoanalysis changes rapidly and cipher once secure may not be such on the next day (vide latest, still theoretical, attacks on AES published this week). I consider hardcoding one cipher algorithm into Python standard library a Wrong Thing. Alex -- C _-=-_ H Janusz A. Urbanowicz, stomil at jabber.org, PGP 0x21939169 * ; (_O : ----------------------------------------------------------- --+~| ! &~) ? P?yn?? chc? na Wsch?d, za Suez, gdzie jest dobrem ka?de z?o l_|/ A ~-=-~ O Gdzie przykaza? brak dziesi?ciu, a pi? mo?na a? po dno; | From akuchlin at mems-exchange.org Thu Apr 18 17:29:47 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Thu, 18 Apr 2002 11:29:47 -0400 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: <1019140864.30219.15.camel@wildfire.net.ic.ac.uk> References: <20020418104447.23412.qmail@brouhaha.com> <1019140864.30219.15.camel@wildfire.net.ic.ac.uk> Message-ID: <20020418152947.GB1262@ute.mems-exchange.org> On Thu, Apr 18, 2002 at 03:41:04PM +0100, Phil Mayers wrote: >TLS (I could do this in native python with the code I have), Kerberos >(this is one I'm actually doing, in native Python), SPKM (ditto). It's I was thinking of PGP/GPG, and perhaps if you wanted to write tools to work with IPSEC packets (say, a smart tcpdump type of thing). >I think the point is more - you *are* going to have block ciphers in >Python. Why *not* have a standard API for them? Failing to do so means >implementors can do what they like. This is inconsistent, difficult to >learn, inelegant and wasteful. Note, however, that this PEP is of type 'informational', and an informational PEP carries as much weight as an informational RFC; that is, none whatsoever. Implementors *can* do what they like, and are free to ignore the PEP if they don't like it or if it's unsuitable for their application. (At least that's what I've always thought 'informational' meant, though PEP 1 doesn't actually make that explicit. I'll talk to Barry Warsaw about clarifying this.) For example, the other informational PEPs are the Database API specifications, and there are database modules that don't follow them. --amk (www.amk.ca) "I know this TARDIS like the back of my hand." -- The Doctor, in "The Invasion of Time" From Paul at CRYPTORIGHTS.ORG Thu Apr 18 17:34:08 2002 From: Paul at CRYPTORIGHTS.ORG (Paul) Date: Thu, 18 Apr 2002 08:34:08 -0700 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: <20020418104447.23412.qmail@brouhaha.com> References: <20020418104447.23412.qmail@brouhaha.com> Message-ID: >I'd like it very much if this PEP was withdrawn and we weren't >constrained by it. Let's write an implementation, bang it around for >a while, and then if necessary rewrite the PEP based on what we've >found the implementation needs to do. This is the internet way--rough >consensus, running code, write the RFC afterwards. This might be better ... at this point it seems easier to write code than get changes made to a set of requirements. I still feel that padding and IV need to be transparent, modes should be unique classes and that support for "streaming" encryption for large files would be very useful. Paul ( another one :-) At 10:44 AM +0000 4/18/02, Paul Rubin wrote: > For good or ill, here's another revision of PEP 272 making minor > changes to address some of the comments made in the previous round of > discussion. I have resisted the temptation to increase the PEP's > scope by adding padding, or random number generation, or public-key > cryptography, because I think those should be separate PEPs. > >I don't think any of them should be PEP's--did we have a discussion >of that already? Here's the reasoning: > > I think PEP 272 shouldn't exist, because it aims to solve a > non-existent problem, which is providing a standard API for multiple > block cipher modules that plug interchangeably into applications. > There's just no need for that. PEP 272 seems to be written from a > 1980's point of view when we were afraid of the NSA's influence on > DES and everyone had their favorite alternative block cipher that > they wanted to use instead. But that situation has sorted itself > out now. DES/3DES has been a standard for 25 years with no > surprising problems. Its paranoia-inspiring design mysteries were > unravelled and published in the 90's so no one is afraid of it any > more; and the current AES standard has better performance for new > applications while supporting all the operations DES supports. > > So a new application should use AES and stick with it. A program to > interoperate with a legacy app (say with old versions of PGP that > need the IDEA cipher) has to use whatever cipher the legacy app > needs--again, it doesn't need pluggable ciphers since it has no > choice what cipher to use. PEP 272 might be a good starting point > for the AES/DES cipher module documentation, but it doesn't seem > useful as a spec for further such modules. It would be like having > a PEP for the API of trig functions in the math library. If someone > wants to add a hyperbolic haversine function, they should just > implement and document it--a PEP for the API just isn't that useful. > > So I think PEP 272 should be withdrawn, and its text massaged into > documentation for a DES/AES module submitted to the standard library. > > Note that AES and DES will share code to implement the modes of operation. > If someone wants to implement another cipher, they may want to share > that same code--it will be obvious how to do that, but it's a matter > of low-level C interfaces, which this PEP so far says nothing about. > >Anyway though, I'll remark on the contents again: > > The 'Changes' section reads as follows: > > 2002-04: Removed references to stream ciphers; retitled PEP; > prefixed feedback mode constants with MODE_; removed PGP feedback > mode; added CTR and OFB feedback modes; clarified where numbers > are measured in bytes and where in bits. > >These are fine, PGP feedback mode can be restored too, but these >things should be optional per-implementation. > > The following table lists keyword arguments defined by this PEP: > > Keyword Meaning > counter Callable object that returns counter blocks > (see below; CTR mode only) > >What's the purpose of this? I thought we were just going to pass in >an integer, not a callable object. Let's try to keep things simple. > > rounds Number of rounds of encryption to use > >I don't remember this in the last version. IMO it should definitely >be removed. There are no ciphers I know of where the number of rounds >is a user parameter. In DES, the # of rounds is intimately related to >the sequents of rotations in the key schedule and if you mess with it >in the slightest way, you weaken the cipher. In AES, the # of rounds >is determined by the key length. In IDEA, Skipjack, and GOST, it's >fixed. OK, it's sort of a parameter in RC5, but this is a rarity. >Anyway, the PEP shouldn't specify this. > > The Counter feedback mode requires a sequence of input blocks, > >CTR mode is not a feedback mode. > > When 'mode' is MODE_CTR, the 'counter' keyword argument must be > provided, and its value must be a callable object, such as a > function or method. Successive calls to this callable object must > return a sequence of strings that are of the length 'block_size' > and that never repeats. > >IMO this is a poor idea since it complicates the implementation and >slows it down a lot if the callable function is written in Python. >It means the cipher module has to make back-calls into the interpreter. >I think we should leave it the way we discussed on the list earlier. > > The CFB mode operates on segments of the plaintext and ciphertext > that are 'segment_size' bits long.... > Implementors are allowed to constrain 'segment_size' to be a > multiple of 8 for simplicity, but they're encouraged to support > arbitrary values for generality. > >I don't understand why 8-bit CFB should be required if 1-bit CFB isn't >required. I say don't require anything. Don't require CFB to be >supported at all, but if it's supported, it's ok if the only segment >size supported is the same as the block size. > > Secret-key encryption modules should define two variables: > > block_size > > An integer value; the size of the blocks encrypted by this > module, measured in bytes. For all feedback modes, the length > of strings passed to the encrypt() and decrypt() must be a > multiple of the block size. > >For RC5, this is variable. So if anything, this parameter should be >in the cipher object and maybe a keyword parameter to the constructor, >not something the module itself defines. > > key_size > > An integer value; the size of the keys required by this > module, measured in bytes. If key_size is None, then the > algorithm accepts arbitrary-length keys. You cannot pass a > key of length 0 (that is, the null string '') as such a > variable-length key. > >For AES, this is 128, 192, or 256 bits; it's not a single integer and >not arbitrary. > >Really though, I think this amount of parametrization is excessive. >The application should know what cipher it's trying to use, and know >the characteristics of that cipher. The cipher object constructor >should just throw an exception if you pass in a key length that the >cipher doesn't support. > > IV > > Contains the initial value which will be used to start a > cipher feedback mode; > >It's used for modes other than CFB as well--different wording is needed. >IV - contains the initialization vector used by CBC, CFB, and OFB modes. > > Cipher objects require the following methods: > > decrypt(string) > > Decrypts 'string', using the key-dependent data in the object > ... > >This really needs to understand the padding mode. I don't think it's >sensible for a block cipher API spec to go into such detail about >feedback modes and say nothing about padding modes. > >Over the past couple of weeks, we've had long discussions on the list >about the API of the AES module that Bram and I are hoping to write >tomorrow. That API departed somewhat from PEP 272 but I thought we'd >reached consensus about it, at least til we have some code to try out. > >Now the PEP has made another appearance and it's mostly the same as >before--what's the story? > >I'd like it very much if this PEP was withdrawn and we weren't >constrained by it. Let's write an implementation, bang it around for >a while, and then if necessary rewrite the PEP based on what we've >found the implementation needs to do. This is the internet way--rough >consensus, running code, write the RFC afterwards. -- From Paul at CRYPTORIGHTS.ORG Thu Apr 18 17:40:45 2002 From: Paul at CRYPTORIGHTS.ORG (Paul) Date: Thu, 18 Apr 2002 08:40:45 -0700 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: References: <20020418104447.23412.qmail@brouhaha.com> Message-ID: At 10:26 AM -0400 4/18/02, Bryan Mongeau wrote: >-----BEGIN PGP SIGNED MESSAGE----- >Hash: SHA1 > >On Thursday 18 April 2002 06:44 am, Paul Rubin wrote: > >> I think PEP 272 shouldn't exist, because it aims to solve a >> non-existent problem, which is providing a standard API for multiple >> block cipher modules that plug interchangeably into applications. >> There's just no need for that. > >I concur wholeheartedly with Paul. Perhaps someone could provide an example >of an existing or imagined application that would require pluggable block >ciphers? IPsec, SSL, S/MIME, PGP all allow several blcok algorithms to be used .... It depends what you call "pluggable, but a common api/class is useful. Paul > >- -- >Bryan Mongeau >http://eevolved.com/bryan >- -- >"A man's ethical behavior should be based effectually on sympathy, education, >and social ties; no religious basis is necessary. Man would indeed be in a >poor way if he had to be restrained by fear of punishment and hope of reward >after death."-- Einstein >-----BEGIN PGP SIGNATURE----- >Version: GnuPG v1.0.6 (GNU/Linux) >Comment: For info see http://www.gnupg.org > >iEYEARECAAYFAjy+138ACgkQ3SCd0lDF8natCgCcCkCdmPmGedR5LcJEFXxQMxnc >SbYAniTOkWNRXtUJ7uGOrKG8SOYpO1d6 >=TFln >-----END PGP SIGNATURE----- -- From bram at GAWTH.COM Thu Apr 18 19:28:30 2002 From: bram at GAWTH.COM (Bram Cohen) Date: Thu, 18 Apr 2002 10:28:30 -0700 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: Message-ID: Janusz A. Urbanowicz wrote: > GNU Privacy Guard has a pluggable block ciphers. SILC has pluggable block > ciphers. Note that those are pluggable ciphers, not pluggable modes - I know of nothing which has pluggable modes. > In my opinion any program that supports more block cipher algorithms than > 3DES shoud support pluggable block ciphers on some level. The world of > cryptoanalysis changes rapidly and cipher once secure may not be such on the > next day (vide latest, still theoretical, attacks on AES published this week). > I consider hardcoding one cipher algorithm into Python standard library a > Wrong Thing. The consensus I've heard among cryptographers is that these days one should use AES and be done with it. Switching to another algorithm in the remote case that AES may become obsolete is something which is best done only if necessary, I don't think it would involve any less pain dealing with it now than later. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From phr-pycrypt at nightsong.com Thu Apr 18 21:54:53 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Thu, 18 Apr 2002 19:54:53 -0000 Subject: [PYTHON-CRYPTO] PEP 272 version 2 Message-ID: <20020418195453.29289.qmail@brouhaha.com> I have a meeting in a few minutes but will send out a short reply now and add some more later. Alex said I consider hardcoding one cipher algorithm into Python standard library a Wrong Thing. but I don't think there's any suggestion of "hardcoding one cipher algorithm into the Python standard library". I'm not even sure what "hardcoded" means here. I have nothing against more block ciphers being added, if applications need them. That's just like there are already multiple hash functions (MD5, SHA). If there's a need to add HAVAL or TIGER, they can be added without any fuss. Yet there's no PEP for hash function API's and there doesn't need to be one. [Andrew] Note, however, that this PEP is of type 'informational', and an informational PEP carries as much weight as an informational RFC; that is, none whatsoever. Implementors *can* do what they like, and are free to ignore the PEP if they don't like it or if it's unsuitable for their application. (At least that's what I've always thought 'informational' meant, though PEP 1 doesn't actually make that explicit. I'll talk to Barry Warsaw about clarifying this.) This is confusing--if the PEP has no weight, why have it at all? "Implementers can do what they like" is fine for people writing their own apps, but we're talking about a submission to the standard library which we're proposing be distributed to every Python user. If the PEP has any meaning, it's for that situation. When our AES module doesn't follow the PEP chapter and verse--and it won't-- Guido (or whoever) is going to look at the submission and say "Why doesn't it follow the PEP? Please revise it til it does." The possible resolutions are: 1) Make the library follow the PEP--bad for technical reasons described earlier. It will be pretty close, but some details will be different, and I'd thought we'd agreed there were some features we weren't going to bother with. 2) Make the PEP follow the library--maybe better, but seems kind of pointless and constraining. 3) End up not getting the module accepted for the library, and stay stuck in the bad old days--an outcome to be avoided. 4) Eliminate the PEP and submit the library without the PEP, as worked perfectly well for hash functions. Why fix what isn't broken? So IMO, this is the best route to take. I think writing a PEP for something as low-level as block ciphers is excessively concerned with process. I'd rather concentrate on getting code running. For example, the other informational PEPs are the Database API specifications, and there are database modules that don't follow them. The database API PEP is more useful than a block cipher PEP because there's a real need for applications to be easily switchable between different databases, and the interface to database is far more complicated than the the interface to block ciphers. I do like the idea of a cryptography PEP for Python, but IMO it should be at a much higher architectural level than PEP 272, i.e. it should be more along the lines of JCA. I'd be happy to help write such a PEP after the AES module is done. From akuchlin at mems-exchange.org Thu Apr 18 22:09:57 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Thu, 18 Apr 2002 16:09:57 -0400 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: <20020418195453.29289.qmail@brouhaha.com> References: <20020418195453.29289.qmail@brouhaha.com> Message-ID: <20020418200957.GB3208@ute.mems-exchange.org> On Thu, Apr 18, 2002 at 07:54:53PM -0000, Paul Rubin wrote: >HAVAL or TIGER, they can be added without any fuss. Yet there's no >PEP for hash function API's and there doesn't need to be one. That would be PEP 247, "API for Cryptographic Hash Functions", and posted to this list way back when. >has any meaning, it's for that situation. When our AES module doesn't >follow the PEP chapter and verse--and it won't-- Guido (or whoever) is >going to look at the submission and say "Why doesn't it follow the >PEP? Please revise it til it does." The possible resolutions are: No he won't, because PEP 272 is informational. (I'll spank him if he does...) >I do like the idea of a cryptography PEP for Python, but IMO it should >be at a much higher architectural level than PEP 272, i.e. it should >be more along the lines of JCA. I'd be happy to help write such a PEP >after the AES module is done. Go for it! --amk (www.amk.ca) [Thomas] Paine is useful as a rabble-rouser; but rabble-rousers are needed before revolutions, not after. -- St. Just, in SANDMAN #29: "Thermidor" From alex at BOFH.TORUN.PL Fri Apr 19 17:41:28 2002 From: alex at BOFH.TORUN.PL (Janusz A. Urbanowicz) Date: Fri, 19 Apr 2002 17:41:28 +0200 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: from Bram Cohen at "Apr 18, 2002 10:28:30 am" Message-ID: Bram Cohen wrote/napisa?[a]/schrieb: > Janusz A. Urbanowicz wrote: > > > GNU Privacy Guard has a pluggable block ciphers. SILC has pluggable block > > ciphers. > > Note that those are pluggable ciphers, not pluggable modes - I know of > nothing which has pluggable modes. The question was on algirithms. Although SILC definition of 'cipher module' includes mode as well. > The consensus I've heard among cryptographers is that these days one > should use AES and be done with it. Switching to another algorithm in the > remote case that AES may become obsolete is something which is best done > only if necessary, I don't think it would involve any less pain dealing > with it now than later. Cryptographers, maybe. In real life security engineering as far as I know, flexibility of a toolkit is a important thing. And remember that Python is a toolkit, not an application. Alex -- C _-=-_ H Janusz A. Urbanowicz, stomil at jabber.org, PGP 0x21939169 * ; (_O : ----------------------------------------------------------- --+~| ! &~) ? P?yn?? chc? na Wsch?d, za Suez, gdzie jest dobrem ka?de z?o l_|/ A ~-=-~ O Gdzie przykaza? brak dziesi?ciu, a pi? mo?na a? po dno; | From zooko at zooko.com Sat Apr 20 14:32:54 2002 From: zooko at zooko.com (Zooko) Date: Sat, 20 Apr 2002 05:32:54 -0700 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: Message from Paul Rubin of "Fri, 05 Apr 2002 01:52:06 GMT." <20020405015206.18425.qmail@brouhaha.com> References: <20020405015206.18425.qmail@brouhaha.com> Message-ID: two weeks ago, on Apr 5, Paul Rubin wrote: > > ECB, CBC, CFB and OFB are described here: > > http://www.iks-jena.de/mitarb/lutz/security/cryptfaq/q82.html > http://www.iks-jena.de/mitarb/lutz/security/cryptfaq/q83.html > > CTR is done by just encrypting the blocks 0001, 0002, 0003, ... > and xoring the resulting stream against the plaintext stream. > > It has the advantage of being very simple, and not needing special > padding if you want to encrypt a plaintext of 23 bytes or something > like that. Its disadvantages are that you must never re-use a key, Untrue -- you must never re-use a (key, IV) *pair*. This is the same constraint that CBC, CFB and OFB have as far as I understand. (I'm not very familiar with those modes.) > and it is somewhat more subject to ciphertext modification attacks > than other modes. Flipping a single bit in the ciphertext results in > flipping the same bit in the plaintext without disturbing the > surrounding bits. This is true, but it isn't a novel vulnerability so much as a more dramatic version of a vulnerability that all the modes under discussion share. One really should not rely on the bit-disturbing properties of the other modes, either. If you want non-malleability or authentication or whatever, then you need to use an algorithm that guarantees it, not rely on some bit-disturbing side effect of your encryption mode. By the way, I'm a lot less positive about CTR mode after the conversation about it between you (Paul Rubin), Bram Cohen and I on this list a few weeks ago. All three of us, experienced crypto hackers all, made substantial errors about usage (management of the key and IV). I think this demonstrates that it really is harder for people to use CTR mode properly. Regards, Zooko Zooko.Com -- Security and Distributed Systems Engineering From mike at id-vault.com Sat Apr 20 19:27:25 2002 From: mike at id-vault.com (Michael Schwartz) Date: Sat, 20 Apr 2002 13:27:25 -0400 (EDT) Subject: trouble installing 2.0.0pre04 Message-ID: <20020420131700.I18505-100000@willow.id-vault.com> Hello, I'm having trouble building python-ldap 2.0.0pre04... any hints / suggestions would be greatly appeciated! tia, Mike Schwartz mike at id-vault.com ========================= output =============================== # python setup.py build running build running build_py warning: build_py: file Lib/ldap.py (for module ldap) not found not copying Lib/ldap/__init__.py (output up-to-date) not copying Lib/ldap/async.py (output up-to-date) not copying Lib/ldap/functions.py (output up-to-date) not copying Lib/ldap/ldapobject.py (output up-to-date) not copying Lib/ldap/modlist.py (output up-to-date) not copying Lib/ldapurl.py (output up-to-date) not copying Lib/ldif.py (output up-to-date) warning: build_py: file Lib/ldap.py (for module ldap) not found running build_ext building '_ldap' extension skipping Modules/LDAPObject.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/LDAPObject.o up-to-date) skipping Modules/common.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/common.o up-to-date) skipping Modules/constants.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/constants.o up-to-date) skipping Modules/errors.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/errors.o up-to-date) skipping Modules/functions.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/functions.o up-to-date) skipping Modules/ldapmodule.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/ldapmodule.o up-to-date) skipping Modules/linkedlist.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/linkedlist.o up-to-date) skipping Modules/message.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/message.o up-to-date) skipping Modules/template.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/template.o up-to-date) skipping Modules/version.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/version.o up-to-date) skipping Modules/options.c (build/temp.freebsd-4.4-RELEASE-i386-2.1/options.o up-to-date) cc -shared -pthread build/temp.freebsd-4.4-RELEASE-i386-2.1/LDAPObject.o build/temp.freebsd-4.4-RELEASE-i386-2.1/common.o build/temp.freebsd-4.4-RELEASE-i386-2.1/constants.o build/temp.freebsd-4.4-RELEASE-i386-2.1/errors.o build/temp.freebsd-4.4-RELEASE-i386-2.1/functions.o build/temp.freebsd-4.4-RELEASE-i386-2.1/ldapmodule.o build/temp.freebsd-4.4-RELEASE-i386-2.1/linkedlist.o build/temp.freebsd-4.4-RELEASE-i386-2.1/message.o build/temp.freebsd-4.4-RELEASE-i386-2.1/template.o build/temp.freebsd-4.4-RELEASE-i386-2.1/version.o build/temp.freebsd-4.4-RELEASE-i386-2.1/options.o -L/usr/lib -L/usr/local/lib -llber -lldap -lresolv -o build/lib.freebsd-4.4-RELEASE-i386-2.1/_ldap.so /usr/libexec/elf/ld: cannot find -lresolv error: command 'cc' failed with exit status 1 ========================= setup.cfg ========================= # Section for compiling the C extension module # for wrapping OpenLDAP 2 libs [_ldap] class = OpenLDAP2 library_dirs = /usr/lib /usr/local/lib include_dirs = /usr/include /usr/local/include libs = lber ldap resolv # Installation options [install] compile = 1 optimize = 1 ============================================================= _______________________________________________ Python-LDAP-dev mailing list Python-LDAP-dev at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/python-ldap-dev From Paul at CRYPTORIGHTS.ORG Sat Apr 20 23:23:33 2002 From: Paul at CRYPTORIGHTS.ORG (Paul) Date: Sat, 20 Apr 2002 14:23:33 -0700 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: References: <20020405015206.18425.qmail@brouhaha.com> Message-ID: At 5:32 AM -0700 4/20/02, Zooko wrote: > two weeks ago, on Apr 5, Paul Rubin wrote: >> >> ECB, CBC, CFB and OFB are described here: >> >> http://www.iks-jena.de/mitarb/lutz/security/cryptfaq/q82.html >> http://www.iks-jena.de/mitarb/lutz/security/cryptfaq/q83.html >> >> CTR is done by just encrypting the blocks 0001, 0002, 0003, ... >> and xoring the resulting stream against the plaintext stream. >> >> It has the advantage of being very simple, and not needing special >> padding if you want to encrypt a plaintext of 23 bytes or something >> like that. Its disadvantages are that you must never re-use a key, > >Untrue -- you must never re-use a (key, IV) *pair*. This is the same >constraint that CBC, CFB and OFB have as far as I understand. (I'm not very >familiar with those modes.) > >> and it is somewhat more subject to ciphertext modification attacks >> than other modes. Flipping a single bit in the ciphertext results in >> flipping the same bit in the plaintext without disturbing the >> surrounding bits. > >This is true, but it isn't a novel vulnerability so much as a more dramatic >version of a vulnerability that all the modes under discussion share. One >really should not rely on the bit-disturbing properties of the other modes, >either. If you want non-malleability or authentication or whatever, then you >need to use an algorithm that guarantees it, not rely on some bit-disturbing >side effect of your encryption mode. Yes ... repeated IV totally breaks CTR. For CBC, it's a very very minor risk that at worst may expose that two encrypted items are similar. > > >By the way, I'm a lot less positive about CTR mode after the conversation >about it between you (Paul Rubin), Bram Cohen and I on this list a few weeks >ago. All three of us, experienced crypto hackers all, made substantial errors >about usage (management of the key and IV). I think this demonstrates that it >really is harder for people to use CTR mode properly. It also shows why IV should be part of the algorithm processing and not a parameter. Paul -- From phr-pycrypt at nightsong.com Sun Apr 21 00:20:51 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Sat, 20 Apr 2002 22:20:51 -0000 Subject: [PYTHON-CRYPTO] PEP 272 version 2 Message-ID: <20020420222051.7179.qmail@brouhaha.com> [Zooko] > CTR is done by just encrypting the blocks 0001, 0002, 0003, ... > and xoring the resulting stream against the plaintext stream. > > It has the advantage of being very simple, and not needing special > padding if you want to encrypt a plaintext of 23 bytes or something > like that. Its disadvantages are that you must never re-use a key, Untrue -- you must never re-use a (key, IV) *pair*. This is the same constraint that CBC, CFB and OFB have as far as I understand. (I'm not very familiar with those modes.) CTR as normally described doesn't have an IV. You must never re-use a key. The FIPS pub about modes of operation gives some advice about re-using the key for multiple messages by (e.g.) continuing to increment the counter value across messages (so that message 1 might use counter values 1-48, message 2 could use 49-83, etc.) but in my view that amounts to treating the whole collection of message as one message. It's really asking for trouble. Nothing really terrible happens if you re-use a key-IV pair in CBC mode, except if two messages have a common prefix then an eavesdropper can tell that they have a common prefix (or if they have the same contents). He can't read the message the way he can with CTR mode. Using CBC mode with the IV always = 0 is one of the defined modes in IEEE 1363 (or was it 1363A). > and it is somewhat more subject to ciphertext modification attacks > than other modes. Flipping a single bit in the ciphertext results in > flipping the same bit in the plaintext without disturbing the > surrounding bits. This is true, but it isn't a novel vulnerability so much as a more dramatic version of a vulnerability that all the modes under discussion share. One really should not rely on the bit-disturbing properties of the other modes, either. If you want non-malleability or authentication or whatever, then you need to use an algorithm that guarantees it, not rely on some bit-disturbing side effect of your encryption mode. This depends on the application and an application designer choosing what mode to use needs to be aware of the issues. An application designer who doesn't want to think about such issues should use the simple encrypt(string) interface that we'll provide, that chooses the mode and supplies integrity checking automatically. By the way, I'm a lot less positive about CTR mode after the conversation about it between you (Paul Rubin), Bram Cohen and I on this list a few weeks ago. All three of us, experienced crypto hackers all, made substantial errors about usage (management of the key and IV). I think this demonstrates that it really is harder for people to use CTR mode properly. CTR mode is fine if you know what you're doing enough to use it properly. If you don't know what you're doing, you shouldn't be using any mode, or at least shouldn't be choosing modes yourself. [Alex] > Note that those are pluggable ciphers, not pluggable modes - I > know of nothing which has pluggable modes. The question was on algirithms. Although SILC definition of 'cipher module' includes mode as well. What is SILC? I'm not familiar with it. > The consensus I've heard among cryptographers is that these days > one should use AES and be done with it. Switching to another > algorithm in the remote case that AES may become obsolete is > something which is best done only if necessary, I don't think it > would involve any less pain dealing with it now than later. Cryptographers, maybe. In real life security engineering as far as I know, flexibility of a toolkit is a important thing. And remember that Python is a toolkit, not an application. IMO, Python is not a toolkit in the sense that OpenSSL or Cryptlib is a toolkit. It's especially not a *crypto* toolkit, in that it's gotten along til now with no serious built-in encryption function whatsoever. Python is a system for building applications, either new ones (which should use AES) or ones that must interoperate with some legacy app (so they must use some cipher that the legacy app supports). That is, it must support ONE such cipher, not necessarily ALL such ciphers. Sure, there are examples of apps that *can* switch between ciphers, but I haven't heard of any yet that *need* to switch between ciphers. OpenPGP, TLS, and Kerberos all support 3DES, for example. They might also support (say) Blowfish for higher speed, but it's going to be a weird situation where a high speed application needs to be written in Python and can't use AES. Nautilus (http://www.lila.com/nautilus), which I worked on, can switch between ciphers but I think that was partly a mistake and partly a result of considerations that don't exist any more. Again, it supports 3DES, so an interoperating Python program doesn't need other ciphers. (It also supported IDEA to satisfy the paranoid PGP crowd that didn't trust anything having to do with DES, and Blowfish because it was fastest and wasn't patented like IDEA.) The AES library that Bram and I have started writing will support pluggable ciphers at the C level so that AES and DES can share the code that implements the modes of operation. The mechanism is easy to extend to other ciphers at the C interface level. I think the right way to support pluggible cipher modules is to make them create opaque objects that don't support any encryption operations at all at the Python level. There'd be a "block cipher" module that did encryption with the different modes using an arbitrary cipher, and individual cipher modules that the "block cipher" module calls at the C level. So the API would change slightly yet again, e.g. import blockcipher, AES key = "abcddefghijklmno" # key string iv = "01234567890abcde" cipher = blockcipher.CBC(AES(k1), iv) plaintext = "green is the color of my true love's hair" ciphertext = cipher.encrypt(plaintext) It makes implementing further cipher modules (DES, Blowfish, IDEA, Khufu, or whatever) easier because they don't have nearly as complicated an interface -- they just create a C struct that the blockcipher module knows how to access. I guess I could live with something like that. It basically brings out to the Python level what we're already doing at the C level. Comments on this? From aarchiba at YAHOO.COM Sun Apr 21 03:33:44 2002 From: aarchiba at YAHOO.COM (Andrew Archibald) Date: Sat, 20 Apr 2002 21:33:44 -0400 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: References: <20020405015206.18425.qmail@brouhaha.com> Message-ID: <20020421013343.GA2626@hedgehog> On Sat, Apr 20, 2002 at 02:23:33PM -0700, Paul wrote: > It also shows why IV should be part of the algorithm processing and > not a parameter. It's not very clear to me quite what this means, but I think we should take as a typical sort of task implementing the TLS or SSH protocols; for such a task we want a list of good ciphers, hash functions, and MACs, callable similarly; and in particular we need to be able to control exactly how padding is done and IVs are computed (the SSH protocol, for example, gives an algorithm for computing all IVs from the initial connection secret and the shared secret obtained from it). So doing the IVs for the poor dumb user is not necessarily a good idea. Providing convenience functions, say a way to wrap up a hunk of data so it's non-malleable and opaque (perhaps by computing the HMAC with SHA-160, padding in some standard way and then using AES-128 in CBC mode with a random IV drawn from /dev/urandom) might be a good idea, although most crypto applications will either (a) be able to use TLS entire or (b) want to implement a standard (possibly of their own design), so it's not clear how much of this to automate. Andrew From phr-pycrypt at nightsong.com Sun Apr 21 04:20:44 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Sun, 21 Apr 2002 02:20:44 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020421022044.9555.qmail@brouhaha.com> It's not very clear to me quite what this means, but I think we should take as a typical sort of task implementing the TLS or SSH protocols; for such a task we want a list of good ciphers, hash functions, and MACs, callable similarly; and in particular we need to be able to control exactly how padding is done and IVs are computed (the SSH protocol, for example, gives an algorithm for computing all IVs from the initial connection secret and the shared secret obtained from it). So doing the IVs for the poor dumb user is not necessarily a good idea. In practice, 3DES and RC4 are the only ciphers anyone uses in these protocols, with AES starting to be used in SSH. All conformant TLS implementations are required to support 3DES, so everything else can be ignored, unless you're trying to write a protocol analyzer rather than a communication stack. I believe the same is true of SSH. Re crypto interface: I hope to provide simple encryption/decryption functions that a non-cryptographer can call, that does as much as possible (algorithm selection, IV, chaining mode, padding, integrity check) automatically without the user have to think about these things. There will also be a more advanced interface that gives access to the raw algorithms and modes, that takes more knowledge to use. The simple interface would just be a Python wrapper around the advanced interface. A low level protocol implementation like TLS or SSH would use the advanced interface directly. If there's a real desire to implement TLS in Python instead of using mxCrypto (which interfaces to OpenSSL and has billions of features), then we should probably also provide an RC4 stream cipher module that's needed for SSL 3.0 and under. That's totally separate from the block cipher discussion and should have a very simple interface. From zooko at zooko.com Sun Apr 21 12:37:49 2002 From: zooko at zooko.com (Zooko) Date: Sun, 21 Apr 2002 03:37:49 -0700 Subject: [PYTHON-CRYPTO] PEP 272 version 2 In-Reply-To: Message from Paul Rubin of "Sat, 20 Apr 2002 22:20:51 -0000." <20020420222051.7179.qmail@brouhaha.com> References: <20020420222051.7179.qmail@brouhaha.com> Message-ID: Paul Rubin wrote: > > > It has the advantage of being very simple, and not needing special > > padding if you want to encrypt a plaintext of 23 bytes or something > > like that. Its disadvantages are that you must never re-use a key, > > Untrue -- you must never re-use a (key, IV) *pair*. This is the > same constraint that CBC, CFB and OFB have as far as I understand. > (I'm not very familiar with those modes.) > > CTR as normally described doesn't have an IV. You must never re-use > a key. I named the data element "IV" in the letter that you quote in order to compare the different modes more easily. (In fact, I had named it "nonce" when I wrote the letter, which is a more technically accurate name for it, but then I changed it to "IV" before sending.) But whatever it is called, it isn't true that you must never re-use a key in counter mode encryption. What's true is that you must never re-use a (key, nonce/counter/IV) *pair*. (Also you have to be careful not to use nonces which are near one another and could reach one another by incrementing.) I'm not writing this to persuade you (Paul Rubin) of anything (you've already heard my arguments), but to make sure that a reader following this thread didn't think that I agreed with your repeated assertion that "You must never re-use a key." in counter-mode encryption. By the way thanks for informing me about CBC mode's resilience to that usage. So in an attempt to return to the topic of this list... > CTR mode is fine if you know what you're doing enough to use it > properly. If you don't know what you're doing, you shouldn't be using > any mode, or at least shouldn't be choosing modes yourself. Right. If I were designing crypto modules for Python, I wouldn't offer CTR mode to the newbie. I think it should be relegated to secondary status so that only people who know they want it will find it. Regards, Zooko Zooko.Com -- Security and Distributed Systems Engineering From p.mayers at IC.AC.UK Mon Apr 22 01:27:39 2002 From: p.mayers at IC.AC.UK (Phil Mayers) Date: Mon, 22 Apr 2002 00:27:39 +0100 Subject: [PYTHON-CRYPTO] PEP (was Re: [PYTHON-CRYPTO] aes library) In-Reply-To: <20020421022044.9555.qmail@brouhaha.com> References: <20020421022044.9555.qmail@brouhaha.com> Message-ID: <1019431659.3cc34aeb90660@wildfire.net.ic.ac.uk> There seems to be an opinion: "None of these applications actually *need* more than one cipher, therefore no application would ever need more than one cipher, and all new applications will be using AES anyway, because. So, we don't need a crypto interface." I (and others, it seems) disagree with this entirely. Kerberos can and does require two (A DES variant and MS ARCFOUR) or even three (DES, 3DES, ARCFOUR) or four (DES, 3DES, ARCFOUR, AES) depending on how many legacy systems you have to interoperate with. It is possible (and not unreasonable) to get a TGT in ARCFOUR in a Win2K realm, then get a service ticket in 3DES for a MIT machine in that realm. Or, a TGT in AES, and a user2user ticket in ARCFOUR for a downlevel user. And so on... There's another point as well. If there isn't a standard interface, anyone who wants pluggable crypto systems (a reasonable amount of people) will just have to write, debug and maintain their own each time. Python is supposed to be "batteries included" - but not crypto included? The area of DB adaptors has been mentioned - several of those (e.g. psycopg) implement extensions. If any algorithm has problems with the standard interface being too limiting, it can provide an extension. It can provide a whole other class in it's module, or a whole other module. But that doesn't eliminate the utility of having a standard interface. -- Regards, Phil +------------------------------------------+ | Phil Mayers | | Network & Infrastructure Group | | Information & Communication Technologies | | Imperial College | +------------------------------------------+ Quoting Paul Rubin : > It's not very clear to me quite what this means, but I think we should > take as a typical sort of task implementing the TLS or SSH protocols; > for such a task we want a list of good ciphers, hash functions, and > MACs, callable similarly; and in particular we need to be able to > control exactly how padding is done and IVs are computed (the SSH > protocol, for example, gives an algorithm for computing all IVs from > the initial connection secret and the shared secret obtained from it). > So doing the IVs for the poor dumb user is not necessarily a good > idea. > > In practice, 3DES and RC4 are the only ciphers anyone uses in these > protocols, with AES starting to be used in SSH. All conformant TLS > implementations are required to support 3DES, so everything else can > be ignored, unless you're trying to write a protocol analyzer rather > than a communication stack. I believe the same is true of SSH. > ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From akuchlin at mems-exchange.org Tue Apr 23 01:56:10 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Mon, 22 Apr 2002 23:56:10 -0000 Subject: [PYTHON-CRYPTO] ANNOUNCE: Python Cryptography Toolkit 1.9alpha1 Message-ID: <20020422235610.7001.qmail@mozart.mems-exchange.org> I've wrapped up a new, alpha-quality, release of the Python Cryptography Toolkit. It's not backward-compatible with earlier releases, in an effort to fix the design errors I've made in the past. Dodgy algorithms have been dropped, and one new one, AES, has been added. The public-key code is also gone. A list of the changes in this version: * Added Crypto.Cipher.AES. * Added the CTR mode and the variable-sized CFB mode from the NIST standard on feedback modes. * Removed Diamond, HAVAL, MD5, Sapphire, SHA, and Skipjack. MD5 and SHA are included with Python; the others are all of marginal usefulness in the real world. * Renamed the module-level constants ECB, CFB, &c., to MODE_ECB, MODE_CFB, as part of making the block encryption modules compliant with PEP 272. (I'm not sure about this change; if enough users complain about it, I might back it out.) * Made the hashing modules compliant with PEP 247 (not backward compatible -- the major changes are that the constructor is now MD2.new and not MD2.MD2, and the size of the digest is now given as 'digest_size', not 'digestsize'. * The Crypto.PublicKey package is no longer installed; the interfaces are all wrong, and I have no idea what the right interfaces should be. See the TODO file for a list of other fixes I'd like to make before finalizing this as version 2.0. CVS is available through the pycrypto project on SourceForge. The source code is available at http://www.amk.ca/files/python/pycrypto-1.9a1.tar.gz . -- A.M. Kuchling http://www.amk.ca run-time. n. (QA testing) the moment when the programmer shouts "Must run!" and disappears. -- Stan Kelly-Bootle, _The Computer Contradictionary_ From phr-pycrypt at nightsong.com Tue Apr 23 04:49:21 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Tue, 23 Apr 2002 02:49:21 -0000 Subject: [PYTHON-CRYPTO] aes library Message-ID: <20020423024921.20880.qmail@brouhaha.com> [Zooko] > CTR as normally described doesn't have an IV. You must never re-use > a key. But whatever it is called, it isn't true that you must never re-use a key in counter mode encryption. What's true is that you must never re-use a (key, nonce/counter/IV) *pair*. ... I'm not writing this to persuade you (Paul Rubin) of anything (you've already heard my arguments), but to make sure that a reader following this thread didn't think that I agreed with your repeated assertion that "You must never re-use a key." in counter-mode encryption. We don't really disagree; we're just not using the terminology the same way. It's not that I'm using "IV" to refer to what you call something else. Rather, I'm referring to CTR mode as having no IV, no nonce, no counter value supplied separately from the message, nothing like that. Every message uses the same counter sequence: 0, 1, 2, 3, ... . In that usage, you must never encrypt two messages with the same key. Yes, obviously, if use different counter sequences for each message and make sure they never overlap, they can all use the same key. Beware of security failures that might make you re-use a counter value. Right. If I were designing crypto modules for Python, I wouldn't offer CTR mode to the newbie. I think it should be relegated to secondary status so that only people who know they want it will find it. If we're putting something into the standard library, we don't get to choose whether newbies use it or not, and we shouldn't leave standard modes undocumented. However, we should certainly put suitable warnings into the documentation, recommending that newbies use a simplified interface that we'll supply, rather than messing with low level modes. [Phil Mayers] There's another point as well. If there isn't a standard interface, anyone who wants pluggable crypto systems (a reasonable amount of people) will just have to write, debug and maintain their own each time. Python is supposed to be "batteries included" - but not crypto included? PEP 272 requires each module developer to implement a bunch of modes of operation that all do pretty much the same thing. I think if we have a standard interface for block ciphers, it should specify just one mode of operation, namely ECB mode. All other modes can be synthesized from ECB mode by a separate library that we'll supply with the AES module. [Fredrik Huldtgren] I am looking for some information about the blowfish algoritm, mainly how it works. The best description I know of is the Dr. Dobbs article where Blowfish was originally published. It might be available on counterpane.com somewhere. It's also described in Applied Cryptography, by Bruce Schneier (the designer of Blowfish), of course. If you're looking for a general book to learn cryptography implementation from, Applied Cryptography is the standard reference on the subject. Most people on this list have probably read it, so you might like to get a copy too (amazon.de might be a good place to order from, if you're in Sweden). I am planning on writing a Blowfish module in Python(not so much because I am planning on using it, more that I have an intrest for Cryptology and I want to understand it) If anyone knows any good links to pages explaining the algoritm it would be much appreciated(Of course if anyone feels like explaining it to me over e-mail or the such, that works just fine with me too..:) There's already a Blowfish implementation in Python that someone just published a url for, on comp.lang.python. If anyone knows any good links to pages explaining the algoritm it would be much appreciated(Of course if anyone feels like explaining it to me over e-mail or the such, that works just fine with me too..:) You're best off reading the Dr. Dobbs article, or trying a web search, or just reading the code of one of the C implementations, rather than asking people on the list to write long descriptions repeating what's already out there. >>> Dan Stromberg 04/22/02 23:21 PM >>> I'd recommend ignoring blowfish and implementing AES (rijndael) ... I am afraid that AES is a bit too hard to implement? AES is more complicated than Blowfish, but I think a pure Python implementation that made reasonable efforts to run fast would be useful. There's already a pure-Python implementation that was intended mostly for testing purposes and is extremely slow. There's a couple different AES modules written in C, and Bram and I have been writing yet another one, for submission to the standard Python library. [Andrew Kuchling] I've wrapped up a new, alpha-quality, release of the Python Cryptography Toolkit. What are your intentions for this toolkit? Are you going to ask Guido to put it in the standard Python library? Bram and I started work on a module for that purpose, but if you're doing the same thing then we don't need to duplicate your effort. * Renamed the module-level constants ECB, CFB, &c., to MODE_ECB, MODE_CFB, as part of making the block encryption modules compliant with PEP 272. (I'm not sure about this change; if enough users complain about it, I might back it out.) The renaming is fine--don't worry about backward compatibility as long as there's not a lot of users and you haven't yet claimed it's a production release. Re PEP 272 though, my current view is that it should be simplified almost completely, so it only requires one operation to be supported, namely ECB mode encryption or decryption on a single plaintext or ciphertext block. All the other modes should be done by a separate standard module, that can implement the current PEP 272 interfaces or whatever else is desired, using ECB encryptions as a building block. From akuchlin at mems-exchange.org Tue Apr 23 17:24:41 2002 From: akuchlin at mems-exchange.org (Andrew Kuchling) Date: Tue, 23 Apr 2002 11:24:41 -0400 Subject: [PYTHON-CRYPTO] aes library In-Reply-To: <20020423024921.20880.qmail@brouhaha.com> References: <20020423024921.20880.qmail@brouhaha.com> Message-ID: <20020423152441.GD4009@ute.mems-exchange.org> On Tue, Apr 23, 2002 at 02:49:21AM -0000, Paul Rubin wrote: >What are your intentions for this toolkit? Are you going to ask Guido >to put it in the standard Python library? No, I have no plans to do that. --amk From tdi01fhu at SYD.KTH.SE Mon Apr 22 22:58:18 2002 From: tdi01fhu at SYD.KTH.SE (FREDRIK HULDTGREN) Date: Mon, 22 Apr 2002 22:58:18 +0200 Subject: [PYTHON-CRYPTO] Blowfish Message-ID: I am looking for some information about the blowfish algoritm, mainly how it works. :) I am planning on writing a Blowfish module in Python(not so much because I am planning on using it, more that I have an intrest for Cryptology and I want to understand it) If anyone knows any good links to pages explaining the algoritm it would be much appreciated(Of course if anyone feels like explaining it to me over e-mail or the such, that works just fine with me too..:) /Fredrik From tdi01fhu at SYD.KTH.SE Mon Apr 22 23:31:25 2002 From: tdi01fhu at SYD.KTH.SE (FREDRIK HULDTGREN) Date: Mon, 22 Apr 2002 23:31:25 +0200 Subject: [PYTHON-CRYPTO] Blowfish Message-ID: >>> Dan Stromberg 04/22/02 23:21 PM >>> I'd recommend ignoring blowfish and implementing AES (rijndael) if there isn't such a module already. Blowfish's creator, I believe, considers blowfish too immature for production use. AES is going to be the one getting the scrutiny now. I am afraid that AES is a bit too hard to implement? I havn't tried, I've written a few cryto programs in C++ before(A "one-touch-pad" type, and an RSA prog), but nothing advanced when it comes to symmetrical cyphers. If anything , the math might be a bit too much for me.. Got any good pages on AES then? Maybe I'll write both...:) /fredrik On Mon, Apr 22, 2002 at 10:58:18PM +0200, FREDRIK HULDTGREN wrote: > I am looking for some information about the blowfish algoritm, mainly > how it works. > :) From strombrg at NIS.ACS.UCI.EDU Mon Apr 22 23:58:58 2002 From: strombrg at NIS.ACS.UCI.EDU (Dan Stromberg) Date: Mon, 22 Apr 2002 14:58:58 -0700 Subject: [PYTHON-CRYPTO] Blowfish In-Reply-To: ; from tdi01fhu@SYD.KTH.SE on Mon, Apr 22, 2002 at 11:31:25PM +0200 References: Message-ID: <20020422145858.Z1280@seki.acs.uci.edu> On Mon, Apr 22, 2002 at 11:31:25PM +0200, FREDRIK HULDTGREN wrote: > >>> Dan Stromberg 04/22/02 23:21 PM >>> > I'd recommend ignoring blowfish and implementing AES (rijndael) if there > isn't such a module already. Blowfish's creator, I believe, considers > blowfish too immature for production use. AES is going to be the one > getting the scrutiny now. > > I am afraid that AES is a bit too hard to implement? > I havn't tried, I've written a few cryto programs in C++ before(A > "one-touch-pad" type, and an RSA prog), but nothing advanced when it > comes to symmetrical cyphers. > If anything , the math might be a bit too much for me.. > Got any good pages on AES then? Maybe I'll write both...:) I'm afraid I don't have any handy. Apologies. -- Dan Stromberg UCI/NACS/DCS -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 232 bytes Desc: not available URL: From phr-pycrypt at nightsong.com Wed Apr 24 03:15:58 2002 From: phr-pycrypt at nightsong.com (Paul Rubin) Date: Wed, 24 Apr 2002 01:15:58 -0000 Subject: [PYTHON-CRYPTO] SHA-based encryption function Message-ID: <20020424011558.2366.qmail@brouhaha.com> I've put together an encryption function written in Python using the SHA module to provide a keystream in output feedback mode. It's nowhere near as good as AES, but should be a big improvement over the rotor module. It's at http://www.nightsong.com/phr/crypto/p2.py if anyone wants to look at it and make comments. Just say import p2 and then to encrypt, say ciphertext = p2.p2_encrypt(plaintext, passphrase) to decrypt, say plaintext = p2.p2_decrypt(ciphertext, passphrase) There is considerable plaintext expansion and it's slow for short strings, but the module provides an integrity check, and is reasonably fast on long strings (by pure-Python standards). It's about 5x as fast as coding RC4 in pure Python. I'll probably also announce it on sci.crypt and ask for review. If nobody finds any problems with it in maybe a week or two, then I'll release another version without the date restrictions. Future versions are NOT guaranteed to be compatible. This version is for testing only. I believe this code is exportable from the US under BXA exemption TSU and that I've properly notified BXA of its publication on the web. From douglas_shand at YAHOO.CO.UK Tue Apr 30 13:43:46 2002 From: douglas_shand at YAHOO.CO.UK (Douglas Shand) Date: Tue, 30 Apr 2002 04:43:46 -0700 Subject: [PYTHON-CRYPTO] problems compiling m2crypto-0.07-snap3 on OSX Message-ID: <20020430114346.18309.qmail@web12501.mail.yahoo.com> Hi, I am trying to compile m2crypto-0.07-snap3 on OSX to use with Python2.2. I can't seem to get past the following problem _m2crypto_wrap.c:204: header file 'Python.h' not found I have edited the sample OSX makefile to my expected values but I can't get past this. Any help would be most grateful. Thanks, Douglas. __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com From mal at LEMBURG.COM Tue Apr 30 13:59:55 2002 From: mal at LEMBURG.COM (M.-A. Lemburg) Date: Tue, 30 Apr 2002 13:59:55 +0200 Subject: [PYTHON-CRYPTO] problems compiling m2crypto-0.07-snap3 on OSX References: <20020430114346.18309.qmail@web12501.mail.yahoo.com> Message-ID: <3CCE873B.5443F5BE@lemburg.com> Douglas Shand wrote: > > Hi, > > I am trying to compile m2crypto-0.07-snap3 on OSX to > use with Python2.2. > > I can't seem to get past the following problem > > _m2crypto_wrap.c:204: header file 'Python.h' not found Looks like you are missing the Python development package for OS X (if there is such a thing). Python.h is provided by Python, not m2crypto. > I have edited the sample OSX makefile to my expected > values but I can't get past this. > > Any help would be most grateful. > > Thanks, > > Douglas. > > __________________________________________________ > Do You Yahoo!? > Yahoo! Health - your guide to health and wellness > http://health.yahoo.com -- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From douglas_shand at YAHOO.CO.UK Tue Apr 30 14:14:23 2002 From: douglas_shand at YAHOO.CO.UK (Douglas Shand) Date: Tue, 30 Apr 2002 05:14:23 -0700 Subject: [PYTHON-CRYPTO] problems compiling m2crypto-0.07-snap3 on OSX In-Reply-To: <3CCE873B.5443F5BE@lemburg.com> Message-ID: <20020430121423.49577.qmail@web12507.mail.yahoo.com> I have installed Python from source and I have Python.h available in /usr/local/include/python2.2/Python.h but I am not sure which parm in the Makefile.osx actually uses this. Thanks, Douglas. --- "M.-A. Lemburg" wrote: > Douglas Shand wrote: > > > > Hi, > > > > I am trying to compile m2crypto-0.07-snap3 on OSX > to > > use with Python2.2. > > > > I can't seem to get past the following problem > > > > _m2crypto_wrap.c:204: header file 'Python.h' not > found > > Looks like you are missing the Python development > package for OS X (if there is such a thing). > Python.h > is provided by Python, not m2crypto. > > > I have edited the sample OSX makefile to my > expected > > values but I can't get past this. > > > > Any help would be most grateful. > > > > Thanks, > > > > Douglas. > > > > __________________________________________________ > > Do You Yahoo!? > > Yahoo! Health - your guide to health and wellness > > http://health.yahoo.com > > -- > Marc-Andre Lemburg > CEO eGenix.com Software GmbH > ______________________________________________________________________ > Company & Consulting: > http://www.egenix.com/ > Python Software: http://www.egenix.com/files/python/ __________________________________________________ Do You Yahoo!? Yahoo! Health - your guide to health and wellness http://health.yahoo.com