From hjtoi at COMCAST.NET Sat Nov 1 08:00:18 2008 From: hjtoi at COMCAST.NET (Heikki Toivonen) Date: Sat, 1 Nov 2008 00:00:18 -0700 Subject: [PYTHON-CRYPTO] Are verify callbacks global in M2Crypto? In-Reply-To: <490B1F5E.3010008@gmail.com> References: <490B1F5E.3010008@gmail.com> Message-ID: <490BFE82.2070709@comcast.net> Aaron C Spike wrote: > Using M2Crypto 0.18.2 on Ubuntu Hardy Heron, it seems that the callbacks > passed to Context.set_verify() are shared between Contexts and therefore > only a single callback is used. I've created a test case to illustrate Yes, this is correct. There is an old bug on this issue: https://bugzilla.osafoundation.org/show_bug.cgi?id=3125 > In the above example cb1.counter remains 0. My goal is to serve requests > on two sockets with different requirements for the clients which connect > to each. While this example is seen from the client perspective I > experience the same behavior on the server. I would appreciate advice on > how I can correct my code to function as I expect. The question I have is: do you really need different contexts to have different verify callback functions? I would assume most applications need just a single verify callback. After all, in what situations would you want to change certificate verification? The post connection check where you'll check the certificate hostname matches peer's address can be done after certificate validation, so this does not need a connection specific verification function. The verification callback can carry connection specific information, and people did this with the old style verification callbacks. This is now deprecated (although it still works). AFAIK it should be possible to do this with the new style callbacks as well, but I run into some problems in trying to implement this for Python. Unfortunately I don't remember the exact issues. I just remember I saw how to do it in C, but couldn't figure out a good way to implement that for M2Crypto. -- Heikki Toivonen - http://heikkitoivonen.net From acspike at GMAIL.COM Sat Nov 1 17:10:40 2008 From: acspike at GMAIL.COM (Aaron Spike) Date: Sat, 1 Nov 2008 11:10:40 -0500 Subject: [PYTHON-CRYPTO] Are verify callbacks global in M2Crypto? In-Reply-To: <490BFE82.2070709@comcast.net> References: <490B1F5E.3010008@gmail.com> <490BFE82.2070709@comcast.net> Message-ID: <7ccdda150811010910t78d86c40nc4b74756d5d1e2dc@mail.gmail.com> On Sat, Nov 1, 2008 at 2:00 AM, Heikki Toivonen wrote: > > Aaron C Spike wrote: > > Using M2Crypto 0.18.2 on Ubuntu Hardy Heron, it seems that the callbacks > > passed to Context.set_verify() are shared between Contexts and therefore > > only a single callback is used. I've created a test case to illustrate > > Yes, this is correct. There is an old bug on this issue: > https://bugzilla.osafoundation.org/show_bug.cgi?id=3125 Thanks for pointing that out. Sorry I missed it. I think the arguments presented in the bug report are convincing. Perhaps if this behavior can't be corrected it could simply be documented more clearly in the API docs. > > In the above example cb1.counter remains 0. My goal is to serve requests > > on two sockets with different requirements for the clients which connect > > to each. While this example is seen from the client perspective I > > experience the same behavior on the server. I would appreciate advice on > > how I can correct my code to function as I expect. > > The question I have is: do you really need different contexts to have > different verify callback functions? This whole SSL thing is pretty new to me, so I apologize if I don't follow all of your logic. >From my understanding of the API the ratio of verify callbacks to ssl contexts is 1:1. If that is true, wouldn't I need different contexts to have different verify callback functions? > I would assume most applications need just a single verify callback. > After all, in what situations would you want to change certificate > verification? It isn't that I want to change certificate verification during execution. I want verification to be different for each socket. I would like to provide two xmlrpc services. Some clients will be authorized to connect to one service; others will be authorized to connect to the other service. I planned to do this authorization check with the verify callback by checking the cert that the clients present against a predetermined list of acceptable certs for each service. Perhaps there is a better way to accomplish this goal? > The post connection check where you'll check the certificate hostname > matches peer's address can be done after certificate validation, so this > does not need a connection specific verification function. Where can I look for an example of this? I suppose I could follow the same process to check that the presented cert is in the list of acceptable certs. > The verification callback can carry connection specific information, and > people did this with the old style verification callbacks. This is now > deprecated (although it still works). AFAIK it should be possible to do > this with the new style callbacks as well, but I run into some problems > in trying to implement this for Python. Unfortunately I don't remember > the exact issues. I just remember I saw how to do it in C, but couldn't > figure out a good way to implement that for M2Crypto. I'm not sure I understand how connection specific information can flow through the new style callbacks. Do you know how any of the other OpenSSL wrappers for python handle this issue? Aaron Spike From simon at SUPERDUPER.NET Tue Nov 4 20:59:56 2008 From: simon at SUPERDUPER.NET (Simon Barber) Date: Tue, 4 Nov 2008 11:59:56 -0800 Subject: [PYTHON-CRYPTO] certificate verification Message-ID: <016001c93eb7$ec195430$276f010d@corp.ad.parc.com> I am writing a test case, and want to verify a certificate I've made - without using an SSL connection. I've tried following the certificate verification example in the "OpenSSL" O'Reilly book - Is this a good approach or is there a better way? The example code does this: X509_STORE_new() X509_STORE_load_locations() X509_STORE_set_detault_paths() X509_STORE_set_flags() X509_STORE_CTX_new() X509_STORE_CTX_init() X509_verify_cert() but some of the C functions don't appear in the M2Crypto wrappers. My first problem is that X509.X509_Store_Context() requires a x509_store_ctx be passed in - where do I get one? Next problem - what is the right way to call the C functions that are not wrapped? I have my root CA cert in an X509 structure, not in a file, so I'd rather not use X509_STORE_load_locations(). Any help much appreciated. Simon -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at EGENIX.COM Wed Nov 5 12:17:21 2008 From: info at EGENIX.COM (eGenix Team: M.-A. Lemburg) Date: Wed, 5 Nov 2008 12:17:21 +0100 Subject: [PYTHON-CRYPTO] certificate verification In-Reply-To: <016001c93eb7$ec195430$276f010d@corp.ad.parc.com> References: <016001c93eb7$ec195430$276f010d@corp.ad.parc.com> Message-ID: <491180C1.6060406@egenix.com> On 2008-11-04 20:59, Simon Barber wrote: > I am writing a test case, and want to verify a certificate I've made - > without using an SSL connection. I've tried following the certificate > verification example in the "OpenSSL" O'Reilly book - Is this a good > approach or is there a better way? The example code does this: > > > > X509_STORE_new() > > X509_STORE_load_locations() > > X509_STORE_set_detault_paths() > > X509_STORE_set_flags() > > X509_STORE_CTX_new() > > X509_STORE_CTX_init() > > X509_verify_cert() > > > > but some of the C functions don't appear in the M2Crypto wrappers. My first > problem is that X509.X509_Store_Context() requires a x509_store_ctx be > passed in - where do I get one? Next problem - what is the right way to call > the C functions that are not wrapped? I have my root CA cert in an X509 > structure, not in a file, so I'd rather not use X509_STORE_load_locations(). I'm not sure how you'd do this in M2Crypto, but in pyOpenSSL, you'd create an SSL Context object, load the certs and then create a loopback Connection object to test the SSL verification. http://pyopenssl.sourceforge.net/pyOpenSSL.html/ You can even use a loopback socket object that only behaves like a socket and never actually uses any connections. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 05 2008) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ :::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 From morel.jeremy at GMAIL.COM Thu Nov 6 12:54:36 2008 From: morel.jeremy at GMAIL.COM (Jeremy Morel) Date: Thu, 6 Nov 2008 12:54:36 +0100 Subject: [PYTHON-CRYPTO] M2Crypto and CAcert.org certificates Message-ID: Hi ! I was wondering if you had the time to take a look at my issue. I know I'm a little pushy, but I really can't figure out why I get a "no content" error and this has been holding me back on my project for months now. J?r?my From hjtoi at COMCAST.NET Sat Nov 8 07:14:44 2008 From: hjtoi at COMCAST.NET (Heikki Toivonen) Date: Fri, 7 Nov 2008 22:14:44 -0800 Subject: [PYTHON-CRYPTO] M2Crypto and CAcert.org certificates In-Reply-To: References: Message-ID: <49152E54.6000106@comcast.net> Jeremy Morel wrote: > I was wondering if you had the time to take a look at my issue. I know I'm a > little pushy, but I really can't figure out why I get a "no content" error > and this has been holding me back on my project for months now. Sorry, I haven't had the time. Hopefully this weekend. -- Heikki Toivonen - http://heikkitoivonen.net From hjtoi at COMCAST.NET Sat Nov 8 07:21:59 2008 From: hjtoi at COMCAST.NET (Heikki Toivonen) Date: Fri, 7 Nov 2008 22:21:59 -0800 Subject: [PYTHON-CRYPTO] Looking for volunteers to run M2Crypto Tinderbox buildslaves Message-ID: <49153007.90308@comcast.net> I have set up a Tinderbox2 instance for M2Crypto. If you are unfamiliar with Tinderbox, it is a tool to do continuous builds and tests, similar to Buildbot and CruiseControl. Now I am looking for people who would be able to run buildlaves. Ideally computers that are online 24/7, able to do a build + test of M2Crypto once an hour, but even once a day would be useful. The computer should have the ability to pull sources from svn and email the build & test log. The build and test cycle takes less than a minute on my laptop, so we are talking minimal resource usage. -- Heikki Toivonen - http://heikkitoivonen.net From koniosis at GMAIL.COM Mon Nov 17 11:16:49 2008 From: koniosis at GMAIL.COM (C. Collis) Date: Mon, 17 Nov 2008 11:16:49 +0100 Subject: [PYTHON-CRYPTO] M2Crypto RSA PKCS #1 v2.1 PSS (RSASSA-PSS) Padding Message-ID: I have a question I was hoping someone could answer. I need to use PKCS #1 v2.1 PSS (RSASSA-PSS) padding as part of my RSA sign and verify process, described here: ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-1/pkcs-1v2-1.pdf I know this can be done using the OpenSSL library using the functions from: openssl\crypto\rsa\rsa_pss.c RSA_verify_PKCS1_PSS() And RSA_padding_add_PKCS1_PSS() Does M2Crypto support this ability, if not, I would be happy to help add it. If there is no support how hard would it be to gain access to these two functions from rsa_pcc.c from within Python? Any help greatly appriciated. I can't seem to find hardly any information on PKCS-PSS with OpenSSL or just in general! Thank you, Chris From koniosis at GMAIL.COM Tue Nov 18 14:21:11 2008 From: koniosis at GMAIL.COM (C. Collis) Date: Tue, 18 Nov 2008 14:21:11 +0100 Subject: [PYTHON-CRYPTO] M2Crypto RSA PKCS #1 v2.1 PSS (RSASSA-PSS) Padding Message-ID: Ok, I added this support and have submitted a patch under BugZilla: https://bugzilla.osafoundation.org/show_bug.cgi?id=12442 I also added a unit test. Feedback welcome. From koniosis at GMAIL.COM Fri Nov 21 11:02:54 2008 From: koniosis at GMAIL.COM (C. Collis) Date: Fri, 21 Nov 2008 11:02:54 +0100 Subject: [PYTHON-CRYPTO] AES 128 CBC encrypt decrypt size problems Message-ID: Hey, I'm trying to implement rfc 3566 in Python using M2Crypto. There seem to be some issues however! Given this python code: -------------------------------------- from M2Crypto import EVP import StringIO def encrypt(data, key, iv): buffer = StringIO.StringIO() cipher = EVP.Cipher('aes_128_cbc', key=key, iv=iv, op=1) buffer.write(cipher.update(data)) buffer.write(cipher.final()) data = buffer.getvalue() return data def decrypt(data, key, iv): buffer = StringIO.StringIO() cipher = EVP.Cipher('aes_128_cbc', key=key, iv=iv, op=0) buffer.write(cipher.update(data)) buffer.write(cipher.final()) data = buffer.getvalue() return data key = bytearray('\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f') pt = bytearray('\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01') iv = bytearray('\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00') ct = encrypt(pt, key, iv) print "PT Len:", len(pt), "Data: ", repr([hex(x) for x in pt]) print "CT Len:", len(ct), "Data: ", repr([hex(ord(x)) for x in ct]) pt2 = decrypt(ct, key, iv) assert pt == pt2 -------------------------------------- The CT len is 32 bytes! According to the RFC it should be only 16 bytes. I tried exactly the same test in C# .NET using the System.Security.Cryptography library and I get the correct, expected 16 byte CT value. What is strange is that M2Crypto and C# produce the same first 16 bytes of encrypted data, but M2Crypto then adds another 16 bytes of junk to the end! If I try and decrypt the CT data (32 bytes) that M2Crypto produces in C#, I get a 32 byte PT value where the first 16 bytes are the correct value and the second 16 bytes are junk! If I try the other way around and use the encrypted CT data from C# (16 bytes) and try and decrypt it using M2Crypto I get a 15 byte PT value that is correct but missing the last byte! Why is M2Crypto doubling the size of my PT when encrypting and why can't it handle a 16 byte CT value correctly? In C# I can decrypt the 16 byte CT value back to the 16 byte PT value as expected. This is making it impossible to implement the RFC as M2Crypto doesn't appear to produce the correct results for AES-128-CBC mode encryption! While M2Crypto can encrypt and decrypt data processed by itself it is unable to work with other crypto implementations, what am I missing? The C# Program is: --------------------------- using System; using System.IO; using System.Security.Cryptography; namespace ConsoleApplication1 { class Program { static void Main(string[] args) { EncryptK1(); DecryptPythonK1CT(); Console.ReadKey(); } static byte[] K1 = new byte[] { 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01 }; static byte[] IV = new byte[] { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }; static byte[] KEY = new byte[] { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f }; public static void DecryptPythonK1CT() { Console.WriteLine("Decrypting Python Encrypted K1 Value using C#..."); byte[] ct = new byte[] { 195, 82, 128, 87, 84, 35, 127, 49, 26, 192, 255, 244, 227, 224, 62, 120, 169, 125, 222, 82, 14, 200, 65, 189, 3, 69, 101, 3, 189, 71, 131, 144 }; byte[] pt = Decrypt(KEY, IV, ct); Console.Write("K1 Python CT:\t"); Print(ct); Console.Write("K1 C# PT:\t"); Print(pt); Console.WriteLine(""); } public static void EncryptK1() { Console.WriteLine("Encrypting K1 Value using C#..."); byte[] ct = Encrypt(KEY, IV, K1); Console.Write("K1 PT:\t\t"); Print(K1); Console.Write("K1 C# CT:\t"); Print(ct); Console.WriteLine(""); } static private void Print(byte[] b) { for (int i = 0; i < b.Length; i++) { Console.Write(string.Format("{0:X}", b[i])); } Console.WriteLine(); } private static byte[] Encrypt(byte[] key, byte[] iv, byte[] data) { byte[] ciphertext = new byte[0]; Rijndael cipher = RijndaelManaged.Create(); cipher.Padding = PaddingMode.None; cipher.Mode = CipherMode.CBC; ICryptoTransform encryptor = cipher.CreateEncryptor(key, iv); using (MemoryStream memoryStream = new MemoryStream()) { using (CryptoStream cryptoStream = new CryptoStream( memoryStream, encryptor, CryptoStreamMode.Write)) { cryptoStream.Write(data, 0, data.Length); cryptoStream.FlushFinalBlock(); ciphertext = memoryStream.ToArray(); } } return ciphertext; } private static byte[] Decrypt(byte[] key, byte[] iv, byte[] data) { byte[] plaintext = new byte[0]; Rijndael cipher = RijndaelManaged.Create(); cipher.Padding = PaddingMode.None; cipher.Mode = CipherMode.CBC; ICryptoTransform decryptor = cipher.CreateDecryptor(key, iv); using (MemoryStream memoryStream = new MemoryStream()) { using (CryptoStream cryptoStream = new CryptoStream( memoryStream, decryptor, CryptoStreamMode.Write)) { cryptoStream.Write(data, 0, data.Length); cryptoStream.FlushFinalBlock(); plaintext = memoryStream.ToArray(); } } return plaintext; } } } --------------------------- The output of this C# program is: ----------------------- Encrypting K1 Value using C#... K1 PT: 1111111111111111 K1 C# CT: C352805754237F311AC0FFF4E3E03E78 Decrypting Python Encrypted K1 Value using C#... K1 Python CT: C352805754237F311AC0FFF4E3E03E78A97DDE52EC841BD345653BD478390 K1 C# PT: 111111111111111110101010101010101010101010101010 ----------------------- Any help greatly appriciated!! Thanks From koniosis at GMAIL.COM Fri Nov 21 11:21:42 2008 From: koniosis at GMAIL.COM (C. Collis) Date: Fri, 21 Nov 2008 11:21:42 +0100 Subject: [PYTHON-CRYPTO] AES 128 CBC encrypt decrypt size problems Message-ID: I may have found something, if I set the padding in C# to PKCS7 rather than None I get the same 32 byte output as M2Crypto. Now I just need to figure out how to turn padding off in M2Crypto... From koniosis at GMAIL.COM Fri Nov 21 12:05:25 2008 From: koniosis at GMAIL.COM (C. Collis) Date: Fri, 21 Nov 2008 12:05:25 +0100 Subject: [PYTHON-CRYPTO] AES 128 CBC encrypt decrypt size problems Message-ID: I've implemented this for the EVP module and added a patch to bugzilla: https://bugzilla.osafoundation.org/show_bug.cgi?id=12445 From barmeier at BARMEIER.COM Mon Nov 24 18:14:45 2008 From: barmeier at BARMEIER.COM (Matthias Barmeier) Date: Mon, 24 Nov 2008 18:14:45 +0100 Subject: [PYTHON-CRYPTO] SSL Through Proxy with M2Crypto always gives: NoCertificate: peer did not return certificate Message-ID: <492AE105.5020907@barmeier.com> Hi, I tried to setup a SSL Connection through a Squid proxy. When I observe the traffic with tshark everything looks fine. The proxies thsark output looks like this : 0.000000 192.168.2.99 -> 82.165.xx.xx TCP 38952 > https [SYN] Seq=0 Len=0 MSS=1460 TSV=245716826 TSER=0 WS=6 0.055182 82.165.xx.xx -> 192.168.2.99 TCP https > 38952 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1452 TSV=763650716 TSER=245716826 WS=5 0.055232 192.168.2.99 -> 82.165.xx.xx TCP 38952 > https [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=245716840 TSER=763650716 0.058386 192.168.2.99 -> 82.165.xx.xx SSL Client Hello 0.060498 192.168.2.99 -> 82.165.xx.xx TCP 38952 > https [FIN, ACK] Seq=76 Ack=1 Win=5888 Len=0 TSV=245716842 TSER=763650716 0.113449 82.165.xx.xx -> 192.168.2.99 TCP https > 38952 [ACK] Seq=1 Ack=76 Win=5792 Len=0 TSV=763650731 TSER=245716841 0.153361 82.165.xx.xx -> 192.168.2.99 TCP https > 38952 [ACK] Seq=1 Ack=77 Win=5792 Len=0 TSV=763650741 TSER=245716842 0.973093 82.165.xx.xx -> 192.168.2.99 TLSv1 Server Hello, 0.973135 192.168.2.99 -> 82.165.xx.xx TCP 38952 > https [RST] Seq=77 Len=0 0.974988 82.165.xx.xx -> 192.168.2.99 TLSv1 Certificate 0.975018 192.168.2.99 -> 82.165.xx.xx TCP 38952 > https [RST] Seq=77 Len=0 0.975091 82.165.xx.xx -> 192.168.2.99 TLSv1 Server Key Exchange 0.975112 192.168.2.99 -> 82.165.xx.xx TCP 38952 > https [RST] Seq=77 Len=0 0.975862 82.165.xx.xx -> 192.168.2.99 TCP https > 38952 [FIN, ACK] Seq=3046 Ack=77 Win=5792 Len=0 TSV=763650946 TSER=245716842 0.975898 192.168.2.99 -> 82.165.xx.xx TCP 38952 > https [RST] Seq=77 Len=0 But the tshark output on the client side is: 0.000000 192.168.38.175 -> 192.168.38.2 TCP 36523 > ndl-aas [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=7661849 TSER=0 WS=7 0.000133 192.168.38.2 -> 192.168.38.175 TCP ndl-aas > 36523 [SYN, ACK] Seq=0 Ack=1 Win=5792 Len=0 MSS=1460 TSV=245777285 TSER=7661849 WS=6 0.000165 192.168.38.175 -> 192.168.38.2 TCP 36523 > ndl-aas [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=7661849 TSER=245777285 0.000200 192.168.38.175 -> 192.168.38.2 HTTP CONNECT ms2.sourcepark.de:443 HTTP/1.0 0.000323 192.168.38.2 -> 192.168.38.175 TCP ndl-aas > 36523 [ACK] Seq=1 Ack=104 Win=5824 Len=0 TSV=245777285 TSER=7661849 0.055803 192.168.38.2 -> 192.168.38.175 HTTP HTTP/1.0 200 Connection established 0.055814 192.168.38.175 -> 192.168.38.2 TCP 36523 > ndl-aas [ACK] Seq=104 Ack=40 Win=5888 Len=0 TSV=7661863 TSER=245777299 0.056706 192.168.38.175 -> 192.168.38.2 SSL Client Hello 0.056813 192.168.38.2 -> 192.168.38.175 TCP ndl-aas > 36523 [ACK] Seq=40 Ack=179 Win=5824 Len=0 TSV=245777299 TSER=7661864 0.058911 192.168.38.175 -> 192.168.38.2 TCP 36523 > ndl-aas [FIN, ACK] Seq=179 Ack=40 Win=5888 Len=0 TSV=7661864 TSER=245777299 0.059234 192.168.38.2 -> 192.168.38.175 TCP ndl-aas > 36523 [FIN, ACK] Seq=40 Ack=180 Win=5824 Len=0 TSV=245777300 TSER=7661864 0.059249 192.168.38.175 -> 192.168.38.2 TCP 36523 > ndl-aas [ACK] Seq=180 Ack=41 Win=5888 Len=0 TSV=7661864 TSER=245777300 as you can see the "TLSv1 Server Key Exchange" never reaches the client. For me it looks like the client closes the connection before the packet arrives. What can I do make it work ?? Ciao Matthias