[Twisted-Python] SSLContext not valid for TLS Server
Hi, I'm running Twisted 17.5.0 on Python 3.5.3 and want to create a TLS server with Twisted. I strictly sticked to the example of echoserv_ssl.py on http://twistedmatrix.com/documents/current/core/howto/ssl.html, but TLS server is not running properly. Despite the server starts correctly, it doesn't offer any cipher suites to the client, no matter what kind of client is trying to connect. Any time a client connects, the connection is immediately aborted by server with the error message of: <class 'OpenSSL.SSL.Error'>: [('SSL routines','tls_post_process_client_hello',no shared cipher')] I tried to connect to server with the TLS echo client example of echoclient_ssl.py, with openssl s_client command and with nmap by using the --ssl-enum-ciphers script. Each time it failed with the above error message. If I do not use the options offered by ssl.CertificateOptions() as suggested in the example, but instead create a ssl.DefaultOpenSSLContextFactory() where I provide the privateKey and Certificate as filename-strings, the server works correctly and offers the intended cipher suites. But I need to create the SSLContext from a OpenSSL PKey-Object (private Key) and a OpenSSL Certificate-Object. Therefore I adjusted the server code of the TLS server example to: certificate = ssl.Certificate(cert_obj) privkey = ssl.KeyPair(pkey) # pkey is the OpenSSL PKey object prkey_and_cert = ssl.PrivateCertificate.fromCertificateAndKeyPair(certificate,privkey) factory = protocol.Factory.forProtocol(Echo) reactor.listenSSL(7498,factory,prkey_and_cert.options()) Again, the server starts, but it does not provide cipher suites so that no client can connect. Same as above! Appreciate any help!
On Aug 16, 2017, at 1:15 PM, ceeborraa <ceeborraa@gmx.de> wrote:
Hi,
I'm running Twisted 17.5.0 on Python 3.5.3 and want to create a TLS server with Twisted. I strictly sticked to the example of echoserv_ssl.py on http://twistedmatrix.com/documents/current/core/howto/ssl.html, but TLS server is not running properly.
Despite the server starts correctly, it doesn't offer any cipher suites to the client, no matter what kind of client is trying to connect. Any time a client connects, the connection is immediately aborted by server with the error message of:
<class 'OpenSSL.SSL.Error'>: [('SSL routines','tls_post_process_client_hello',no shared cipher')]
I tried to connect to server with the TLS echo client example of echoclient_ssl.py, with openssl s_client command and with nmap by using the --ssl-enum-ciphers script. Each time it failed with the above error message.
If I do not use the options offered by ssl.CertificateOptions() as suggested in the example, but instead create a ssl.DefaultOpenSSLContextFactory() where I provide the privateKey and Certificate as filename-strings, the server works correctly and offers the intended cipher suites.
But I need to create the SSLContext from a OpenSSL PKey-Object (private Key) and a OpenSSL Certificate-Object. Therefore I adjusted the server code of the TLS server example to:
certificate = ssl.Certificate(cert_obj) privkey = ssl.KeyPair(pkey) # pkey is the OpenSSL PKey object prkey_and_cert = ssl.PrivateCertificate.fromCertificateAndKeyPair(certificate,privkey)
factory = protocol.Factory.forProtocol(Echo) reactor.listenSSL(7498,factory,prkey_and_cert.options())
Again, the server starts, but it does not provide cipher suites so that no client can connect. Same as above!
Appreciate any help!
This sounds like it might be a configuration problem with your build of OpenSSL. Can you post the most detailed explanation of what versions of everything you're using that you can? Particularly: your operating system, any installed version of OpenSSL, cryptography? (If you know how to check which OpenSSL cryptography is linked to, that would be great too.) What attributes does the private key have? Thanks for using Twisted, -g
openssl version -a: OpenSSL 1.1.0f 25 May 2017 built on: reproducible build, date unspecified
Again as text mail... sorry! Ok, let me try to provide some detailled information, hope it helps: platform: linux-x86_64 compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/local/ssl\"" -DENGINESDIR="\"/usr/local/lib/engines-1.1\"" -Wa,--noexecstack OPENSSLDIR: "/usr/local/ssl" ENGINESDIR: "/usr/local/lib/engines-1.1"
uname -vr 4.9.0-3-amd64 #1 SMP Debian 4.9.30-2+deb9u2 (2017-06-26)
Installed python3 (3.5.3) packages via pip: cryptography 2.0.3 pyopenssl 17.2.0 Twisted 17.5.0
I linked cryptography with the following flags: CFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib" I think I know where the problem is. It's obviously not Twisted. I'm using an ECC private key from curve secp521r1 and as far as I understand, pyOpenSSL still has some problems with supporting this type of private keys. Its some kind of strange behaviour. I can parse the ECC key from file and do some operations with it (e.g. signing a file) which works well. But if I just call the check() method of the pkey object, I get the error: "Unsupported key type" from pyOpenSSL. I think this is the reason why Twisted cannot get the TLS options from it. A similar situation is being described here: https://github.com/pyca/pyopenssl/issues/291 I tried to follow the workaround being described in the issue from the link above, namely to serialise the private key to cryptography and later to convert it to a pyOpenSSL pkey object, but it still does not support the key type. Maybe you have an idea of how to get those things work? Thank you Gesendet: Freitag, 18. August 2017 um 07:23 Uhr Von: Glyph <glyph@twistedmatrix.com> An: "Twisted general discussion" <twisted-python@twistedmatrix.com> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On Aug 16, 2017, at 1:15 PM, ceeborraa <ceeborraa@gmx.de> wrote:
Hi,
I'm running Twisted 17.5.0 on Python 3.5.3 and want to create a TLS server with Twisted. I strictly sticked to the example of echoserv_ssl.py on http://twistedmatrix.com/documents/current/core/howto/ssl.html, but TLS server is not running properly.
Despite the server starts correctly, it doesn't offer any cipher suites to the client, no matter what kind of client is trying to connect. Any time a client connects, the connection is immediately aborted by server with the error message of:
<class 'OpenSSL.SSL.Error'>: [('SSL routines','tls_post_process_client_hello',no shared cipher')]
I tried to connect to server with the TLS echo client example of echoclient_ssl.py, with openssl s_client command and with nmap by using the --ssl-enum-ciphers script. Each time it failed with the above error message.
If I do not use the options offered by ssl.CertificateOptions() as suggested in the example, but instead create a ssl.DefaultOpenSSLContextFactory() where I provide the privateKey and Certificate as filename-strings, the server works correctly and offers the intended cipher suites.
But I need to create the SSLContext from a OpenSSL PKey-Object (private Key) and a OpenSSL Certificate-Object. Therefore I adjusted the server code of the TLS server example to:
certificate = ssl.Certificate(cert_obj) privkey = ssl.KeyPair(pkey) # pkey is the OpenSSL PKey object prkey_and_cert = ssl.PrivateCertificate.fromCertificateAndKeyPair(certificate,privkey)
factory = protocol.Factory.forProtocol(Echo) reactor.listenSSL(7498,factory,prkey_and_cert.options())
Again, the server starts, but it does not provide cipher suites so that no client can connect. Same as above!
Appreciate any help!
This sounds like it might be a configuration problem with your build of OpenSSL. Can you post the most detailed explanation of what versions of everything you're using that you can? Particularly: your operating system, any installed version of OpenSSL, cryptography? (If you know how to check which OpenSSL cryptography is linked to, that would be great too.) What attributes does the private key have? Thanks for using Twisted, -g _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
Ok, I finally got a solution for my problem. As I know, the TLS server was working with DefaultOpenSSLContextFactory but this only takes file paths to private key/certificate, I created my own SSL-Context file. For anybody who has the same problem: class MySSLContext(ssl.ContextFactory): _context = None def __init__(self,privateKey, certificate, sslmethod=SSL.TLSv1_2_METHOD, _contextFactory=SSL.Context): self.privateKey = privateKey self.certificate = certificate self._contextFactory = _contextFactory self.sslmethod = sslmethod self.cacheContext() def cacheContext(self): if(self._context is None): ctx = self._contextFactory(self.sslmethod) ctx.set_options(SSL.OP_NO_SSLv2) ctx.set_options(SSL.OP_NO_SSLv3) ctx.use_privatekey(self.privateKey) ctx.use_certificate(self.certificate) self._context = ctx def getContext(self): return self._context This context can now work with the EC private key from secp521r1! Gesendet: Mittwoch, 16. August 2017 um 22:15 Uhr Von: ceeborraa <ceeborraa@gmx.de> An: twisted-python@twistedmatrix.com Betreff: [Twisted-Python] SSLContext not valid for TLS Server Hi, I'm running Twisted 17.5.0 on Python 3.5.3 and want to create a TLS server with Twisted. I strictly sticked to the example of echoserv_ssl.py on http://twistedmatrix.com/documents/current/core/howto/ssl.html, but TLS server is not running properly. Despite the server starts correctly, it doesn't offer any cipher suites to the client, no matter what kind of client is trying to connect. Any time a client connects, the connection is immediately aborted by server with the error message of: <class 'OpenSSL.SSL.Error'>: [('SSL routines','tls_post_process_client_hello',no shared cipher')] I tried to connect to server with the TLS echo client example of echoclient_ssl.py, with openssl s_client command and with nmap by using the --ssl-enum-ciphers script. Each time it failed with the above error message. If I do not use the options offered by ssl.CertificateOptions() as suggested in the example, but instead create a ssl.DefaultOpenSSLContextFactory() where I provide the privateKey and Certificate as filename-strings, the server works correctly and offers the intended cipher suites. But I need to create the SSLContext from a OpenSSL PKey-Object (private Key) and a OpenSSL Certificate-Object. Therefore I adjusted the server code of the TLS server example to: certificate = ssl.Certificate(cert_obj) privkey = ssl.KeyPair(pkey) # pkey is the OpenSSL PKey object prkey_and_cert = ssl.PrivateCertificate.fromCertificateAndKeyPair(certificate,privkey) factory = protocol.Factory.forProtocol(Echo) reactor.listenSSL(7498,factory,prkey_and_cert.options()) Again, the server starts, but it does not provide cipher suites so that no client can connect. Same as above! Appreciate any help! _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
On Aug 20, 2017, at 9:30 AM, Thomas Hartwich <ceeborraa@gmx.de> wrote:
Ok, I finally got a solution for my problem. As I know, the TLS server was working with DefaultOpenSSLContextFactory but this only takes file paths to private key/certificate, I created my own SSL-Context file.
For anybody who has the same problem:
Please note that this solution will prevent the use of TLS 1.3 when it is available, among other problems. DefaultOpenSSLContextFactory should be deprecated (I hope someone has the time to do it soon), as is the 'getContext' interface that you're using (you should be using https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.I... <https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.I...> ) so it would be really good to understand what part of the non-deprecated TLS stack is broken for you. -glyph
Yes, you're right for sure. As an alternative I tried to instantiate an object from twisted.internet._sslverify.OpenSSLCertificateOptions (as it is used by PrivateCertificate e.g.): co = OpenSSLCertificateOptions(privateKey=pkey,certificate=cert_obj) Despite it provides a SSL-context, it does not work similarly to the options() method I tried before from PrivateCertificate(). Can you tell me how I can make use of IOpenSSLServerConnectionCreator to create a valid SSL-Context for the TLS server in my case? Thank you! Gesendet: Sonntag, 20. August 2017 um 22:36 Uhr Von: Glyph <glyph@twistedmatrix.com> An: "Twisted general discussion" <twisted-python@twistedmatrix.com> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server On Aug 20, 2017, at 9:30 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]> wrote: Ok, I finally got a solution for my problem. As I know, the TLS server was working with DefaultOpenSSLContextFactory but this only takes file paths to private key/certificate, I created my own SSL-Context file. For anybody who has the same problem: Please note that this solution will prevent the use of TLS 1.3 when it is available, among other problems. DefaultOpenSSLContextFactory should be deprecated (I hope someone has the time to do it soon), as is the 'getContext' interface that you're using (you should be using https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html] ) so it would be really good to understand what part of the non-deprecated TLS stack is broken for you. -glyph_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
On Aug 22, 2017, at 9:16 AM, Thomas Hartwich <ceeborraa@gmx.de> wrote:
Yes, you're right for sure. As an alternative I tried to instantiate an object from twisted.internet._sslverify.OpenSSLCertificateOptions (as it is used by PrivateCertificate e.g.):
co = OpenSSLCertificateOptions(privateKey=pkey,certificate=cert_obj)
Please note that importing names with "._" in them is relying on private API :). The public alias for this is `twisted.internet.ssl.CertificateOptions` https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.Certific... <https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.Certific...>
Despite it provides a SSL-context, it does not work similarly to the options() method I tried before from PrivateCertificate().
Can you tell me how I can make use of IOpenSSLServerConnectionCreator to create a valid SSL-Context for the TLS server in my case?
You should probably just use CertificateOptions - I still would like to understand why it doesn't work ;-). https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.I... <https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.I...> is documented here; this is just the interface you should implement (rather than subclassing ContextFactory and implementing getContext) if you want to do something totally custom with the OpenSSL API rather than Twisted's API; I'd still rather understand why Twisted's API, i.e. CertificateOptions, doesn't work for you. -glyph
Thank you!
Gesendet: Sonntag, 20. August 2017 um 22:36 Uhr Von: Glyph <glyph@twistedmatrix.com> An: "Twisted general discussion" <twisted-python@twistedmatrix.com> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On Aug 20, 2017, at 9:30 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]> wrote: Ok, I finally got a solution for my problem. As I know, the TLS server was working with DefaultOpenSSLContextFactory but this only takes file paths to private key/certificate, I created my own SSL-Context file.
For anybody who has the same problem: Please note that this solution will prevent the use of TLS 1.3 when it is available, among other problems.
DefaultOpenSSLContextFactory should be deprecated (I hope someone has the time to do it soon), as is the 'getContext' interface that you're using (you should be using https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html] ) so it would be really good to understand what part of the non-deprecated TLS stack is broken for you.
-glyph_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
I think I now know why it is not working. As I initially suspected that ECC could be the reasons, it seems to have come true. No matter what kind of ECC curve I use, the current implementation of Twisted always uses prime256v1 curve. Maybe because pyOpenSSL hasn't got full ECC support currently!? (got it from some comments in _sslverify.py) In my setting I use secp521r1 curve and for testing purpose I created a key pair of prime256v1 and this works with CertificateOptions. If you have a look at the implementations of twisted.internet._sslverify you will see that prime256v1 is always used as default curve and it seems that no other curve is being accepted. This should be the reason why CertificateOptions does not work for my ECC key. But somehow it works even with secp521r1, if I use the DefaultOpenSSLContextFactory. So do you know any workaround how it can be fixed that twisted accepts other curves than prime256v1? Thank you! Gesendet: Mittwoch, 23. August 2017 um 06:21 Uhr Von: Glyph <glyph@twistedmatrix.com> An: "Twisted general discussion" <twisted-python@twistedmatrix.com> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server On Aug 22, 2017, at 9:16 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]> wrote: Yes, you're right for sure. As an alternative I tried to instantiate an object from twisted.internet._sslverify.OpenSSLCertificateOptions (as it is used by PrivateCertificate e.g.): co = OpenSSLCertificateOptions(privateKey=pkey,certificate=cert_obj) Please note that importing names with "._" in them is relying on private API :). The public alias for this is `twisted.internet.ssl.CertificateOptions` https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html] Despite it provides a SSL-context, it does not work similarly to the options() method I tried before from PrivateCertificate(). Can you tell me how I can make use of IOpenSSLServerConnectionCreator to create a valid SSL-Context for the TLS server in my case? You should probably just use CertificateOptions - I still would like to understand why it doesn't work ;-). https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html] is documented here; this is just the interface you should implement (rather than subclassing ContextFactory and implementing getContext) if you want to do something totally custom with the OpenSSL API rather than Twisted's API; I'd still rather understand why Twisted's API, i.e. CertificateOptions, doesn't work for you. -glyph Thank you! Gesendet: Sonntag, 20. August 2017 um 22:36 Uhr Von: Glyph <glyph@twistedmatrix.com[mailto:glyph@twistedmatrix.com]> An: "Twisted general discussion" <twisted-python@twistedmatrix.com[mailto:twisted-python@twistedmatrix.com]> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server On Aug 20, 2017, at 9:30 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de][mailto:ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]]> wrote: Ok, I finally got a solution for my problem. As I know, the TLS server was working with DefaultOpenSSLContextFactory but this only takes file paths to private key/certificate, I created my own SSL-Context file. For anybody who has the same problem: Please note that this solution will prevent the use of TLS 1.3 when it is available, among other problems. DefaultOpenSSLContextFactory should be deprecated (I hope someone has the time to do it soon), as is the 'getContext' interface that you're using (you should be using https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html]] ) so it would be really good to understand what part of the non-deprecated TLS stack is broken for you. -glyph_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com[mailto:Twisted-Python@twistedmatrix.com] https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python] _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com[mailto:Twisted-Python@twistedmatrix.com] https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
This is a somewhat-known issue that I’ve had bubbling on the backburner for some time. For a long time PyOpenSSL didn’t automatically load all EC curves and didn’t provide any API to do so, so Twisted told OpenSSL which curve to use. Some time ago PyOpenSSL changed this behaviour to automatically load all curves, which would resolve this issue. The most comprehensive fix here is to do some history spelunking in PyOpenSSL to find out what the lowest version is that has this code block[1] in it, and then only execute the current ecCurve logic if that code block doesn’t appear to have worked. Cory [1]: https://github.com/pyca/pyopenssl/blob/master/src/OpenSSL/SSL.py#L632-L636
On 24 Aug 2017, at 20:40, Thomas Hartwich <ceeborraa@gmx.de> wrote:
I think I now know why it is not working. As I initially suspected that ECC could be the reasons, it seems to have come true. No matter what kind of ECC curve I use, the current implementation of Twisted always uses prime256v1 curve. Maybe because pyOpenSSL hasn't got full ECC support currently!? (got it from some comments in _sslverify.py)
In my setting I use secp521r1 curve and for testing purpose I created a key pair of prime256v1 and this works with CertificateOptions. If you have a look at the implementations of twisted.internet._sslverify you will see that prime256v1 is always used as default curve and it seems that no other curve is being accepted. This should be the reason why CertificateOptions does not work for my ECC key.
But somehow it works even with secp521r1, if I use the DefaultOpenSSLContextFactory. So do you know any workaround how it can be fixed that twisted accepts other curves than prime256v1?
Thank you!
Gesendet: Mittwoch, 23. August 2017 um 06:21 Uhr Von: Glyph <glyph@twistedmatrix.com> An: "Twisted general discussion" <twisted-python@twistedmatrix.com> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On Aug 22, 2017, at 9:16 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]> wrote:
Yes, you're right for sure. As an alternative I tried to instantiate an object from twisted.internet._sslverify.OpenSSLCertificateOptions (as it is used by PrivateCertificate e.g.):
co = OpenSSLCertificateOptions(privateKey=pkey,certificate=cert_obj)
Please note that importing names with "._" in them is relying on private API :). The public alias for this is `twisted.internet.ssl.CertificateOptions` https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html]
Despite it provides a SSL-context, it does not work similarly to the options() method I tried before from PrivateCertificate().
Can you tell me how I can make use of IOpenSSLServerConnectionCreator to create a valid SSL-Context for the TLS server in my case?
You should probably just use CertificateOptions - I still would like to understand why it doesn't work ;-).
https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html] is documented here; this is just the interface you should implement (rather than subclassing ContextFactory and implementing getContext) if you want to do something totally custom with the OpenSSL API rather than Twisted's API; I'd still rather understand why Twisted's API, i.e. CertificateOptions, doesn't work for you.
-glyph
Thank you!
Gesendet: Sonntag, 20. August 2017 um 22:36 Uhr Von: Glyph <glyph@twistedmatrix.com[mailto:glyph@twistedmatrix.com]> An: "Twisted general discussion" <twisted-python@twistedmatrix.com[mailto:twisted-python@twistedmatrix.com]> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On Aug 20, 2017, at 9:30 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de][mailto:ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]]> wrote: Ok, I finally got a solution for my problem. As I know, the TLS server was working with DefaultOpenSSLContextFactory but this only takes file paths to private key/certificate, I created my own SSL-Context file.
For anybody who has the same problem: Please note that this solution will prevent the use of TLS 1.3 when it is available, among other problems.
DefaultOpenSSLContextFactory should be deprecated (I hope someone has the time to do it soon), as is the 'getContext' interface that you're using (you should be using https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html]] ) so it would be really good to understand what part of the non-deprecated TLS stack is broken for you.
-glyph_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com[mailto:Twisted-Python@twistedmatrix.com] https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com[mailto:Twisted-Python@twistedmatrix.com] https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
A quick git blame says that it landed in https://github.com/pyca/pyopenssl/commit/6c6bf865acdd3c5ca5f47b1dbc2903023fd... <https://github.com/pyca/pyopenssl/commit/6c6bf865acdd3c5ca5f47b1dbc2903023fd...>, which exists in 17.0.0+. We require 16.0.0+, so it ought to be a relatively simple version check. - Amber
On 25 Aug 2017, at 18:19, Cory Benfield <cory@lukasa.co.uk> wrote:
This is a somewhat-known issue that I’ve had bubbling on the backburner for some time. For a long time PyOpenSSL didn’t automatically load all EC curves and didn’t provide any API to do so, so Twisted told OpenSSL which curve to use. Some time ago PyOpenSSL changed this behaviour to automatically load all curves, which would resolve this issue.
The most comprehensive fix here is to do some history spelunking in PyOpenSSL to find out what the lowest version is that has this code block[1] in it, and then only execute the current ecCurve logic if that code block doesn’t appear to have worked.
Cory
[1]: https://github.com/pyca/pyopenssl/blob/master/src/OpenSSL/SSL.py#L632-L636
On 24 Aug 2017, at 20:40, Thomas Hartwich <ceeborraa@gmx.de> wrote:
I think I now know why it is not working. As I initially suspected that ECC could be the reasons, it seems to have come true. No matter what kind of ECC curve I use, the current implementation of Twisted always uses prime256v1 curve. Maybe because pyOpenSSL hasn't got full ECC support currently!? (got it from some comments in _sslverify.py)
In my setting I use secp521r1 curve and for testing purpose I created a key pair of prime256v1 and this works with CertificateOptions. If you have a look at the implementations of twisted.internet._sslverify you will see that prime256v1 is always used as default curve and it seems that no other curve is being accepted. This should be the reason why CertificateOptions does not work for my ECC key.
But somehow it works even with secp521r1, if I use the DefaultOpenSSLContextFactory. So do you know any workaround how it can be fixed that twisted accepts other curves than prime256v1?
Thank you!
Gesendet: Mittwoch, 23. August 2017 um 06:21 Uhr Von: Glyph <glyph@twistedmatrix.com> An: "Twisted general discussion" <twisted-python@twistedmatrix.com> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On Aug 22, 2017, at 9:16 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]> wrote:
Yes, you're right for sure. As an alternative I tried to instantiate an object from twisted.internet._sslverify.OpenSSLCertificateOptions (as it is used by PrivateCertificate e.g.):
co = OpenSSLCertificateOptions(privateKey=pkey,certificate=cert_obj)
Please note that importing names with "._" in them is relying on private API :). The public alias for this is `twisted.internet.ssl.CertificateOptions` https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html]
Despite it provides a SSL-context, it does not work similarly to the options() method I tried before from PrivateCertificate().
Can you tell me how I can make use of IOpenSSLServerConnectionCreator to create a valid SSL-Context for the TLS server in my case?
You should probably just use CertificateOptions - I still would like to understand why it doesn't work ;-).
https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html] is documented here; this is just the interface you should implement (rather than subclassing ContextFactory and implementing getContext) if you want to do something totally custom with the OpenSSL API rather than Twisted's API; I'd still rather understand why Twisted's API, i.e. CertificateOptions, doesn't work for you.
-glyph
Thank you!
Gesendet: Sonntag, 20. August 2017 um 22:36 Uhr Von: Glyph <glyph@twistedmatrix.com[mailto:glyph@twistedmatrix.com]> An: "Twisted general discussion" <twisted-python@twistedmatrix.com[mailto:twisted-python@twistedmatrix.com]> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On Aug 20, 2017, at 9:30 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de][mailto:ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]]> wrote: Ok, I finally got a solution for my problem. As I know, the TLS server was working with DefaultOpenSSLContextFactory but this only takes file paths to private key/certificate, I created my own SSL-Context file.
For anybody who has the same problem: Please note that this solution will prevent the use of TLS 1.3 when it is available, among other problems.
DefaultOpenSSLContextFactory should be deprecated (I hope someone has the time to do it soon), as is the 'getContext' interface that you're using (you should be using https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html]] ) so it would be really good to understand what part of the non-deprecated TLS stack is broken for you.
-glyph_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com[mailto:Twisted-Python@twistedmatrix.com] https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com[mailto:Twisted-Python@twistedmatrix.com] https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Thank you for your reply.
This is a somewhat-known issue that I’ve had bubbling on the backburner for some time. For a long time PyOpenSSL didn’t automatically load all EC curves and didn’t provide any API to do so, so Twisted told OpenSSL which curve to use. Some time ago PyOpenSSL changed this behaviour to automatically load all curves, which would resolve this issue. This, I understand so far.
The most comprehensive fix here is to do some history spelunking in PyOpenSSL to find out what the lowest version is that has this code block[1] in it, and then only execute the current ecCurve logic if that code block doesn’t appear to have worked. I don't really get what implies this to me and how I can come around with this issue. What do you mean with "only execute the current ecCurve logic..."? How can this be done?
I have the current versions of Twisted and pyOpenSSL running, so how can I make sure that the latter loads the right curve properly? Thank you!
On 24 Aug 2017, at 20:40, Thomas Hartwich <ceeborraa@gmx.de> wrote:
I think I now know why it is not working. As I initially suspected that ECC could be the reasons, it seems to have come true. No matter what kind of ECC curve I use, the current implementation of Twisted always uses prime256v1 curve. Maybe because pyOpenSSL hasn't got full ECC support currently!? (got it from some comments in _sslverify.py)
In my setting I use secp521r1 curve and for testing purpose I created a key pair of prime256v1 and this works with CertificateOptions. If you have a look at the implementations of twisted.internet._sslverify you will see that prime256v1 is always used as default curve and it seems that no other curve is being accepted. This should be the reason why CertificateOptions does not work for my ECC key.
But somehow it works even with secp521r1, if I use the DefaultOpenSSLContextFactory. So do you know any workaround how it can be fixed that twisted accepts other curves than prime256v1?
Thank you!
Gesendet: Mittwoch, 23. August 2017 um 06:21 Uhr Von: Glyph <glyph@twistedmatrix.com> An: "Twisted general discussion" <twisted-python@twistedmatrix.com> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On Aug 22, 2017, at 9:16 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]> wrote:
Yes, you're right for sure. As an alternative I tried to instantiate an object from twisted.internet._sslverify.OpenSSLCertificateOptions (as it is used by PrivateCertificate e.g.):
co = OpenSSLCertificateOptions(privateKey=pkey,certificate=cert_obj)
Please note that importing names with "._" in them is relying on private API :). The public alias for this is `twisted.internet.ssl.CertificateOptions` https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html][https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.ssl.CertificateOptions.html]]
Despite it provides a SSL-context, it does not work similarly to the options() method I tried before from PrivateCertificate().
Can you tell me how I can make use of IOpenSSLServerConnectionCreator to create a valid SSL-Context for the TLS server in my case?
You should probably just use CertificateOptions - I still would like to understand why it doesn't work ;-).
https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html][https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html]] is documented here; this is just the interface you should implement (rather than subclassing ContextFactory and implementing getContext) if you want to do something totally custom with the OpenSSL API rather than Twisted's API; I'd still rather understand why Twisted's API, i.e. CertificateOptions, doesn't work for you.
-glyph
Thank you!
Gesendet: Sonntag, 20. August 2017 um 22:36 Uhr Von: Glyph <glyph@twistedmatrix.com[mailto:glyph@twistedmatrix.com]> An: "Twisted general discussion" <twisted-python@twistedmatrix.com[mailto:twisted-python@twistedmatrix.com]> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On Aug 20, 2017, at 9:30 AM, Thomas Hartwich <ceeborraa@gmx.de[mailto:ceeborraa@gmx.de][mailto:ceeborraa@gmx.de[mailto:ceeborraa@gmx.de]]> wrote: Ok, I finally got a solution for my problem. As I know, the TLS server was working with DefaultOpenSSLContextFactory but this only takes file paths to private key/certificate, I created my own SSL-Context file.
For anybody who has the same problem: Please note that this solution will prevent the use of TLS 1.3 when it is available, among other problems.
DefaultOpenSSLContextFactory should be deprecated (I hope someone has the time to do it soon), as is the 'getContext' interface that you're using (you should be using https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html][https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html][https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html][https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html[https://twistedmatrix.com/documents/17.5.0/api/twisted.internet.interfaces.IOpenSSLServerConnectionCreator.html]]] ) so it would be really good to understand what part of the non-deprecated TLS stack is broken for you.
-glyph_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com[mailto:Twisted-Python@twistedmatrix.com] https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python][https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]]
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com[mailto:Twisted-Python@twistedmatrix.com] https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python] _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python][https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]]
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python[https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python]
On 27 Aug 2017, at 09:44, Thomas Hartwich <ceeborraa@gmx.de> wrote:
=
The most comprehensive fix here is to do some history spelunking in PyOpenSSL to find out what the lowest version is that has this code block[1] in it, and then only execute the current ecCurve logic if that code block doesn’t appear to have worked. I don't really get what implies this to me and how I can come around with this issue. What do you mean with "only execute the current ecCurve logic..."? How can this be done?
I have the current versions of Twisted and pyOpenSSL running, so how can I make sure that the latter loads the right curve properly?
You cannot, at least not using public APIs. This was a suggestion of an update to Twisted, not something you should be doing from your own code. Cory
Ok thank you! I gonna use the old version with DefaultOpenSSLContextFactory as long as Twisted hasn't been updated. Thomas Gesendet: Montag, 28. August 2017 um 09:02 Uhr Von: "Cory Benfield" <cory@lukasa.co.uk> An: "Twisted general discussion" <twisted-python@twistedmatrix.com> Betreff: Re: [Twisted-Python] SSLContext not valid for TLS Server
On 27 Aug 2017, at 09:44, Thomas Hartwich <ceeborraa@gmx.de> wrote:
=
The most comprehensive fix here is to do some history spelunking in PyOpenSSL to find out what the lowest version is that has this code block[1] in it, and then only execute the current ecCurve logic if that code block doesn’t appear to have worked. I don't really get what implies this to me and how I can come around with this issue. What do you mean with "only execute the current ecCurve logic..."? How can this be done?
I have the current versions of Twisted and pyOpenSSL running, so how can I make sure that the latter loads the right curve properly?
You cannot, at least not using public APIs. This was a suggestion of an update to Twisted, not something you should be doing from your own code. Cory _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (5)
-
Amber Hawkie Brown -
ceeborraa -
Cory Benfield -
Glyph -
Thomas Hartwich