Re: [Security-sig] Unified TLS API for Python
On Jan 13, 2017 3:17 AM, "Nick Coghlan" <ncoghlan@gmail.com> wrote: On 13 January 2017 at 19:33, Nathaniel Smith <njs@pobox.com> wrote:
On Fri, Jan 13, 2017 at 1:09 AM, Cory Benfield <cory@lukasa.co.uk> wrote: I get what you're saying about a typed API -- basically in the current setup, the way _BaseContext is written as a bunch of Python methods means that the interpreter will automatically catch if someone tries to call set_cihpers, whereas in the dict version each implementation would have to catch this itself. But in practice this difference seems really minimal to me -- either way it's a runtime check
The difference isn't minimal at all from the perspective of: - static type analysis - runtime introspection - IDEs (which use a mixture of both) - documentation Config dictionary based APIs have *major* problems in all those respects, as they're ultimately just a bunch of opaque-to-the-compiler keys and values. Network security is a sufficiently hard problem that we want to make it as easy as possible for developers to bring additional tools for ensuring code correctness to bear :)
and it's really difficult to write a loop like
for key, value in config.items(): do_some_quirky_cffi_thing_based_on_key()
that doesn't implicitly validate the keys anyway. There's also the option of using e.g. JSON-Schema to write down a formal description of what goes in the dict -- this could actually be substantially stricter than what Python gets you, because you can actually statically enforce that validate_certificates is a bool. For whatever that's worth -- probably not much.
I've generally found that it's easier to build a declarative API atop a programmatic API than it is to tackle a problem the other way around: 1. Even if the only public API is declarative, there's going to be a programmatic API that implements the actual processing of the declarative requests 2. Working with the programmatic API provides insight into which features the declarative API should cover, and which it can skip 3. Programmatic APIs are often easier to test, since you can more readily isolate the effects of the individual operations 4. A public programmatic API serves as a natural constraint and test case for any subsequent declarative API I totally buy these as advantages of a generic programmatic API over a generic declarative API, but I'm not sure how much it applies given that in this case the "programmatic" API we're talking about is literally *nothing* but getters and setters. Spelling set_ciphers correctly is certainly important and it's nice if your IDE can help, no doubt. But let's keep this in perspective - I don't think this is the *hard* part of network security, really :-). And I can't resist pointing out that in your other email sent at the same time, you're musing about out how nice it would be if only there were some way the stdlib could export its idea of a "default tls configuration" as some sort of concrete object that arbitrary tls implementations could ingest... which is *exactly* what the config-dict approach provides :-). I guess another possible point in the design space would be to split the difference: instead of an abstract context class or a dict, define a concrete context class that acts as a simple transparent container for these settings, effectively layering the type safety etc over its __dict__. -n
On 13 Jan 2017, at 13:09, Nathaniel Smith <njs@pobox.com> wrote:
I guess another possible point in the design space would be to split the difference: instead of an abstract context class or a dict, define a concrete context class that acts as a simple transparent container for these settings, effectively layering the type safety etc over its __dict__.
I think I’d want to do it this way if we were going to do it all: I’m extremely reluctant to use dicts and strings for this if we can possibly avoid it. In fact, you’ll note that the API goes to substantial lengths to avoid passing strings around in almost all cases, except where such a use is required essentially by convention (e.g. paths). While we’re here, I should point out that this does not replace the abstract contexts entirely, because we still need the wrap_* methods. These would now just be fed a Configuration object (I’m more comfortable with the name Configuration than Policy for this usage) in their constructor, and then could use the wrap_* methods as needed. How does that idea sound to the rest of the list? Cory
On 13 January 2017 at 23:56, Cory Benfield <cory@lukasa.co.uk> wrote:
While we’re here, I should point out that this does not replace the abstract contexts entirely, because we still need the wrap_* methods. These would now just be fed a Configuration object (I’m more comfortable with the name Configuration than Policy for this usage) in their constructor, and then could use the wrap_* methods as needed.
Having a relatively passive configuration object sounds OK to me, and I agree it's distinct from a Policy object (which would store constraints on what an acceptable configuration looks like, rather than a specific configuration).
How does that idea sound to the rest of the list?
It would definitely address my concern about making it easy for people to re-use the standard library's default TLS configuration settings, and it would also make it easier to have different defaults for different purposes. So the essential stack would look like: TLSConfig[uration?]: implementation independent, settings only TLSClientContext: ABC to combine settings with a specific TLS implementation TLSServerContext: ABC to combine settings with a specific TLS implementation TLSSocket: ABC to combine a context with a network socket TLSBuffer: ABC to combine a context with a pair of data buffers And then TLSPolicy would be a potential future implementation independent addition that could be used to constrain acceptable TLS configurations. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 13 Jan 2017, at 15:45, Nick Coghlan <ncoghlan@gmail.com> wrote:
So the essential stack would look like:
TLSConfig[uration?]: implementation independent, settings only TLSClientContext: ABC to combine settings with a specific TLS implementation TLSServerContext: ABC to combine settings with a specific TLS implementation TLSSocket: ABC to combine a context with a network socket TLSBuffer: ABC to combine a context with a pair of data buffers
And then TLSPolicy would be a potential future implementation independent addition that could be used to constrain acceptable TLS configurations.
If we were going this way, I’d want to add one extra caveat: I think I’d want the Contexts to become immutable. The logic for the SNI callback would then become: you are called with the Context that created the socket/buffer, and you return a Configuration object that contains any changes you want to make, and the Context applies them if it can (or errors out if it cannot). A new Context is created. This relegates Context to the role of “socket/buffer factory”. The advantage of this is that we have vastly reduced the moving parts: a Context can ensure that, once initiated, the Policy that belongs to it will not change under its feet. It also allows the Context to refuse to change settings that a given concrete implementation cannot change in the SNI callback. Essentially, the logic in the callback would be: def sni_callback(buffer, hostname, context): # This creates a writable copy of the configuration: it does not # mutate the original. configuration = context.configuration configuration.certificates = certs_for_hostname(hostname) configuration.inner_protocols = [NextProtocol.H2, NextProtocol.HTTP1] return configuration This would almost certainly make Context implementation easier, as there is no longer a requirement to monitor your configuration and support live-updates. Cory
On 14 January 2017 at 01:58, Cory Benfield <cory@lukasa.co.uk> wrote:
On 13 Jan 2017, at 15:45, Nick Coghlan <ncoghlan@gmail.com> wrote:
So the essential stack would look like:
TLSConfig[uration?]: implementation independent, settings only TLSClientContext: ABC to combine settings with a specific TLS implementation TLSServerContext: ABC to combine settings with a specific TLS implementation TLSSocket: ABC to combine a context with a network socket TLSBuffer: ABC to combine a context with a pair of data buffers
And then TLSPolicy would be a potential future implementation independent addition that could be used to constrain acceptable TLS configurations.
If we were going this way, I’d want to add one extra caveat: I think I’d want the Contexts to become immutable.
The logic for the SNI callback would then become: you are called with the Context that created the socket/buffer, and you return a Configuration object that contains any changes you want to make, and the Context applies them if it can (or errors out if it cannot). A new Context is created. This relegates Context to the role of “socket/buffer factory”. The advantage of this is that we have vastly reduced the moving parts: a Context can ensure that, once initiated, the Policy that belongs to it will not change under its feet. It also allows the Context to refuse to change settings that a given concrete implementation cannot change in the SNI callback.
Having immutable-by-default config with explicitly managed state changes in particular well-defined scenarios sounds like a big win to me.
This would almost certainly make Context implementation easier, as there is no longer a requirement to monitor your configuration and support live-updates.
It also drastically reduces the test matrix required, as eliminating any potential dependence on the order in which settings are applied, means you only need to test interesting *combinations* of settings, and not different configuration *sequences*. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 2017-01-13 16:58, Cory Benfield wrote:
On 13 Jan 2017, at 15:45, Nick Coghlan <ncoghlan@gmail.com> wrote:
So the essential stack would look like:
TLSConfig[uration?]: implementation independent, settings only TLSClientContext: ABC to combine settings with a specific TLS implementation TLSServerContext: ABC to combine settings with a specific TLS implementation TLSSocket: ABC to combine a context with a network socket TLSBuffer: ABC to combine a context with a pair of data buffers
And then TLSPolicy would be a potential future implementation independent addition that could be used to constrain acceptable TLS configurations.
If we were going this way, I’d want to add one extra caveat: I think I’d want the Contexts to become immutable.
The logic for the SNI callback would then become: you are called with the Context that created the socket/buffer, and you return a Configuration object that contains any changes you want to make, and the Context applies them if it can (or errors out if it cannot). A new Context is created. This relegates Context to the role of “socket/buffer factory”. The advantage of this is that we have vastly reduced the moving parts: a Context can ensure that, once initiated, the Policy that belongs to it will not change under its feet. It also allows the Context to refuse to change settings that a given concrete implementation cannot change in the SNI callback.
Essentially, the logic in the callback would be:
def sni_callback(buffer, hostname, context): # This creates a writable copy of the configuration: it does not # mutate the original. configuration = context.configuration configuration.certificates = certs_for_hostname(hostname) configuration.inner_protocols = [NextProtocol.H2, NextProtocol.HTTP1] return configuration
This would almost certainly make Context implementation easier, as there is no longer a requirement to monitor your configuration and support live-updates.
How would this work for OpenSSL? In OpenSSL the SNI callback replaces the SSL_CTX of a SSL socket pointer with another SSL_CTX. The new SSL_CTX takes care of cipher negotiation, certs and other handshake details. The SSL_CTX should be reused in order to benefit from cached certs, HSM stuff and cached sessions. OpenSSL binds sessions to SSL_CTX instances. A callback looks more like this: contexts = { 'www.example.org': SSLContext(cert1, key1), 'internal.example.com': SSLContext(cert2, key2), } def sni_callback(sock, hostname): sock.context = contexts[hostname] Christian
On 13 Jan 2017, at 16:35, Christian Heimes <christian@cheimes.de> wrote:
How would this work for OpenSSL? In OpenSSL the SNI callback replaces the SSL_CTX of a SSL socket pointer with another SSL_CTX. The new SSL_CTX takes care of cipher negotiation, certs and other handshake details. The SSL_CTX should be reused in order to benefit from cached certs, HSM stuff and cached sessions. OpenSSL binds sessions to SSL_CTX instances.
A callback looks more like this:
contexts = { 'www.example.org': SSLContext(cert1, key1), 'internal.example.com': SSLContext(cert2, key2), }
def sni_callback(sock, hostname): sock.context = contexts[hostname]
If the goal is to keep those contexts static, the best thing to do is to cache the context based on the configuration. Because configurations should be static they should be hashable, which would mean that the ServerContext can keep an internal dictionary of {configuration: SSLContext}. When the new configuration is returned, it can simply pull the context out of the cache as needed. Cory
On 2017-01-13 17:37, Cory Benfield wrote:
On 13 Jan 2017, at 16:35, Christian Heimes <christian@cheimes.de> wrote:
How would this work for OpenSSL? In OpenSSL the SNI callback replaces the SSL_CTX of a SSL socket pointer with another SSL_CTX. The new SSL_CTX takes care of cipher negotiation, certs and other handshake details. The SSL_CTX should be reused in order to benefit from cached certs, HSM stuff and cached sessions. OpenSSL binds sessions to SSL_CTX instances.
A callback looks more like this:
contexts = { 'www.example.org': SSLContext(cert1, key1), 'internal.example.com': SSLContext(cert2, key2), }
def sni_callback(sock, hostname): sock.context = contexts[hostname]
If the goal is to keep those contexts static, the best thing to do is to cache the context based on the configuration. Because configurations should be static they should be hashable, which would mean that the ServerContext can keep an internal dictionary of {configuration: SSLContext}. When the new configuration is returned, it can simply pull the context out of the cache as needed.
You lost me and I'm tired. My brain is no longer able to follow. I'm calling it a day. A working example or PoC might help to understand your point... :) Christian
On 2017-01-13 16:45, Nick Coghlan wrote:
On 13 January 2017 at 23:56, Cory Benfield <cory@lukasa.co.uk> wrote:
While we’re here, I should point out that this does not replace the abstract contexts entirely, because we still need the wrap_* methods. These would now just be fed a Configuration object (I’m more comfortable with the name Configuration than Policy for this usage) in their constructor, and then could use the wrap_* methods as needed.
Having a relatively passive configuration object sounds OK to me, and I agree it's distinct from a Policy object (which would store constraints on what an acceptable configuration looks like, rather than a specific configuration).
How does that idea sound to the rest of the list?
It would definitely address my concern about making it easy for people to re-use the standard library's default TLS configuration settings, and it would also make it easier to have different defaults for different purposes.
So the essential stack would look like:
TLSConfig[uration?]: implementation independent, settings only TLSClientContext: ABC to combine settings with a specific TLS implementation TLSServerContext: ABC to combine settings with a specific TLS implementation TLSSocket: ABC to combine a context with a network socket TLSBuffer: ABC to combine a context with a pair of data buffers
And then TLSPolicy would be a potential future implementation independent addition that could be used to constrain acceptable TLS configurations.
There aren't many settings that are truly implementation independent. Even ciphers depend on the implementation and version of the TLS provider. Some implementations do support more or less ciphers. Some allow ordering or black listing of algorithms, some do not. Apparently it's even hard to not use the system trust store in some implementations (SecureTransport). How should we deal with e.g. when a TLS implementation does not accept or now about a cipher? I would rather invest time to make TLSConfiguration optional for 99% of all users. Unless you need to connect to a crappy legacy device, a user or developer should not need to mess with TLS settings to get secure options. Some TLS libraries already set sane defaults. OpenSSL is getting there, too. Browsers get it right as well. Even for OpenSSL 1.0.2 (first 1.0.2 release) it is possible to set a secure and future proof cipher list: [openssl/1.0.2]$ bin/openssl ciphers 'DEFAULT:!3DES:!EXPORT:!RC4:!DES:!MD5:!SRP:!PSK:!IDEA:!SEED' | sed 's/:/\n/g' ECDHE-RSA-AES256-GCM-SHA384 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-SHA384 ECDHE-ECDSA-AES256-SHA384 ECDHE-RSA-AES256-SHA ECDHE-ECDSA-AES256-SHA DH-DSS-AES256-GCM-SHA384 DHE-DSS-AES256-GCM-SHA384 DH-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-SHA256 DHE-DSS-AES256-SHA256 DH-RSA-AES256-SHA256 DH-DSS-AES256-SHA256 DHE-RSA-AES256-SHA DHE-DSS-AES256-SHA DH-RSA-AES256-SHA DH-DSS-AES256-SHA DHE-RSA-CAMELLIA256-SHA DHE-DSS-CAMELLIA256-SHA DH-RSA-CAMELLIA256-SHA DH-DSS-CAMELLIA256-SHA ECDH-RSA-AES256-GCM-SHA384 ECDH-ECDSA-AES256-GCM-SHA384 ECDH-RSA-AES256-SHA384 ECDH-ECDSA-AES256-SHA384 ECDH-RSA-AES256-SHA ECDH-ECDSA-AES256-SHA AES256-GCM-SHA384 AES256-SHA256 AES256-SHA CAMELLIA256-SHA ECDHE-RSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-SHA256 ECDHE-ECDSA-AES128-SHA256 ECDHE-RSA-AES128-SHA ECDHE-ECDSA-AES128-SHA DH-DSS-AES128-GCM-SHA256 DHE-DSS-AES128-GCM-SHA256 DH-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-SHA256 DHE-DSS-AES128-SHA256 DH-RSA-AES128-SHA256 DH-DSS-AES128-SHA256 DHE-RSA-AES128-SHA DHE-DSS-AES128-SHA DH-RSA-AES128-SHA DH-DSS-AES128-SHA DHE-RSA-CAMELLIA128-SHA DHE-DSS-CAMELLIA128-SHA DH-RSA-CAMELLIA128-SHA DH-DSS-CAMELLIA128-SHA ECDH-RSA-AES128-GCM-SHA256 ECDH-ECDSA-AES128-GCM-SHA256 ECDH-RSA-AES128-SHA256 ECDH-ECDSA-AES128-SHA256 ECDH-RSA-AES128-SHA ECDH-ECDSA-AES128-SHA AES128-GCM-SHA256 AES128-SHA256 AES128-SHA CAMELLIA128-SHA
On 14 January 2017 at 02:20, Christian Heimes <christian@cheimes.de> wrote:
There aren't many settings that are truly implementation independent. Even ciphers depend on the implementation and version of the TLS provider. Some implementations do support more or less ciphers. Some allow ordering or black listing of algorithms, some do not. Apparently it's even hard to not use the system trust store in some implementations (SecureTransport). How should we deal with e.g. when a TLS implementation does not accept or now about a cipher?
That problem is one we're going to have to resolve regardless, and could potentially become its own configuration setting.
I would rather invest time to make TLSConfiguration optional for 99% of all users. Unless you need to connect to a crappy legacy device, a user or developer should not need to mess with TLS settings to get secure options. Some TLS libraries already set sane defaults. OpenSSL is getting there, too. Browsers get it right as well.
Sure, but there are also plenty of folks that *are* going to need to tinker with those settings: - stdlib maintainers - folks writing TLS servers - folks writing TLS client libraries - folks writing test cases for TLS servers - folks writing test cases for TLS clients - penetration testers - network security researchers - folks integrating with legacy infrastructure - embedded software developers - etc We want to provide those folks with a reasonable learning curve as they move away from "the defaults are good enough for me" to "I need to make these changes, for these reasons", even as we attempt to make sure that "routine" use cases are secure by default. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
participants (4)
-
Christian Heimes -
Cory Benfield -
Nathaniel Smith -
Nick Coghlan