[Python-Dev] LibreSSL support

[Christian Heimes]

On 2018-01-19 15:42, Christian Heimes wrote:
> On 2018-01-19 10:43, Steve Holden wrote:
>> On Fri, Jan 19, 2018 at 12:09 AM, Nathaniel Smith <njs at pobox.com
>> <mailto:njs at pobox.com>> wrote:
>>
>>     On Jan 18, 2018 07:34, "Christian Heimes" <christian at python.org
>>     <mailto:christian at python.org>> wrote:
>>
>>         On 2018-01-16 21:17, Christian Heimes wrote:
>>         > FYI, master on Travis CI now builds and uses OpenSSL 1.1.0g [1]. I have
>>         > created a daily cronjob to populate Travis' cache with OpenSSL builds.
>>         > Until the cache is filled, Linux CI will take an extra 5 minute.
>>
>>         I have messed up my initial research. :( When I was checking
>>         LibreSSL
>>         and OpenSSL for features, I draw a wrong conclusion. LibreSSL is
>>         *not*
>>         OpenSSL 1.0.2 compatible. It only implements some of the required
>>         features from 1.0.2 (e.g. X509_check_hostname) but not
>>         X509_VERIFY_PARAM_set1_host.
>>
>>         X509_VERIFY_PARAM_set1_host() is required to perform hostname
>>         verification during the TLS handshake. Without the function, I'm
>>         unable
>>         to fix Python's hostname matching code [1]. LibreSSL upstream knows
>>         about the issue since 2016 [2]. I have opened another bug report
>>         [3].
>>
>>         We have two options until LibreSSL has addressed the issue:
>>
>>         1) Make the SSL module more secure, simpler and standard conform
>>         2) Support LibreSSL
>>
>>
>>     ​[...]
>>
>>  
>>
>>     We have *very* few people qualified to maintain the ssl module, so
>>     given the new landscape I think we should focus on keeping our core
>>     OpenSSL support solid and not worry about LibreSSL. If LibreSSL
>>     wants to be supported as well then – like any other 2nd tier
>>     platform – they need to find someone to do the work. And if people
>>     are worried about supporting more diversity in SSL implementations,
>>     then PEP 543 is probably the thing to focus on.
>>
>> ​Given the hard limit on resources it seems only sensible to focus on
>> the "industry standard" library​. I'm rather disappointed that LibreSSL
>> isn't a choice, but given the lack of compatibility that's hardly
>> Python's problem.
> 
> Thanks!
> 
> I'd prefer to support LibreSSL, too. Paul Kehrer from PyCA summed up the
> issue with LibreSSL nicely:
> 
>> It was marketed as an API compatible drop-in replacement and it is
> failing in that capacity. Additionally, it is missing features needed to
> improve the security and ease the maintenance burden of CPython’s dev team.
> 
> 
> Since I haven given up on LibreSSL, I spent some time and implemented
> some autoconf magic in https://github.com/python/cpython/pull/5242. It
> checks for the presence of libssl and X509_VERIFY_PARAM_set1_host()
> function family:
> 
> ...
> checking whether compiling and linking against OpenSSL works... yes
> checking for X509_VERIFY_PARAM_set1_host in libssl... yes
> ...
> 
> The ssl module will regain compatibility with LibreSSL as soon as a new
> release provides the necessary functions.

No core developer has vetoed against my proposal. I also spoke to
several members of Python Cryptographic Authority and Python Packaging
Authority. They are all in favor of my proposal, too.

There I have decided to move forward and require OpenSSL 1.0.2 API. This
means Python 3.7 temporarily suspends support for LibreSSL until
https://github.com/libressl-portable/portable/issues/381 is resolved. I
have appended a lengthy explanation to my LibreSSL ticket, too.

I also informed LibreSSL developers that Python 3.8 will most likely
require an OpenSSL 1.1 compatible API. With OpenSSL 1.0.2 support, I can
drop a considerable amount of legacy code, e.g. custom thread locking,
initialization and a bunch of shim functions.

Regards,
Christian