Hello. Can you help me to learn to debug tls problems in twisted?

Hi Daniel! Thanks so much for trying to improve this aspect of the Twisted ecosystem.

I was disappointed that txacme, an automatic way to get certificates for twisted web, stopped working, so I'm trying to add a responder for the new challenge type.

This situation is definitely bad; as JP tersely put it on https://github.com/twisted/txacme/issues/148, "The published documentation makes it appear as though the server endpoint works though it has apparently been broken for about a year".

But to prevent onlookers from thinking the situation is even worse than it is, please allow me to clarify:

txacme still "works" in the sense that it supports the http-01 challenge (which is still supported for the forseeable future, AFAIK), and its challenge/response implementation also powers https://pypi.org/project/lancer/, which uses the dns-01 challenge. However, its endpoints (and particularly its string endpoint plugin) doesn't support this challenge directly.

The thing that's broken is sni-01, which was disabled globally by Let's Encrypt for security reasons: https://www.zdnet.com/article/lets-encrypt-disables-tls-sni-01-validation/

One approach to fixing this is to implement a string endpoint syntax that listens on port 80 and supports HTTP-01 rather than trying to make TLS support TLS-ALPN-01.

See discussion here: https://github.com/twisted/txacme/issues/129.

Brian Warner already has a branch here: https://github.com/warner/txacme/tree/129-http01-endpoint but has not made a PR yet. Perhaps you could resurrect this code, polish anything that needs polishing, and get it integrated and released :).

It sends a special certificate if the CA negotiates acme using alpn, proving domain control, then you get a certificate.

Seems to work with openssl s_client but letsencrypt says "no application protocol". Anyone know where that error comes from?

https://github.com/glyph/txsni/pull/26

I suspect one thing you might be running into is this: https://github.com/pyca/pyopenssl/issues/769, i.e. the issue that caused Certbot to revert their own ALPN support: https://github.com/certbot/certbot/pull/6100. This is an OpenSSL bug and there's not a whole lot Twisted can do about it. See here where I ran into it myself, for this exact same reason: https://github.com/openssl/openssl/issues/7660#issuecomment-462104869

Fundamentally, the ALPN callback in OpenSSL is poorly designed, and optimized for the very specific negotiation process that HTTP/2 requires. The guidance from people familiar with OpenSSL is - and I know this sounds like a joke, I wish I were kidding here, but no, this is seriously the recommendation - implement your own parser for the ClientHello, extract the ALPN field yourself, then construct an OpenSSL connection object based on what you find there, and hand it the bytes you just parsed.