[Python-Dev] Design and direction of the SSL module (was Re: frozenset C API?)

[glyph at divmod.com]

Sorry for the late response.  As always, I have a lot of other stuff 
going on at the moment, but I'm very interested in this subject.

On 6 Sep, 06:15 pm, janssen at parc.com wrote:
>>PyOpenSSL, in particular, is both a popular de-facto
>>standard *and* almost completely unmaintained; python's standard 
>>library
>>could absorb/improve it with little fuss.
>
>Good idea, go for it!  A full wrapper for OpenSSL is beyond the scope
>of my ambition; I'm simply trying to add a simple fix to what's
>already in the standard library.

I guess I'd like to know two things.  One, what *is* the scope of your 
amibition?  I feel silly for asking, because I am pretty sure that 
somewhere in the beginning of this thread I missed either a proposal, a 
PEP reference, or a ticket number, but I've poked around a little and I 
can't seem to find it.  Can you provide a reference, or describe what it 
is you're trying to do?

Two, what's the scope of "the" plans for the SSL module in general for 
Python?  I think I misinterpreted several things that you said as "the 
plan" rather than your own personal requirements: but if in reality, I 
can "go for it", I'd really like to help make the stdlib SSL module to 
be a really good, full-featured OpenSSL implementation for Python so we 
can have it elsewhere.  (If I recall correctly you mentioned you'd like 
to use it with earlier Python versions as well...?)

Many of the things that you recommend using another SSL library for, 
like pulling out arbitrary extensions, are incredibly unweildy or flat- 
out broken in these libraries.  It's not that I mind going to a 
different source for this functionality; it's that in many cases, there 
*isn't* another source :).  I think I might have said this already, but 
subjectAltName, for example, isn't exposed in any way by PyOpenSSL.

I didn't particularly want to start my own brand-new SSL wrapper 
project, and contributing to the actively-maintained stdlib 
implementation is a lot more appealing than forking the moribund 
PyOpenSSL.

However, even with lots of help on the maintenance, converting the 
current SSL module into a complete SSL library is a lot of work.  Here 
are the questions that I'd like answers to before starting to think 
seriously about it:

    * Is this idea even congruent with the overall goals of other 
developers interested in SSL for Python?  If not, I'm obviously barking 
up the wrong tree.
    * Would it be possible to distribute as a separate library?  (I think 
I remember Bill saying something about that already...)
    * When would such work have to be completed by to fit into the 2.6 
release?  (I just want a rough estimate, here.)
    * Should someone - and I guess by someone I mean me - write up a PEP 
describing this?

My own design for an SSL wrapper - although this simply a Python layer 
around PyOpenSSL - is here:

http://twistedmatrix.com/trac/browser/trunk/twisted/internet/_sslverify.py

This isn't really complete - in particular, the documentation is 
lacking, and it can't implement the stuff PyOpenSSL is missing - but I 
definitely like the idea of having objects for DNs, certificates, CRs, 
keys, key pairs, and the ubiquitous certificate-plus-matching-private- 
key-in-one-file that you need to run an HTTPS server :).  If I am going 
to write a PEP, it will look a lot like that file.

_sslverify was originally designed for a system that does lots of 
automatic signing, so I am particularly interested in it being easy to 
implement a method like  PrivateCertificate.signCertificateRequest - 
it's always such a pain to get all the calls for signing a CR in any 
given library *just so*.
>>This begs the question: M2Crypto and PyOpenSSL already do what you're
>>proposing to do, as far as I can tell, and are, as you say, "more
>>powerful".

To clarify my point here, when I say that they "already do" what you're 
doing, what I mean is, they already wrap SSL, and you are trying to wrap 
SSL :).
>I'm trying to give the application the ability to do some level of
>authorization without requiring either of those packages.

I'd say "why wouldn't you want to require either of those packages?" but 
actually, I know why you wouldn't want to, and it's that they're bad. 
So, given that we don't want to require them, wouldn't it be nice if we 
didn't need to require them at all? :).
>Like being
>able to tell who's on the other side of the connection :-).  Right
>now, I think the right fields to expose are

I don't quite understand what you mean by "right" fields.  Right fields 
for what use case?  This definitely isn't "right" for what I want to use 
SSL for.
>  "subject" (I see little point to exposing "issuer"),

This is a good example of what I mean.  For HTTPS, the relationship 
between the subject and the issuer is moot, but in my own projects, the 
relationship is very interesting.  Specifically, properties of the 
issuer define what properties the subject may have, in the verification 
scheme for Vertex ( http://divmod.org/trac/wiki/DivmodVertex ).  (On the 
other hand, Vertex requires STARTTLS, so it itself can't be an *actual* 
use-case for this SSL library until it also starts supporting mid- 
connection TLS startup.)

I can understand that you might not have use-cases for exposing these 
features, but your phrasing suggests that it would be a bad idea to 
expose them, not just that it's too much work.  Am I misinterpreting? 
Are you just saying it isn't worth the work at this point?
>  "notAfter" (you're always guaranteed to be after "notBefore", or the
>  cert wouldn't validate, so I see little point to exposing that, but
>  "notAfter" can be used after the connection has been established),

Wouldn't it be nice to know *why* the cert didn't validate?  To provide 
the user with a message including the notBefore date, in case their 
clock is set wrong or something?
>I don't see how the other fields in the cert can be profitably used.

The entire idea of "extensions" is pretty direct about the fact that the 
original implementor need not understand their profitable use :).
>>When you say "the full DER form", are you simply referring to the full
>>blob, or a broken-down representation by key and by extension?
>
>The full blob.

Obviously, I think the broken-down representation would be nicer :).

I know I'll have to wrangle with a bit of ASN.1 if I want to get 
anything useful out of most extensions, but if it's just the extension 
data there are a lot of cases where I think I could fake it.  Re-parsing 
the whole DER is going to require a real, full-on ASN.1 library.