SSL Certificate Validation
Hi all, I have a few questions about validating SSL certificates. From what I gather, this validation occurs in the OpenSSL code called from _ssl.c. Is this correct? Also, I have looked through the docs and code, but haven't been able to figure out exactly what is included in certificate "validation". Is it just validating the chain? Does it check the NotBefore and NotAfter dates? Does it check that the host the socket is connected to is the same as what's given in the CN field in the certificate? Where I'm going with this is I think all this checking needs to be part of certificate validation in the ssl module. If it isn't yet, I'd be happy to work on a patch for it. Please let me know what you think. Thanks! -Devin Cook
On Tue, Jun 16, 2009 at 3:23 PM, "Martin v. Löwis"<martin@v.loewis.de> wrote:
Martin, I told him to ask his question about _ssl internals on python-dev as he is new, and looking to work on some of the internals/make a patch for core. I didn't think that asking internals questions was a faux pas for the list, especially as he's looking to submit a patch to core.
I don't think that's called for, he is attempting to familiarize himself and simply inquiring about some of the internals. I'm sure he'll know plenty by the time the patch is more fully formed. -jesse
Hmm. For somebody new to Python, I'm fairly skeptical that the SSL module is the best starting point.
But I really do believe that this is what he need to do next: familiarize himself with OpenSSL. There is a lot of APIs in that library, and it takes a while (i.e.: several months) to get productive, in particular since OpenSSL doesn't have the most intuitive API.
From "I want to know what features it currently has" to "I can contribute new features" is really a looong way here.
To give a little more guidance: find out what SSL_CTX_use_certificate_chain_file and SSL_CTX_set_verify do. Finding that out is really out of scope of python-dev, since it has nothing to do with Python. Regards, Martin
Well, I realized this as soon as I looked at the _ssl.c code... I was just hoping that someone would be able to give me a quick clarification on exactly what gets validated. If it's just the chain (which is what I suspect), I would like to submit a patch that does the rest of the validation (that a browser typically does: CN/hostname, NotBefore, NotAfter, etc.) in the ssl module. I was also hoping to find out what the consensus is about this: mainly, *should* that verification be done in the ssl module? Maybe this verification should somehow be done in OpenSSL, which would mean that I need to do a LOT more reading and go pester their mailing list instead. This is for issue 6273 ( http://bugs.python.org/issue6273 ). In your reply to that issue, it seemed to me like you were saying that these things were not getting checked in the ssl module (and, therefore, not in OpenSSL either):
I would like to do validation of server certificates in a project I'm working on, and I figured it would be better to be proactive and try to help create a patch than to just sit back and complain about it. It seems to me that this is a bug that you can't do peer certificate validation in httplib. If this isn't the place to ask these kinds of questions, I apologise. I can take the discussion elsewhere if I need to. Thanks, -Devin
If this isn't the place to ask these kinds of questions, I apologise. I can take the discussion elsewhere if I need to.
It really depends on what "these questions" are. If your question is "I have this patch, is it correct?", then the question is entirely appropriate. If it is "I just have barely looked at the API, can somebody please explain it all to me?", then this isn't appropriate for this list, and probably not appropriate elsewhere: anybody answering this question could just as well fix the original problem right away. So please do try to find the answer for yourself, with the (little) direction I gave. If you find that it takes a lot of effort, then you'll probably have to accept the bug as-is, and live with it. FWIW, I actually don't know the answer for sure, either, so I would have to research this myself, too. In any case, _ssl.c is *not* the place where any of the certificate validation actually happens - nor does it happen elsewhere in the Python source code, IIUC. Regards, Martin
Devin Cook <devin.c.cook@gmail.com> wrote:
I believe so, but you'll have to check the OpenSSL code.
Does it check that the host the socket is connected to is the same as what's given in the CN field in the certificate?
No. That, in general, doesn't work very well. The IETF working group on this is considering deprecating putting a hostname in the CN field at all, and just adding hostnames via the subjectAltName extension. The problem that's come up is that many computers don't have fixed IP addresses, and even with that the hostname is part of a different mapping of hostnames to IP addresses, which can also vary. I think that when the https: protocol scheme was written, it seemed like a good idea, but conventions on the Internet have changed a lot since then.
Where I'm going with this is I think all this checking needs to be part of certificate validation in the ssl module.
I don't think so. I put in hooks to let you do this in user code if you need to. See the archives for more discussion on this -- I'm not going to rehash it again. (This is really a question for OpenSSL mailing lists, or perhaps python-list.) Bill
Bill Janssen <janssen@parc.com> wrote:
Incidentally, the current working draft on this seems to be at <http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00>. Bill
Ok, thanks for all the feedback. Just for clarity, I'll summarize everything as I understand it: * OpenSSL does the all validation of the certificate itself. (http://openssl.org/docs/apps/verify.html) * httplib should have a way to enable validation of the certificate. * httplib should have a way to enable checking of the reference identity. (that complies with section 3 of this draft: http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00) * The reference identity checking (and cert validation, I assume) shouldn't be automatic. (per Bill) Does that sound about right? I'll try to work up a patch tonight implementing this. -Devin
On Tue, Jun 16, 2009 at 3:23 PM, "Martin v. Löwis"<martin@v.loewis.de> wrote:
Martin, I told him to ask his question about _ssl internals on python-dev as he is new, and looking to work on some of the internals/make a patch for core. I didn't think that asking internals questions was a faux pas for the list, especially as he's looking to submit a patch to core.
I don't think that's called for, he is attempting to familiarize himself and simply inquiring about some of the internals. I'm sure he'll know plenty by the time the patch is more fully formed. -jesse
Hmm. For somebody new to Python, I'm fairly skeptical that the SSL module is the best starting point.
But I really do believe that this is what he need to do next: familiarize himself with OpenSSL. There is a lot of APIs in that library, and it takes a while (i.e.: several months) to get productive, in particular since OpenSSL doesn't have the most intuitive API.
From "I want to know what features it currently has" to "I can contribute new features" is really a looong way here.
To give a little more guidance: find out what SSL_CTX_use_certificate_chain_file and SSL_CTX_set_verify do. Finding that out is really out of scope of python-dev, since it has nothing to do with Python. Regards, Martin
Well, I realized this as soon as I looked at the _ssl.c code... I was just hoping that someone would be able to give me a quick clarification on exactly what gets validated. If it's just the chain (which is what I suspect), I would like to submit a patch that does the rest of the validation (that a browser typically does: CN/hostname, NotBefore, NotAfter, etc.) in the ssl module. I was also hoping to find out what the consensus is about this: mainly, *should* that verification be done in the ssl module? Maybe this verification should somehow be done in OpenSSL, which would mean that I need to do a LOT more reading and go pester their mailing list instead. This is for issue 6273 ( http://bugs.python.org/issue6273 ). In your reply to that issue, it seemed to me like you were saying that these things were not getting checked in the ssl module (and, therefore, not in OpenSSL either):
I would like to do validation of server certificates in a project I'm working on, and I figured it would be better to be proactive and try to help create a patch than to just sit back and complain about it. It seems to me that this is a bug that you can't do peer certificate validation in httplib. If this isn't the place to ask these kinds of questions, I apologise. I can take the discussion elsewhere if I need to. Thanks, -Devin
If this isn't the place to ask these kinds of questions, I apologise. I can take the discussion elsewhere if I need to.
It really depends on what "these questions" are. If your question is "I have this patch, is it correct?", then the question is entirely appropriate. If it is "I just have barely looked at the API, can somebody please explain it all to me?", then this isn't appropriate for this list, and probably not appropriate elsewhere: anybody answering this question could just as well fix the original problem right away. So please do try to find the answer for yourself, with the (little) direction I gave. If you find that it takes a lot of effort, then you'll probably have to accept the bug as-is, and live with it. FWIW, I actually don't know the answer for sure, either, so I would have to research this myself, too. In any case, _ssl.c is *not* the place where any of the certificate validation actually happens - nor does it happen elsewhere in the Python source code, IIUC. Regards, Martin
Devin Cook <devin.c.cook@gmail.com> wrote:
I believe so, but you'll have to check the OpenSSL code.
Does it check that the host the socket is connected to is the same as what's given in the CN field in the certificate?
No. That, in general, doesn't work very well. The IETF working group on this is considering deprecating putting a hostname in the CN field at all, and just adding hostnames via the subjectAltName extension. The problem that's come up is that many computers don't have fixed IP addresses, and even with that the hostname is part of a different mapping of hostnames to IP addresses, which can also vary. I think that when the https: protocol scheme was written, it seemed like a good idea, but conventions on the Internet have changed a lot since then.
Where I'm going with this is I think all this checking needs to be part of certificate validation in the ssl module.
I don't think so. I put in hooks to let you do this in user code if you need to. See the archives for more discussion on this -- I'm not going to rehash it again. (This is really a question for OpenSSL mailing lists, or perhaps python-list.) Bill
Bill Janssen <janssen@parc.com> wrote:
Incidentally, the current working draft on this seems to be at <http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00>. Bill
Ok, thanks for all the feedback. Just for clarity, I'll summarize everything as I understand it: * OpenSSL does the all validation of the certificate itself. (http://openssl.org/docs/apps/verify.html) * httplib should have a way to enable validation of the certificate. * httplib should have a way to enable checking of the reference identity. (that complies with section 3 of this draft: http://tools.ietf.org/html/draft-saintandre-tls-server-id-check-00) * The reference identity checking (and cert validation, I assume) shouldn't be automatic. (per Bill) Does that sound about right? I'll try to work up a patch tonight implementing this. -Devin
participants (4)
-
"Martin v. Löwis" -
Bill Janssen -
Devin Cook -
Jesse Noller