[pyOpenSSL] pyopenssl and xmlrpclib

[Andrew Gaffney]

Andrew Gaffney wrote:
> Andrew Gaffney wrote:
>> Dan Williams wrote:
>>> On Thu, 2006-06-15 at 10:14 -0500, Andrew Gaffney wrote:
>>>> I'm trying to integrate xmlrpclib and pyopenssl. I'm mostly there, but I'm 
>>>> running into a problem:
>>> If you need some example code of doing SSL and all things XMLRPC, take a
>>> look at the files here:
>>>
>>> http://cvs.fedora.redhat.com/viewcvs/extras-buildsys/common/?root=fedora
>>>
>>> specifically SSLCommon.py and SSLConnection.py.  Furthermore,
>>> XMLRPCServerProxy.py, URLopener.py, FileTranser.py, FileDownloader.py,
>>> and FileUploader.py might be of of interest as well.
>>>
>>> These implementations use nonblocking sockets by default, which have a
>>> few other complications, but you can safely turn that off by passing
>>> timeout values of None, I think.
>>>
>>> We've been running variations on this code for almost a year now in the
>>> Fedora Extras build system, and it's been working pretty well.
>>>
>>> Dan
>> The RH implementation is close to what I had come up with, so I at least know I 
>> was on the right track. That does get me a little farther, however, there's a 
>> still a problem somewhere. I've modified my code to be as close to the RH code 
>> and still have my own touches.
>>
>> Currently, it looks like the request is sent, the headers are received back from 
>> the server, and then the connection closes (but in a weird way). The 'closed' 
>> var is getting set in the SSLConnection class, but stuff like select.select() 
>> and socket.revc() don't seem to notice...they return "Bad file descriptor". I 
>> added some code to SSLConnection's recv() to check self.closed and return None, 
>> which gets me past that error. At that point, I get an exception from the XML 
>> parsing code in xmlrpclib, apparently because no XML was ever sent back. The 
>> following is a transcript of the session with the client code:
> 
> <transcript snipped>
> 
> Okay, I've gotten a little further in tracking down the problem. It appears to 
> be an issue with the close_refcount thing in SSLConnection. It increments when 
> makefile() is called and decrements when close() is called. Apparently, it looks 
> like when HTTPConnection.getresponse() closes the connection after instantiating 
> the response class, the response class never called makefile(), so the 
> close_refcount is decremented to 0 and closed, even though the response class is 
> using it. Any ideas on how to get around this?

Okay, I finally have it "working". My above statement was actually wrong. It 
looks like HTTPResponse *does* use makefile(), but HTTPConnection does *not*. 
So, it calls close() which decrements the counter without ever calling 
makefile() which increments the counter. The way I made it "work" was by 
commenting all the calls to .close() and .shutdown(). This is technically 
"wrong", but it gets around the faulty close_refcount logic. Can anyone suggest 
a better way to do this?

-- 
Andrew Gaffney                            http://dev.gentoo.org/~agaffney/
Gentoo Linux Developer                                   Installer Project