[Twisted-Python] are there known ssl/threading/socket issues with httplib2/httplib and Twisted ?
Background: I'm using a 3rd party API client, which uses httplib2 in the internals. ( the google-plus api ) I'm placing this blocking code within a DeferredList, that runs within a twisted.enterprise.adbapi.ConnectionPool.runInteraction In my initial tests, everything worked fine -- but I was only making a single GET/POST, as I kept the size of the DeferredList to 1 while I worked on the code. As soon as I increased the limit and had 2 GET/POST events firing off at the same time, I started getting a ssl/socket error ( httplib2 > httplib > socket > ssl ). It works fine in twisted if I make simultaneous requests otherwise (using twisted or the `requests` package). this seems to be tied to how httplib2/httplib is running within twisted. Has anyone experienced this before? if so, do you know if there is a workaround? The easiest fix, is probably for me to write a new ApiClient tomorrow for the methods I need. I'd just like to avoid that if possible
Jonathan Vanasco wrote:
Background: I'm using a 3rd party API client, which uses httplib2 in the internals. ( the google-plus api ) I'm placing this blocking code within a DeferredList, that runs within a twisted.enterprise.adbapi.ConnectionPool.runInteraction
In my initial tests, everything worked fine -- but I was only making a single GET/POST, as I kept the size of the DeferredList to 1 while I worked on the code.
As soon as I increased the limit and had 2 GET/POST events firing off at the same time, I started getting a ssl/socket error ( httplib2 > httplib > socket > ssl ).
What's the error?
It works fine in twisted if I make simultaneous requests otherwise (using twisted or the `requests` package). this seems to be tied to how httplib2/httplib is running within twisted.
There shouldn't be anything special about httplib2/httplib “within twisted”. Unless httplib2 is mucking with global socket defaults or something (and I doubt it does) I wouldn't expect any interaction between the two at all. It's no different to using httplib2 in a thread without Twisted. -Andrew.
Has anyone experienced this before? if so, do you know if there is a workaround?
The easiest fix, is probably for me to write a new ApiClient tomorrow for the methods I need. I'd just like to avoid that if possible _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Are you by any chance using the same HTTPConnection instance in both threads? Dustin On Wed, Sep 17, 2014 at 9:44 PM, Andrew Bennetts <andrew@bemusement.org> wrote:
Jonathan Vanasco wrote:
Background: I'm using a 3rd party API client, which uses httplib2 in the internals. ( the google-plus api ) I'm placing this blocking code within a DeferredList, that runs within a twisted.enterprise.adbapi.ConnectionPool.runInteraction
In my initial tests, everything worked fine -- but I was only making a single GET/POST, as I kept the size of the DeferredList to 1 while I worked on the code.
As soon as I increased the limit and had 2 GET/POST events firing off at the same time, I started getting a ssl/socket error ( httplib2 > httplib > socket > ssl ).
What's the error?
It works fine in twisted if I make simultaneous requests otherwise (using twisted or the `requests` package). this seems to be tied to how httplib2/httplib is running within twisted.
There shouldn't be anything special about httplib2/httplib “within twisted”. Unless httplib2 is mucking with global socket defaults or something (and I doubt it does) I wouldn't expect any interaction between the two at all. It's no different to using httplib2 in a thread without Twisted.
-Andrew.
Has anyone experienced this before? if so, do you know if there is a workaround?
The easiest fix, is probably for me to write a new ApiClient tomorrow for the methods I need. I'd just like to avoid that if possible _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Sep 18, 2014, at 8:35 AM, Dustin J. Mitchell wrote:
Are you by any chance using the same HTTPConnection instance in both threads?
Dustin
Ack! It looks like the snippet of sample code that I used as a constructor did that. I did not see that. This api library and docs are... frustrating. i've been trying to keep everything tied to the sample code to avoid problems. Everything is working perfect now. Thanks. ( the error was below, sorry for not posting that last night ). including here for search engines in case someone else uses this. ---- File "build/bdist.macosx-10.6-intel/egg/apiclient/http.py", line 716, in execute File "/Users/jvanasco/webserver/environments/stddev-2.7.5/lib/python2.7/site-packages/httplib2/__init__.py", line 1570, in request (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) File "/Users/jvanasco/webserver/environments/stddev-2.7.5/lib/python2.7/site-packages/httplib2/__init__.py", line 1317, in _request (response, content) = self._conn_request(conn, request_uri, method, body, headers) File "/Users/jvanasco/webserver/environments/stddev-2.7.5/lib/python2.7/site-packages/httplib2/__init__.py", line 1300, in _conn_request content = response.read() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 548, in read s = self.fp.read() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1311, in read return s + self._file.read() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 351, in read data = self._sock.recv(rbufsize) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 241, in recv return self.read(buflen) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 160, in read return self._sslobj.read(len) ssl.SSLError: [Errno 1] _ssl.c:1363: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
participants (3)
-
Andrew Bennetts -
Dustin J. Mitchell -
Jonathan Vanasco