urllib2.urlopen taking way too much time

Phonethics Mobile Media phonethics at gmail.com
Mon Apr 19 08:44:24 EDT 2010


handler = urllib2.urlopen(req) is taking way too much time to retrieve
the URL. The same code using sockets in PHP doesn't delay this long.
I had 'Authorization':'Basic ' + base64.b64encode("username:password")
in my header though.
[ I didnt use HTTPPasswordMgr & HTTPPasswordMgrWithDefaultRealm
because I was unable to send headers with the example shown here :
http://docs.python.org/howto/urllib2.html#id6 ]

I have data = handler.readline() in a loop - this is an endless
network script (twitter streaming) so cant really wait for the entire
url contents to finish loading.

I had to socket.setdefaulttimeout to 30 to make it work.
Is it because python is getting chunks in high number of KBs ?

Python 2.6.2

--
Phonethics



More information about the Python-list mailing list