urllib.urlretrieve never returns???

Laszlo Nagy gandalf at shopzeus.com
Tue Mar 20 21:12:12 CET 2012


2012.03.20. 8:08 keltezéssel, Laszlo Nagy írta:
> Here you can find the example program and the original post.
>
> http://code.activestate.com/lists/python-list/617894/
>>
>> I gather you are running urlretrieve in a separate thread, inside a GUI?
> Yes.
>>
>> I have learned that whenever I have inexplicable behaviour in a 
>> function,
>> I should check my assumptions. In this case, (1) are you sure you have
>> the right urlretrieve, and (2) are you sure that your self.Log() method
>> is working correctly? Just before the problematic call, do this:
>>
>> # was:
>> fpath = urllib.urlretrieve(imgurl)[0]
>>
>> # becomes:
>> print(urllib.__file__, urlretrieve)
>> self.Log(urllib.__file__, urlretrieve)
>> fpath = urllib.urlretrieve(imgurl)[0]
> I called self.Log() after each line, and also from a general "except:" 
> clause. Definitely, the line after urlretrieve is not executed, and no 
> exception is raised. Number of threads goes up (visible from task 
> manager).
>
> It is true that the program uses another module that uses the socket 
> module and multiple threads. (These are written in pure python.)
>
> If I remove the other module, then there is no error, however it 
> renders the application useless. If I start the program with a console 
> (e.g. with python.exe instead of pythonw.exe) then it works. Looks 
> like opening a console solves the problem, although nothing is ever 
> printed on the console.
>> and ensure that you haven't accidentally shadowed them with something
>> unexpected. Does the output printed to the console match the output
>> logged?
> Well, this cannot be tested. If there is a console, then there is no 
> problem.
>>
>> What happens if you take the call to urlretrieve out of the thread and
>> call it by hand?
> Then it works.
>> Run urllib.urlretrieve(imgurl) directly in the
>> interactive interpreter. Does it still hang forever?
> Then it works perfectly.
>>
>> When you say it "never" returns, do you mean *never* or do you mean "I
>> gave up waiting after five minutes"? What happens if you leave it to run
>> all day?
> I did not try that. But I have already set socket timeout to 10 
> seconds, and definitely it is not waiting for a response from the server.
>>
>> How big are the files you are trying to retrieve?
> 34 KB
>> Try retrieving a really small file. Then try retrieving a 
>> non-existent file.
> Good point. I'll try to retrieve a nonexistent file when I get home. :)

Today I got a different error message printed on console (program 
started with python.exe)



Unhandled exception in thread started by <bound method 
FrameLocEdit.GetThumbnail of <locedit.FrameLocEdit; proxy of <Swig 
Object of type 'wxPanel *' at 0x4f85300>
 >>Unhandled exception in thread started by <bound method 
FrameLocEdit.GetThumbnail of <locedit.FrameLocEdit; proxy of <Swig 
Object of type 'wxPanel *' at 0x4f85300>
 >>Unhandled exception in thread started by <bound method 
FrameLocEdit.GetThumbnail of <locedit.FrameLocEdit; proxy of <Swig 
Object of type 'wxPanel *' at 0x4f85300>
 >>Unhandled exception in thread started by
Traceback (most recent call last):

Traceback (most recent call last):

Traceback (most recent call last):
<bound method FrameLocEdit.GetThumbnail of <locedit.FrameLocEdit; proxy 
of <Swig Object of type 'wxPanel *' at 0x4f85300>
 >>  File "C:\Python\Projects\Warehouserclient_v3\locedit.py", line 917, 
in GetThumbnail
   File "C:\Python\Projects\Warehouserclient_v3\locedit.py", line 917, 
in GetThumbnail
   File "C:\Python\Projects\Warehouserclient_v3\locedit.py", line 917, 
in GetThumbnail

sys.excepthook is missing
Traceback (most recent call last):

I have never seen a traceback like this before. I didn't install any 
excepthook myself.

Program is using wxPython, socket, threads, threading, PIL.

Here is something even more strange. If I click on the button three 
times, then absolutely nothing gets printed on stdout. However, if I 
close the program with file/exit (actually, calling 
wx.PySimpleApp.ExitMainLoop) then suddenly three stack traces are 
printed on stdout, all lines mixed up:

Unhandled exception in thread started by <bound method 
FrameLocEdit.GetThumbnail of <locedit.FrameLocEdit; proxy of <Swig 
Object of type 'wxPanel *' at 0x4fb530
0>
 >>Unhandled exception in thread started by Unhandled exception in 
thread started by <bound method FrameLocEdit.GetThumbnail of 
<locedit.FrameLocEdit; proxy of <Swig Object of type 'wxPanel *' at 
0x4fb5300>
 >>Unhandled exception in thread started by Unhandled exception in 
thread started by <bound method FrameLocEdit.GetThumbnail of 
<locedit.FrameLocEdit; proxy of <Swig Object of type 'wxPanel *' at 
0x4fb5300>
 >>Traceback (most recent call last):<bound method 
FrameLocEdit.GetThumbnail of <locedit.FrameLocEdit; proxy of <Swig 
Object of type 'wxPanel *' at 0x4fb5300>
 >>Traceback (most recent call last):<bound method 
FrameLocEdit.GetThumbnail of <locedit.FrameLocEdit; proxy of <Swig 
Object of type 'wxPanel *' at 0x4fb5300>
 >>Traceback (most recent call last):  File 
"C:\Python\Projects\Warehouserclient_v3\locedit.py", line 917, in 
GetThumbnail


Traceback (most recent call last):

   File "C:\Python\Projects\Warehouserclient_v3\locedit.py", line 917, 
in GetThumbnail
Traceback (most recent call last):
   File "C:\Python\Projects\Warehouserclient_v3\locedit.py", line 917, 
in GetThumbnail
       File "C:\Python\Projects\Warehouserclient_v3\locedit.py", line 
917, in GetThumbnail
       File "C:\Python\Projects\Warehouserclient_v3\locedit.py", line 
917, in GetThumbnail
     self.Log(traceback.format_exc())
     self.Log(traceback.format_exc())
     self.Log(traceback.format_exc())
AttributeErrorself.Log(traceback.format_exc())
AttributeErrorself.Log(traceback.format_exc())
AttributeError: AttributeError: AttributeError:
C:\Python\Projects\Warehouserclient_v3>


And the last attributerror doesn't tell what attribute is missing. 
Probably I'll have to use a different library (pycurl, for example). But 
the error itself is getting more interesting.

Also tried to use an invalid file (that should return with HTTP 404 not 
found) but the effect is exactly the same. Nothing is printed on stdout 
until I try to close the program (stop wxPython's mainloop). Then all 
previously threads throw an AttributeError and all of them print a stack 
trace (at the same time, lines mixed up).

I'll be experimenting with pyCurl now.

Thanks,

    Laszlo








More information about the Python-list mailing list