urllib2 performance on windows, usb connection

[dq]

Martin v. Löwis wrote:
>> So does anyone know what the deal is with this?  Why is the same code so
>> much slower on Windows?  Hope someone can tell me before a holy war
>> erupts :-)
> 
> Only the holy war can give an answer here. It certainly has *nothing* to
> do with Python; Python calls the operating system functions to read from
> the network and write to the disk almost directly. So it must be the
> operating system itself that slows it down.
> 
> To investigate further, you might drop the write operating, and measure
> only source.read(). If that is slower, then, for some reason, the
> network speed is bad on Windows. Maybe you have the network interfaces
> misconfigured? Maybe you are using wireless on Windows, but cable on
> Linux? Maybe you have some network filtering software running on
> Windows? Maybe it's just that Windows sucks?-)
> 
> If the network read speed is fine, but writing slows down, I ask the
> same questions. Perhaps you have some virus scanner installed that
> filters all write operations? Maybe Windows sucks?
> 
> Regards,
> Martin
> 

Thanks for the ideas, Martin.  I ran a couple of experiments to find the 
culprit, by downloading the same 20 MB file from the same fast server. 
I compared:

1.  DL to HD vs USB iPod.
2.  AV on-access protection on vs. off
3.  "source. read()" only vs.  "file.write( source.read() )"

The culprit is definitely the write speed on the iPod.  That is, 
everything runs plenty fast (~1 MB/s down) as long as I'm not writing 
directly to the iPod.  This is kind of odd, because if I copy the file 
over from the HD to the iPod using windows (drag-n-drop), it takes about 
a second or two, so about 10 MB/s.

So the problem is definitely partially Windows, but it also seems that 
Python's file.write() function is not without blame.  It's the 
combination of Windows, iPod and Python's data stream that is slowing me 
down.

I'm not really sure what I can do about this.  I'll experiment a little 
more and see if there's any way around this bottleneck.  If anyone has 
run into a problem like this, I'd love to hear about it...

thanks again,
--danny