urllib slow on FreeBSD 4.7?

dsavitsk dsavitsk at e-coli.net
Tue Nov 19 22:37:18 EST 2002


"Brian Quinlan" <brian at sweetapp.com> wrote in message
news:mailman.1037753912.31805.python-list at python.org...
> dsavitsk wrote:
> > I think the best response is to say that as wget was not installed, I
> went
> > to /usr/ports/ftp/wget
> >
> > typed "make install" and the necessary files (a few MB worth) were
> > retrieved from gnu.org in about 20 seconds.  much faster than the
> files I
> > am geting w/ urllib.
>
> That doesn't reveal anything interesting. For example, the gnu.org site
> might be a lot faster than the one that you were testing, you might have
> been experiencing transient network problems when testing urllib, etc.
>
as I say, the site i am testing is on a computer not 8 inches from the
FreeBSD one (attached to the same kvm no less).  The files must traverse
nearly 6 feet of cable, 3 other computers were able to download much more
quickly, and the freebsd one has failed all day (even switching network
cable and spots in the switch).

further, for a 2 meg file on

Python 2.2.1 (#1, Oct  5 2002, 11:19:44)
[GCC 2.95.4 20020320 [FreeBSD]] on freebsd4
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system('/usr/local/bin/wget
"http://192.168.0.4/index.asp?file=name" -O file.ext')

takes less than 2 seconds

while
>>> f = open('file2.ext', 'w')
>>> u = urllib.urlopen("http://192.168.0.4/index.asp?file=name")
>>> f.write(u.read())
>>> f.close()

takes about 2 minutes

so, it seems that using wget is the proper way to preceed, but i would
rather a python solution.

-doug







More information about the Python-list mailing list