looking for wget-like module for getching software
Robert P. J. Day
rpjday at mindspring.com
Thu Dec 16 17:43:56 CET 2004
On Thu, 16 Dec 2004, Stefan Behnel wrote:
> Robert P. J. Day schrieb:
> > that is, i can just say, "go get file gcc-3.4.2.tar.bz2", and start
> > searching at "ftp://pub.gnu.org/pub/gcc". i may not know how far down
> > in the directory structure that file is, but wget will happily search
> > recursively until it finds it.
> That sounds pretty inefficient and should produce some load on the
> server side. Imagine everyone doing that. Many FTP-servers provide
> files like "ls-R.gz" somewhere, meaning, a list of files and
i realize that and, as much as possible, i try to start the process
with what i *know* is the actual directory location. but
occasionally, if i can't guarantee an exact directory, i just want the
option to start a bit higher up the directory structure. i wasn't
going to make a habit of it, it's more an absolute last resort, if it
comes down to it.
> And then, there's Google. Why do you have to use FTP? Ask Google for
> "yourfile.tar.bz2 site:the.domain.name". It will very likely return
> a suitable URL for you that you can use for download. Just tried it
> with "site:ftp.gnu.org gcc-3.4.2.tar.bz2" - works. You can even use
> that link and try to replace "http://" with "ftp://" for download.
> Should work for many source repositories.
> You can use urllib2 for both querying and download.
and that pretty much answers the question i had. much thanks. and
now that i have my answer, i might have to unsubscribe. you guys sure
are a chatty bunch. :)
p.s. just kidding.
More information about the Python-list