looking for wget-like module for getching software
behnel_ml at gkec.informatik.tu-darmstadt.de
Thu Dec 16 15:57:35 CET 2004
Robert P. J. Day schrieb:
> that is, i can just say, "go get file gcc-3.4.2.tar.bz2", and start
> searching at "ftp://pub.gnu.org/pub/gcc". i may not know how far down
> in the directory structure that file is, but wget will happily search
> recursively until it finds it.
That sounds pretty inefficient and should produce some load on the server
side. Imagine everyone doing that. Many FTP-servers provide files like
"ls-R.gz" somewhere, meaning, a list of files and directories. And then,
there's Google. Why do you have to use FTP? Ask Google for
"yourfile.tar.bz2 site:the.domain.name". It will very likely return a
suitable URL for you that you can use for download. Just tried it with
"site:ftp.gnu.org gcc-3.4.2.tar.bz2" - works. You can even use that link
and try to replace "http://" with "ftp://" for download. Should work for
many source repositories.
You can use urllib2 for both querying and download.
More information about the Python-list