[Baypiggies] urllib2 query

[Jason Culverhouse]

Vikram,
I'm sure you just want to download the files then process them later.
You can just use wget http://www.gnu.org/software/wget/wget.html 
it already has the support for timestamp checking etc.

wget --mirror ftp://ftp.ncbi.nih.gov/refseq/H_sapiens -o /my/local/mirror

You can just schedule that to run weekly?
Jason

On Nov 18, 2010, at 11:18 AM, Isaac wrote:

> Setup a caching proxy and acquire the last-modified header from the http proxy server. Then, here is a demonstration of a conditional GET request:
> 
> http://www.artima.com/forums/flat.jsp?forum=122&thread=15024
> 
> download the file from the proxy if the last-modified header is newer than yours on disk.
> 
> HTH
> 
> 
> On Thu, Nov 18, 2010 at 11:06 AM, Vikram K <kpguy1975 at gmail.com> wrote:
> consider the following ftp site:
> ftp://ftp.ncbi.nih.gov/refseq/H_sapiens/
> 
> I wish to write a program using urllib2 which can automatically download all the contents of one of the folders in this site (say, for example, mRNA_Prot) into a folder in my computer. Subsequently i want my program to be able to automatically update the downloaded folder in my computer in accordance with any changes made in future in the folder in the ftp site. Any suggestions on how i should proceed?
> I am using a windows 7 computer. Will i also have to use the scheduled tasks option in system tools for solving this problem?  
> 
> _______________________________________________
> Baypiggies mailing list
> Baypiggies at python.org
> To change your subscription options or unsubscribe:
> http://mail.python.org/mailman/listinfo/baypiggies
> 
> _______________________________________________
> Baypiggies mailing list
> Baypiggies at python.org
> To change your subscription options or unsubscribe:
> http://mail.python.org/mailman/listinfo/baypiggies

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/baypiggies/attachments/20101118/af2e90c3/attachment.html>