mirroring files and data via http

[Matt Nordhoff]

Steve Potter wrote:
> On Jul 6, 8:19 pm, Matt Nordhoff <mnordh... at mattnordhoff.com> wrote:
>> Steve Potter wrote:
>>> I'm working on a project to create a central administration interface
>>> for several websites located on different physical servers.
>>> You can think of the websites as a blog type application. My
>>> administration application will be used to create new blog posts with
>>> associated media (images, etc..)
>>> So I am thinking to setting up a script on each of the sites to "phone
>>> home" once per day and download any new posts and media files.
>>> I am thinking of transmitting the post data in an xml format that
>>> could then be decoded an recorded into the database on the slave
>>> site.   Are there any easy ways to encode a python dictionary to and
>>> from xml?
>>> For the media files I am thinking that the administration interface
>>> would also provide an xml file with a list of all of the media files
>>> required along with an http path to retrieve them and a checksum of
>>> some sort to verify that they were downloaded correctly.
>>> Basically I am trying to figure out if anything already exists to
>>> perform some of these functions or if I have to make them from
>>> scratch.  I know none of it should be too hard, but I hate to re-
>>> invent the wheel.
>>> Thanks,
>>> Steven Potter
>> It sounds like JSON would be perfect for this. For working with it in
>> Python, the simplejson module is popular:
>>
>> <http://pypi.python.org/pypi/simplejson>
>> --
> 
> Matt,
> 
> You are correct JSON would be much easier to deal with than xml.  That
> takes care of several of the problems.
> 
> Now it really just comes down to the file transfer from one server to
> the other and a verification of some sort that the file transfer was
> not corrupt.  Is there a python module that takes care of file
> downloads and verification?
> 
> Thanks,
> 
> Steve

Not automatically, but this should be easy to do yourself. You could
generate a simple JSON or plain text file listing URLs and hashes, and
then use urllib or urllib2 to download them, and md5, sha1 or (in Python
2.5) hashlib to verify them. All of those modules are in the standard
library.
--