[Catalog-sig] Proposal: Move PyPI static data to the cloud for better availability

[Jesus Cea]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 15/06/10 13:49, M.-A. Lemburg wrote:
> Server side: upload cronjobs
> ----------------------------
> 
> Since the /simple index tree is currently being created dynamically,
> we'd need to create static copies of it at regular intervals in order
> to upload the content to the S3 bucket. This can easily be done using
> tools such as wget or curl.
> 
> Both the static copy of the /simple tree and the static files uploaded
> to /packages then need to be uploaded or updated in the S3 bucket by a
> cronjob running every 10-20 minutes.

I don't comment about the convenience to migrate or not.

But having to wait 20 minutes to deploy my just released package to my
datacenter is a bit inconvenient to me :-).

Would be nice to change PYPI code just to dump "simple" each time the
database changes. Perusing the RSS, the load should be low and actually
less demanding to CPU and database server (if you only update "simple"
with the changes, not rebuilding everything each time).

- -- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBTBe2zplgi5GaxT1NAQLZIAP+JHe5dAVN27FTMD+gMzKntFEbEA3t9gqh
gblEFPc5bigEAvfXxJTm2p+A0meeH7dVNT2akyYU4Cn+DmdV9+LkXY1c+beV7bpY
BD2ROBvmFJ05FXPPkFD/La4Z0Bqb9JuZy7PV2kTQagzMsn3VjLJRDWt5K0kpIwcw
Fntro0K/dRs=
=G2bd
-----END PGP SIGNATURE-----