[Catalog-sig] PyPI down again...

[Justin Ryan]

On Fri, Jun 11, 2010 at 1:56 PM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
...
> Help is certainly appreciated. The type of help depends on the volunteer, of
> course. E.g. I wouldn't want to give root accounts to
> the first person that comes along and asks for them (except when the first
> person is Jannis Leidel, who (I believe) did the Apache restart
> today).
>

Thanks, Jannis. :)

>> What's up with this years-old PEP for expanding the PyPI
>> infrastructure?  Are there resources, relationships, volunteers
>> lacking?
>>
>> What can we do to help? :)
>
> If you are willing to invest *a lot* of time, then it seems that rewriting
> PyPI in Django would make a lot of people happy, because
> they claim they can't contribute to the current code base because
> they don't understand that. I don't want to do such a rewrite on
> my own because I *do* understand the code base (despite not having written
> it in the first place, so I think that if you really want
> to contribute, you can learn how it works); it also violates Joel
> Spolsky's principle of never ever doing rewrites.

I'll avoid deep comments about my general feelings about Django here. ;)

What is it now, just a straight WSGI app?

> It will be a lot of work because it must implement full compatibility with
> the current code, which I can promise will keep you busy. Full
> compatibility is primarily defined in terms of URLs that people may
> have put on the web and into Google, and URLs and API that setuptools
> back to very old releases may use.

Sure..

> That said, I have no idea what is causing the current outages. There must be
> some secret ping of death or something that somebody discovered.

If you want to give me a shell that can just access ps and top for
now, read-only access to log files, I can try and put some time into
keeping an eye.

> For a smaller project, start putting mirror support into setuptools or
> distribute; this would make short (several hours) outages less severe for
> the class of users that want permanent availability for downloading.
> It's unlikely that the mirrors would break when the master goes down;
> they just stop mirroring.

That's a really great idea.  I try to use egg caches in buildout and
the -N option to not look for the newest of everything all the time,
but I think it needs a bit of work as well.  We also have support for
alternate download targets in buildout, but it seems the failure mode
when PyPI is down is weak.

So, there's definitely two sides to this, we all need to be gentler
and calmer users of PyPI, and we all need it to work more.  And anyone
putting time into restarting Apache probably wants to stop doing that.
:)

Anyone interested in helping to add mirror support to distribute?  I
suspect it is distribute / setuptools which are tied to the poor
failure mode I'm encountering with zc.buildout.

Best!

J