On Sep 2, 2014, at 5:36 AM, holger krekel <holger@merlinux.eu> wrote:

On Mon, Sep 01, 2014 at 19:07 -0400, Donald Stufft wrote:

On Sep 1, 2014, at 4:53 PM, holger krekel <holger@merlinux.eu> wrote:

On Thu, Aug 28, 2014 at 14:58 -0400, Donald Stufft wrote:

Right now the “canonical” page for a particular project on PyPI is whatever the
author happened to name their package (e.g. Django). This requires PyPI to have
some "smarts" so that it can redirect things like /simple/django/ to
/simple/Django/ otherwise someone doing ``pip install django`` would fall back
to a much worse behavior.

If this redirect doesn't happen, then pip will issue a request for just
/simple/ and look for a link that, when both sides are normalized, compares
equal to the name it's looking for. It will then follow the link, get
/simple/Django/ and everything works... Except it doesn't. The problem here
comes from the external link classification that we have now. Pip sees the
link to /simple/Django/ as an external link (because it lacks the required
rels) and the installation finally fails.

The /simple/ case rarely happens when installing from PyPI itself because of
the redirect, however it happens quite often when someone is attempting to
instal from a mirror instead. Even when everything works correctly the penality
for not knowing exactly what name to type in results in at least 1 extra http
request, one of which (/simple/) requires pulling down a 2.1MB file.

To fix this I'm going to modify PyPI so that it uses the normalized name in
the /simple/ URL and redirects everything else to the non-normalized name.

Of course you mean redirecting everything to the normalized name.

I'm also going to submit a PR to bandersnatch so that it will use
normalized names ...

devpi-server also broke and I did a hotfix release today.  Older
installs will still have a problem, though (not all companies run the
newest version all the time).  Apart form the fact i was on vacation and
on business travels, the notice for that breaking change was only one
day which i think is a bit too quick.  I'd really appreciate if you send
a mail to Christian for bandersnatch and me for devpi before such
changes happen and with a bit more reasonable ahead time.

Besides, i think it's a good change in principle.

best and thanks,
holger

I can only really replete this with https://xkcd.com/1172/.
This shouldn’t have been a breaking change, anyone following the HTTP
spec dealt with this change just fine. As far as I can tell the only reason
it broke devpi was because of an assertion in the code that was asserting
against an implementation detail, an implementation detail that I changed.

Right, the assertion was there to ensure pypi's "realname" and devpi's
internal "realname" of a project are the same.  This check is now relaxed.

FWIW I'd prefer it we just said in all pypi APIs (http and xmlrpc/json)
that a project name is always kept in canonical form, i.e. you can maybe
register "HeLlo_World" but it just means "hello-world" next time someone
asks for it.  What is the relevance of the "realname" anyway?
Do you keep "realnames" in warehouse? 

I’m sorry it broke devpi and that it happened at a time when you were
on vacation, but honestly I don’t think it’s reasonable to expect every
little thing to have to be run past a list of people. Due to the undocumented
nature of these tools people have put a lot of (also undocumented)
assumptions into their code, many of which are simply depending on
implementation details. I try to test my changes against what I can,
in this case pip, setuptools, and bandersnatch, but I can’t test against
everything.

Thanks for all your work and eagerness to improve things.  I think
it's safe to assume that any change in PyPI's pip/bandersnatch/devpi
facing http API has potential for disruption even if some http
specification says otherwise -- at least until we have some specification
of how tool/pypi interactions work.

best,
holger