On Sep 2, 2014, at 5:36 AM, holger krekel
wrote: On Mon, Sep 01, 2014 at 19:07 -0400, Donald Stufft wrote:
On Sep 1, 2014, at 4:53 PM, holger krekel
wrote: On Thu, Aug 28, 2014 at 14:58 -0400, Donald Stufft wrote:
Right now the “canonical” page for a particular project on PyPI is whatever the author happened to name their package (e.g. Django). This requires PyPI to have some "smarts" so that it can redirect things like /simple/django/ to /simple/Django/ otherwise someone doing ``pip install django`` would fall back to a much worse behavior.
If this redirect doesn't happen, then pip will issue a request for just /simple/ and look for a link that, when both sides are normalized, compares equal to the name it's looking for. It will then follow the link, get /simple/Django/ and everything works... Except it doesn't. The problem here comes from the external link classification that we have now. Pip sees the link to /simple/Django/ as an external link (because it lacks the required rels) and the installation finally fails.
The /simple/ case rarely happens when installing from PyPI itself because of the redirect, however it happens quite often when someone is attempting to instal from a mirror instead. Even when everything works correctly the penality for not knowing exactly what name to type in results in at least 1 extra http request, one of which (/simple/) requires pulling down a 2.1MB file.
To fix this I'm going to modify PyPI so that it uses the normalized name in the /simple/ URL and redirects everything else to the non-normalized name.
Of course you mean redirecting everything to the normalized name.
I'm also going to submit a PR to bandersnatch so that it will use normalized names ...
devpi-server also broke and I did a hotfix release today. Older installs will still have a problem, though (not all companies run the newest version all the time). Apart form the fact i was on vacation and on business travels, the notice for that breaking change was only one day which i think is a bit too quick. I'd really appreciate if you send a mail to Christian for bandersnatch and me for devpi before such changes happen and with a bit more reasonable ahead time.
Besides, i think it's a good change in principle.
best and thanks, holger
I can only really replete this with https://xkcd.com/1172/. This shouldn’t have been a breaking change, anyone following the HTTP spec dealt with this change just fine. As far as I can tell the only reason it broke devpi was because of an assertion in the code that was asserting against an implementation detail, an implementation detail that I changed.
Right, the assertion was there to ensure pypi's "realname" and devpi's internal "realname" of a project are the same. This check is now relaxed.
FWIW I'd prefer it we just said in all pypi APIs (http and xmlrpc/json) that a project name is always kept in canonical form, i.e. you can maybe register "HeLlo_World" but it just means "hello-world" next time someone asks for it. What is the relevance of the "realname" anyway? Do you keep "realnames" in warehouse?
As of right now we do, although I think it’s likely that Warehouse will end up with the normalized name being used as the “identifier” for a project and the name that an author typed in being used as the “display name”.
I’m sorry it broke devpi and that it happened at a time when you were on vacation, but honestly I don’t think it’s reasonable to expect every little thing to have to be run past a list of people. Due to the undocumented nature of these tools people have put a lot of (also undocumented) assumptions into their code, many of which are simply depending on implementation details. I try to test my changes against what I can, in this case pip, setuptools, and bandersnatch, but I can’t test against everything.
Thanks for all your work and eagerness to improve things. I think it's safe to assume that any change in PyPI's pip/bandersnatch/devpi facing http API has potential for disruption even if some http specification says otherwise -- at least until we have some specification of how tool/pypi interactions work.
best, holger
--- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA