[Distutils] Handling Case/Normalization Differences

[Donald Stufft]

> On Sep 1, 2014, at 4:53 PM, holger krekel <holger at merlinux.eu> wrote:
> 
> On Thu, Aug 28, 2014 at 14:58 -0400, Donald Stufft wrote:
>> Right now the “canonical” page for a particular project on PyPI is whatever the
>> author happened to name their package (e.g. Django). This requires PyPI to have
>> some "smarts" so that it can redirect things like /simple/django/ to
>> /simple/Django/ otherwise someone doing ``pip install django`` would fall back
>> to a much worse behavior.
>> 
>> If this redirect doesn't happen, then pip will issue a request for just
>> /simple/ and look for a link that, when both sides are normalized, compares
>> equal to the name it's looking for. It will then follow the link, get
>> /simple/Django/ and everything works... Except it doesn't. The problem here
>> comes from the external link classification that we have now. Pip sees the
>> link to /simple/Django/ as an external link (because it lacks the required
>> rels) and the installation finally fails.
>> 
>> The /simple/ case rarely happens when installing from PyPI itself because of
>> the redirect, however it happens quite often when someone is attempting to
>> instal from a mirror instead. Even when everything works correctly the penality
>> for not knowing exactly what name to type in results in at least 1 extra http
>> request, one of which (/simple/) requires pulling down a 2.1MB file.
>> 
>> To fix this I'm going to modify PyPI so that it uses the normalized name in
>> the /simple/ URL and redirects everything else to the non-normalized name.
> 
> Of course you mean redirecting everything to the normalized name.
> 
>> I'm also going to submit a PR to bandersnatch so that it will use
>> normalized names ...
> 
> devpi-server also broke and I did a hotfix release today.  Older
> installs will still have a problem, though (not all companies run the
> newest version all the time).  Apart form the fact i was on vacation and
> on business travels, the notice for that breaking change was only one
> day which i think is a bit too quick.  I'd really appreciate if you send
> a mail to Christian for bandersnatch and me for devpi before such
> changes happen and with a bit more reasonable ahead time.
> 
> Besides, i think it's a good change in principle.
> 
> best and thanks,
> holger

I can only really replete this with https://xkcd.com/1172/.

This shouldn’t have been a breaking change, anyone following the HTTP
spec dealt with this change just fine. As far as I can tell the only reason
it broke devpi was because of an assertion in the code that was asserting
against an implementation detail, an implementation detail that I changed.

I’m sorry it broke devpi and that it happened at a time when you were
on vacation, but honestly I don’t think it’s reasonable to expect every
little thing to have to be run past a list of people. Due to the undocumented
nature of these tools people have put a lot of (also undocumented)
assumptions into their code, many of which are simply depending on
implementation details. I try to test my changes against what I can,
in this case pip, setuptools, and bandersnatch, but I can’t test against
everything.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140901/ff68b3bc/attachment-0001.html>