<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Aug 28, 2014, at 6:09 PM, Donald Stufft <<a href="mailto:donald@stufft.io" class="">donald@stufft.io</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Aug 28, 2014, at 2:58 PM, Donald Stufft <<a href="mailto:donald@stufft.io" class="">donald@stufft.io</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><div class="">Right now the “canonical” page for a particular project on PyPI is whatever the</div><div class="">author happened to name their package (e.g. Django). This requires PyPI to have</div><div class="">some "smarts" so that it can redirect things like /simple/django/ to</div><div class="">/simple/Django/ otherwise someone doing ``pip install django`` would fall back</div><div class="">to a much worse behavior.</div><div class=""><br class=""></div><div class="">If this redirect doesn't happen, then pip will issue a request for just</div><div class="">/simple/ and look for a link that, when both sides are normalized, compares</div><div class="">equal to the name it's looking for. It will then follow the link, get</div><div class="">/simple/Django/ and everything works... Except it doesn't. The problem here</div><div class="">comes from the external link classification that we have now. Pip sees the</div><div class="">link to /simple/Django/ as an external link (because it lacks the required</div><div class="">rels) and the installation finally fails.</div><div class=""><br class=""></div><div class="">The /simple/ case rarely happens when installing from PyPI itself because of</div><div class="">the redirect, however it happens quite often when someone is attempting to</div><div class="">instal from a mirror instead. Even when everything works correctly the penality</div><div class="">for not knowing exactly what name to type in results in at least 1 extra http</div><div class="">request, one of which (/simple/) requires pulling down a 2.1MB file.</div><div class=""><br class=""></div><div class="">To fix this I'm going to modify PyPI so that it uses the normalized name in</div><div class="">the /simple/ URL and redirects everything else to the non-normalized name. I'm</div><div class="">also going to submit a PR to bandersnatch so that it will use normalized names</div><div class="">for it's directories and such as well. These two changes will make it so that</div><div class="">the client side will know ahead of time exactly what form the server expects</div><div class="">any given name to be in. This will allow a change in pip to happen which</div><div class="">will pre-normalize all names which will make the interaction with mirrors better</div><div class="">and will reduce the number of HTTP requests that a single ``pip install`` needs</div><div class="">to make.</div></div><br class=""><div class="">
<div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">---</div><div class="">Donald Stufft</div><div class="">PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA</div></div></div>
</div>
<br class=""></div>_______________________________________________<br class="">Distutils-SIG maillist - <a href="mailto:Distutils-SIG@python.org" class="">Distutils-SIG@python.org</a><br class=""><a href="https://mail.python.org/mailman/listinfo/distutils-sig" class="">https://mail.python.org/mailman/listinfo/distutils-sig</a><br class=""></div></blockquote></div><div class=""><br class=""></div><div class=""><div class="">Hm, so here’s the problem.</div><div class=""><br class=""></div><div class="">I have this implemented and deployed to TestPyPI, it works great!</div><div class=""><br class=""></div><div class="">However, the next step is to make the change to bandersnatch so that it saves</div><div class="">things using their normalized name instead of using their "proper" name. Doing</div><div class="">this will trigger it so that everyone using pip 1.5 won't be able to install</div><div class="">anything from that mirror unless it's name is specified as the normalized name</div><div class="">(e.g. ``pip install Django`` will fail without --allow-unverified but</div><div class="">``pip install django`` will work). This would be fixed with pip 1.6 (since</div><div class="">it would know to "normalize" the name before fetching the URL).</div><div class=""><br class=""></div><div class="">The same thing will occur if we make the change in pip first, it would</div><div class="">normalize names so you'd need to use --allow-unverified for everything because</div><div class="">it would act as if you typed ``pip install django`` instead of ``pip install</div><div class="">Django``.</div><div class=""><br class=""></div><div class="">To my knowledge, this *only* will affect pip 1.5.x.</div><div class=""><br class=""></div><div class="">So the only way forward I can see to make this change, which I think is a good</div><div class="">change and will remove a big "gotcha" from using a mirror, is to coordinate</div><div class="">a release of bandersnatch that coincides with pip 1.6, and tell people they</div><div class="">need to upgrade in lockstep.</div><div class=""><br class=""></div><div class="">Does anyone have any other ideas?</div></div><br class=""><div class="">
<div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">---</div><div class="">Donald Stufft</div><div class="">PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA</div></div></div>
</div>
<br class=""></div>_______________________________________________<br class="">Distutils-SIG maillist - <a href="mailto:Distutils-SIG@python.org" class="">Distutils-SIG@python.org</a><br class=""><a href="https://mail.python.org/mailman/listinfo/distutils-sig" class="">https://mail.python.org/mailman/listinfo/distutils-sig</a><br class=""></div></blockquote></div><div class=""><br class=""></div><div class=""><div class="">Just thought of this, if the normalized name doesn’t match the "real" name,</div><div class="">then add entries for both. This will make it so that pip 1.5 continues to work</div><div class="">and pip 1.6+.</div></div><br class=""><div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">---</div><div class="">Donald Stufft</div><div class="">PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA</div></div></div>
</div>
<br class=""></body></html>