<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Since pip 1.4 it does yes, however the problem here is that typically bandersnatch</div><div class="">mirrors are simply hosted by plain static web servers and don’t require any sort of runtime logic.</div><br class=""><div><blockquote type="cite" class=""><div class="">On Aug 28, 2014, at 6:39 PM, Joe Smith <<a href="mailto:yasumoto7@gmail.com" class="">yasumoto7@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Naive question- does pip send over a UserAgent (or something) that contains a version number the server can use to determine which behavior to default to?<div class=""><br class=""></div><div class="">That would allow a deprecation cycle of N months or so that will let people upgrade from 1.5 to 1.6. We could then watch usage of 1.5 decrease over time until it's a non-factor.</div>
</div><div class="gmail_extra"><br class=""><br class=""><div class="gmail_quote">On Thu, Aug 28, 2014 at 3:26 PM, Donald Stufft <span dir="ltr" class=""><<a href="mailto:donald@stufft.io" target="_blank" class="">donald@stufft.io</a>></span> wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="h5"><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Aug 28, 2014, at 6:09 PM, Donald Stufft <<a href="mailto:donald@stufft.io" target="_blank" class="">donald@stufft.io</a>> wrote:</div>
<br class=""><div class=""><div style="word-wrap:break-word" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Aug 28, 2014, at 2:58 PM, Donald Stufft <<a href="mailto:donald@stufft.io" target="_blank" class="">donald@stufft.io</a>> wrote:</div><br class=""><div class="">
<div style="word-wrap:break-word" class=""><div class=""><div class="">Right now the “canonical” page for a particular project on PyPI is whatever the</div><div class="">author happened to name their package (e.g. Django). This requires PyPI to have</div><div class="">
some "smarts" so that it can redirect things like /simple/django/ to</div><div class="">/simple/Django/ otherwise someone doing ``pip install django`` would fall back</div><div class="">to a much worse behavior.</div><div class=""><br class=""></div>
<div class="">If this redirect doesn't happen, then pip will issue a request for just</div><div class="">/simple/ and look for a link that, when both sides are normalized, compares</div><div class="">equal to the name it's looking for. It will then follow the link, get</div>
<div class="">/simple/Django/ and everything works... Except it doesn't. The problem here</div><div class="">comes from the external link classification that we have now. Pip sees the</div><div class="">link to /simple/Django/ as an external link (because it lacks the required</div>
<div class="">rels) and the installation finally fails.</div><div class=""><br class=""></div><div class="">The /simple/ case rarely happens when installing from PyPI itself because of</div><div class="">the redirect, however it happens quite often when someone is attempting to</div>
<div class="">instal from a mirror instead. Even when everything works correctly the penality</div><div class="">for not knowing exactly what name to type in results in at least 1 extra http</div><div class="">request, one of which (/simple/) requires pulling down a 2.1MB file.</div>
<div class=""><br class=""></div><div class="">To fix this I'm going to modify PyPI so that it uses the normalized name in</div><div class="">the /simple/ URL and redirects everything else to the non-normalized name. I'm</div><div class="">also going to submit a PR to bandersnatch so that it will use normalized names</div>
<div class="">for it's directories and such as well. These two changes will make it so that</div><div class="">the client side will know ahead of time exactly what form the server expects</div><div class="">any given name to be in. This will allow a change in pip to happen which</div>
<div class="">will pre-normalize all names which will make the interaction with mirrors better</div><div class="">and will reduce the number of HTTP requests that a single ``pip install`` needs</div><div class="">to make.</div></div><br class=""><div class="">
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" class=""><div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" class="">
<div class="">---</div><div class="">Donald Stufft</div><div class="">PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA</div></div></div>
</div>
<br class=""></div>_______________________________________________<br class="">Distutils-SIG maillist - <a href="mailto:Distutils-SIG@python.org" target="_blank" class="">Distutils-SIG@python.org</a><br class=""><a href="https://mail.python.org/mailman/listinfo/distutils-sig" target="_blank" class="">https://mail.python.org/mailman/listinfo/distutils-sig</a><br class="">
</div></blockquote></div><div class=""><br class=""></div><div class=""><div class="">Hm, so here’s the problem.</div><div class=""><br class=""></div><div class="">I have this implemented and deployed to TestPyPI, it works great!</div><div class=""><br class=""></div><div class="">However, the next step is to make the change to bandersnatch so that it saves</div>
<div class="">things using their normalized name instead of using their "proper" name. Doing</div><div class="">this will trigger it so that everyone using pip 1.5 won't be able to install</div><div class="">anything from that mirror unless it's name is specified as the normalized name</div>
<div class="">(e.g. ``pip install Django`` will fail without --allow-unverified but</div><div class="">``pip install django`` will work). This would be fixed with pip 1.6 (since</div><div class="">it would know to "normalize" the name before fetching the URL).</div>
<div class=""><br class=""></div><div class="">The same thing will occur if we make the change in pip first, it would</div><div class="">normalize names so you'd need to use --allow-unverified for everything because</div><div class="">it would act as if you typed ``pip install django`` instead of ``pip install</div>
<div class="">Django``.</div><div class=""><br class=""></div><div class="">To my knowledge, this *only* will affect pip 1.5.x.</div><div class=""><br class=""></div><div class="">So the only way forward I can see to make this change, which I think is a good</div><div class="">change and will remove a big "gotcha" from using a mirror, is to coordinate</div>
<div class="">a release of bandersnatch that coincides with pip 1.6, and tell people they</div><div class="">need to upgrade in lockstep.</div><div class=""><br class=""></div><div class="">Does anyone have any other ideas?</div></div><br class=""><div class="">
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" class=""><div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" class="">
<div class="">---</div><div class="">Donald Stufft</div><div class="">PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA</div></div></div>
</div>
<br class=""></div>_______________________________________________<br class="">Distutils-SIG maillist - <a href="mailto:Distutils-SIG@python.org" target="_blank" class="">Distutils-SIG@python.org</a><br class=""><a href="https://mail.python.org/mailman/listinfo/distutils-sig" target="_blank" class="">https://mail.python.org/mailman/listinfo/distutils-sig</a><br class="">
</div></blockquote></div><div class=""><br class=""></div></div></div><div class=""><div class="">Just thought of this, if the normalized name doesn’t match the "real" name,</div><div class="">then add entries for both. This will make it so that pip 1.5 continues to work</div>
<div class="">and pip 1.6+.</div></div><div class=""><br class=""><div class="">
<div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word;" class="">
<div class="">---</div><div class="">Donald Stufft</div><div class="">PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA</div></div></div>
</div>
<br class=""></div></div><br class="">_______________________________________________<br class="">
Distutils-SIG maillist - <a href="mailto:Distutils-SIG@python.org" class="">Distutils-SIG@python.org</a><br class="">
<a href="https://mail.python.org/mailman/listinfo/distutils-sig" target="_blank" class="">https://mail.python.org/mailman/listinfo/distutils-sig</a><br class="">
<br class=""></blockquote></div><br class=""></div>
</div></blockquote></div><br class=""><div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">---</div><div class="">Donald Stufft</div><div class="">PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA</div></div></div>
</div>
<br class=""></body></html>