<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Sep 28, 2014, at 5:36 PM, M.-A. Lemburg <<a href="mailto:mal@egenix.com" class="">mal@egenix.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">On
28.09.2014 21:31, Donald Stufft wrote:</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
Hello All!<br class="">
<br class="">
I'd like to discuss the idea of moving PyPI to having immutable files. This<br class="">
would mean that once you publish a particular file you can never reupload that<br class="">
file again with different contents. This would still allow deleting the file or<br class="">
reuploading it if the checksums match what was there prior.<br class="">
<br class="">
This would be good for a few reasons:<br class="">
<br class="">
* It represents "best practices" for version numbers. Ideally if two people<br class="">
have version "2.1" of a project, they'll have the same code, however as it<br class="">
stands two people installing at two different times could have two very<br class="">
different versions.<br class="">
<br class="">
* This will make improving the PyPI infrastructure easier, in particular it<br class="">
will make it simpler to move away from using a glusterfs storage array and<br class="">
switch to a redudant set of cloud object stores.<br class="">
<br class="">
<br class="">
In the past this was brought up and a few points were brought against it, those<br class="">
were:<br class="">
<br class="">
1. That authors could simply change files that were hosted on not PyPI anyways<br class="">
so it didn't really do much.<br class="">
<br class="">
2. That it was too hard to test a release prior to uploading it due to the<br class="">
nature of distutils requiring you to build the release in the same command<br class="">
as the upload.<br class="">
<br class="">
With the fact that pip no longer hits external URLs by default, I believe that<br class="">
the first item is no longer that large of a factor. People can do whatever they<br class="">
want on external URLs of course, however if something is coming from PyPI as<br class="">
end users should now be aware of, they can know it is immutable.<br class="">
<br class="">
Now that there is twine, which allows uploading already created packages, I<br class="">
also believe that the second item is no longer a concern. People can easily<br class="">
create a distribution using ``setup.py sdist``, test it, and then upload that<br class="">
exact thing they tested using ``twine upload <path to sdist>``.<br class="">
</blockquote>
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">-1.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">It
does happen that files need to be reuploaded because of a bug</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">in
the release process and how people manage their code is really</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">*their*
business, not that of PyPI.</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
</div>
</blockquote>
<div><br class="">
</div>
<div>Can you describe a reasonable hypothetical situation where this would occur</div>
<div>often enough as to be something that is likely to happen on a consistent</div>
<div>basis? Originally the problem was there was little ability to easily upload</div>
<div>pre-created files so there was a reasonable chance that there may be a</div>
<div>packaging bug that didn’t get exposed until you actually packaged + released.</div>
<div><br class="">
</div>
<div>With the advent of twine though it’s now possible to test the exact bits that</div>
<div>get uploaded to PyPI making that particular issue no longer a problem.</div>
<div><br class="">
</div>
<div>However, the fact that the files are not immutable *do* cause a number of</div>
<div>problems that need to be worked around in the mirroring infrastructure, the</div>
<div>CDN, and for scaling PyPI out and removing the glusterfs component.</div>
<br class="">
<blockquote type="cite" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">FWIW,
I am getting increasingly annoyed how PyPI and pip try to dictate</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">the
way package authors are supposed to build, manage and host their</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Python
packages and release process. Can we please stop this ?</span><br class="">
</blockquote>
<br class="">
</div>
<div>I recognize your annoyance, however I think that the changes that have been</div>
<div>made are overall good changes that negatively affect a minor subset of people</div>
<div>and positively affect a much wider group of people. Speaking as one of the</div>
<div>people who are pushing the hardest for the kinds of changes that I assume you’re</div>
<div>talking about, I do try and figure out ways to continue to enable the “alternative”</div>
<div>methods of doing things while still allowing forward progress on making things</div>
<div>better for the masses.</div>
<div><br class="">
</div>
<div>If there’s something I could have done more to ease that pain other than</div>
<div>*not* making changes at all then I would be gracious to hear them! I don’t want</div>
<div>to make these changes painful for people where that can be helped.</div>
<br class="">
<div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">---</div>
<div class="">Donald Stufft</div>
<div class="">PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA</div>
</div>
</div>
</div>
<br class="">
</body>
</html>