[Distutils] Immutable Files on PyPI

[Donald Stufft]

On Sep 29, 2014, at 4:46 AM, M.-A. Lemburg <mal at egenix.com<mailto:mal at egenix.com>> wrote:

You are missing out on cases, where the release process causes files to
be omitted, human errors where packagers forget to apply changes to
e.g. documentation files, version files, change logs, etc., where
packagers want to add information that doesn't affect the software
itself, but meta information included in the distribution files.

Such changes often do not affect the software itself, and so are not
detected by software tests.

If I understand you correctly, you are essentially suggesting that it
becomes impossible to ever delete anything uploaded to PyPI, i.e.
turning PyPI into a WORM.

This would mean that package authors could never correct mistakes,
remove broken packages distribution files, ones which they may be
forced to remove for legal reasons, ones which they find are infected
with a virus or trojan, ones which they uploaded for fun or
by mistake.

This doesn't have anything to do with making the user experience
a better one. It is ignorant to assume that package authors who
sometimes delete distribution files, or at least want to have the
possibility to do so, don't care for their users. We are in
Python land, so most authors will know what they are doing and
do care for their users.

After all: Why do you think I'm arguing against this proposal ?
Because I want users of our packages to get the best experience
they can get, by downloading complete, correct and working
distribution files.

This whole idea also has another angle, namely a legal one:
the PSF doesn't own the distribution files it hosts on PyPI.

So far, the argument to not fix the much too broad license on PyPI
was that authors were able to delete files on PyPI to work around
the unneeded "irrevocable" part of that license.

With the suggested change, authors would have to give up complete
control over their distribution files to the PSF in order for their
packages to be installable by pip using its default settings.

Others already said it, but let me be clear about it, this proposal does not in
any way seek to prevent authors from being able to delete files from PyPI. It
still allows them to delete anything at anytime and it still publishes that
information for mirrors (although mirrors are certainly under no obligation
to respect it if they desire not to). I completely agree with you that
disallowing authors to *delete* files would be incredibly short sighted and
wrong and I would be one of the people against such a change.

This proposal is strictly limited to the ability to delete a particular file,
let's say "foobar-1.0.tar.gz" and then reupload a different "foobar-1.0.tar.gz"
in it's place. If a mistake is made in the release, that's *ok* it can be
deleted, the only constraint is that with this change you'd need to increment
the version in some way, likely with a .postN or just bumping the last digit,
to signal to users that the bits in this has changed in some way.


This kind of lock-in and removal of author rights is not something
I can support as PSF director. Those authors are the ones that have
created a large part of our Python eco system and they are the ones that
have put in work to get Python to where it is now: one of the best
integrated programming languages you can find. We owe a lot to those
authors and need to care for them.

I *do* deeply care for the experience as an author as well as someone
installing from PyPI. After all I use PyPI in both capacities on a regular
basis.


Finally, changes such as the above will result in more authors
to switch to alternative hosting platforms such as conda/binstar.org<http://binstar.org>
or plain github clone + setup.py install (which is becoming increasingly
popular). Do you really believe that this will make the user experience
a better one in the long run ?

If we want to make it attractive for package authors to host their
packages on PyPI, we have to give them flexibility, respect their
rights and be welcoming.


I don't believe it's accurate to say people are switching away from PyPI in any
sort of relevant numbers. PyPI's usage is increasing, both in the number of
people releasing packages and in the number of people consuming packages.
Particularly the number of people consuming packages has risen massively. Do
you have any numbers or proof to backup the claim that people are switching
away?

To be completly honest the feedback that I get and see is overwhelmingly
positive for every change that has been made so far. That's not to say there
haven't been those who have been against some or all of the changes but those
people are generally in an extreme minority. This isn't to say that the changes
have been globally liked, but that it would be very surprising to me that
people are moving away from PyPI and if you have numbers/proof of that I
would love to see it.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140929/c1a9fe58/attachment-0001.html>