On May 8, 2014, at 11:21 AM, R. David Murray rdmurray@bitdance.com wrote:
On Thu, 08 May 2014 10:37:15 -0400, Donald Stufft donald@stufft.io wrote:
Most users are not going to care up until the point where the external server is unavailable, and then they care a whole lot. On the tin it sounds reasonable to just download the external file if the server is up however we've done that for a long time and the end result has been end user pain.
Now requiring someone to add a flag in order to download an externally hosted file is also end user pain. The difference between the two pains is when they happen. The requiring a flag pain happens at the point of decision, when you decide to make your deployment depend on something hosted externally. The default to allow pain happens sometime in the future, when you may or may not have any idea why suddenly your installs aren't working (and when you look, PyPI is up so you're really very confused why this particular file doesn't work). Even worse is the case when a project has some old files, but the newer versions aren't hosted and suddenly people are getting very old releases which is even more confusing to the end users.
Ah, I understand now.
Your perspective is as someone who is using pip for deployment.
Deployment, or any kind of situation where you want to have a reproducible build. Generally via deployment yes.
I'm speaking from a python+plus+pip end-user perspective, which is going to be even more common now that it is part of the Python distribution.
I'm not sure how you reconcile these two worlds. I would venture to suggest that if you are using it for deployment you really ought to be using a local package repository[*], not the global one; but, as someone observed, the sensible thing to do and what people actually do are often very different; and, since I haven't done this particular kind of deployment scenario in Python myself, there may be reasons I'm missing.
People simply don’t do this[1]. Especially in a world with things like Heroku existing which makes it stupid simple to use pip to install from PyPI but installing from your own server requires standing up some infrastructure.
However, your last mention of "end users" confuses me. Why are "end users" getting old packages in a deployment situation? Isn't it the developer/operations people (and the latter would, I assume, control the 'external packages' flag) who would be seeing that? Maybe you mean something by deployment different from how I use the word?
Someone using pip, this may be a developer who is just trying to recreate their production environment locally, this may be someone using chef/puppet to automate installing via pip, this may be someone pushing to Heroku.
The old versions thing is more that it’s really confusing when you type
pip install foo on a monday and get 2.0, and pip install foo on
weds
and get 0.4.
--David
[] I found it such* a pain to do this for perl/cpan. I have a project for a customer where I have to do this, and the hoops I had to jump through to get a reliable deployment (where packages wouldn't be unexpectedly upgraded under my feet) were nasty. (This was several years ago and I haven't revisited it, so maybe things have gotten better, or I just missed something.)
I haven't had to do it for python yet, oddly enough, so I don't know how hard it is for Python.
For Python with pip you can use a requirements.txt file to create a set of dependencies that are pinned to exact versions like:
foo==2.0 bar==2.3
And pip will (theoretically, our dep solving is real bad ATM) install exactly those versions from your index server. Generally this means PyPI which means the author can delete the version and upload a new file with the same version number. However it’s also trivial to stand up your own server. It can be as easy as pointing nginx/Apache at a static directory with autoindex = True. (See: https://wheels.caremad.io/).
On top of that there is peep which adds a secure message digest on it to make sure that the author/index didn’t swap things out on you, and there is some discussion about how best to add that to pip itself.
Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io
Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA