On Sat, 1/2/14, Nick Coghlan <ncoghlan@gmail.com> wrote:
I have no idea how most of the internal design decisions on pip were made (and I neither need nor particularly want to know - that would be rather missing the point of collaborative development). However, you're trying to make out that the distil approach is so clearly superior that you don't
Not "superior" in terms of "clever". I've made a point of saying that distil *doesn't do anything clever* in this area. If you're saying "a bunch of design work", making "a bunch" sound like "a lot", without any detailed knowledge of the design internals, then how do you know if it was a hard engineering issue or a trivial one? If I say "I didn't find it hard, and I'm not claiming to be especially clever" then at least I'm speaking from a knowledge of what it takes to achieve something, rather than perhaps taking someone else's word for it. Decisions should be properly informed, in my view. Oh, and collaborative development sometimes involves challenging prior work and assumptions. That's one way in which things get better - we build on the past, but don't remain enslaved to it.
understand why anyone would ever have implemented it differently, and I'm trying to point out that there's an entirely plausible answer to that question that doesn't involve assuming poor engineering trade-offs on the part of the pip development team.
I didn't (at least, not intentionally) say anything about poor engineering trade-offs made by anyone - what would be the point of that? I do say that better engineering approaches might exist, and suggest that the way distil does it one such approach. If you can suggest technical flaws in the approach distil takes, that's one thing, and I will be keen to look at any points you make. If you have no time to examine distil and make pronouncements, that's fine too - we're all busy people. But defending the status quo seemingly on nebulous technical grounds like "a bunch", while avowing no knowledge of the technical details, seems wrong. How many milli-hodges are there in a bunch? ;-)
No, things we do in our free time are the *best* opportunities to investigate things like that, because they're not necessarily goal oriented - they're more about satisfying our curiousity.
I don't disagree - my clumsily made point was to to say "volunteer" in the sense of "part-time", and therefore to strengthen the fact that I well understand "limited resources".
Consider the following two engineering design options: [details snipped] Now *a priori*, you don't know how complex B is going to be. On the other hand, you already *know* A will work, because you already have working virtual environments.
That is fine as an *a priori* position, and it's the position I was in when contemplating the development of distil. But we don't stay stuck in *a priori* forever: as I developed distil, it became clear that alternative approaches can work. Of course there were uncertainties at the beginning, but by doing the development work I was able to understand better the nature of those, and I could convert them into certainties of what would and wouldn't work.
And from a design uncertainty point of view, A also has the advantage: there is *no* design uncertainty, because you're just using the already-known-to-work virtual environment mechanism.
So you're saying PEP 405 was a mistake? We should just have stuck with the tried-and-tested virtualenv, which required changing every time a new Python version came out, because it used (of necessity) what might be uncharitable to call "hacks"?
Now, you *have* had time to explore option B in distil, and have discovered that it actually wasn't that hard after all. Cool, that's great and would be a nice feature to have in pip as well. However, the fact that *having already done it*, you discovered it wasn't all that difficult, doesn't change the fact that the level of effort involved was still greater than the nonexistent amount of work needed to get the existing solution working
Yes, but it's not as if I'm just presenting this aspect of distil *now*. I certainly highlighted the salient features on this list *before* the PEP 453 ship set sail and distil is pretty well documented on readthedocs.org.
and I noticed that you cut the part of the email pointing out the concrete technical benefits of being able to isolate the installer in addition to the installed components.
I wasn't trying to gloss over anything. If an installer maintains backward compatibility, then isolating the installer is not especially valuable. Even if the installer doesn't get it right, using different versions of the installer is perfectly possible (and easy with distil, as it's a single-file executable), and in my experience it's not been unheard of to have had to "downgrade" virtualenv or pip because a new release broke something.
It's one thing to disagree with a design decision after we have made the absolutely best possible case that we can for why that decision was the right choice, and still failing to convince ourselves. It's something else entirely to dismiss someone else's use case as invalid and *then* argue that an arguably simpler design (that doesn't handle the dismissed use case and was still only hypothetical at the time the decision was made)
Is this some revisionist view of history? I'm confused :-) The original announcement I made about distil was on 26 March, 2013: http://thread.gmane.org/gmane.comp.python.distutils.devel/17336 The relevant features were fully documented in the version 0.1.0 documentation for distil, which was tagged on 22 March, 2013: https://bitbucket.org/vinay.sajip/docs-distil/commits/04ac4badcab941e12ff6ec... This was on readthedocs.org and pythonhosted.org at the time of the announcement. The date on PEP 453 is 10 August, 2013 - at least four months later: http://www.python.org/dev/peps/pep-0453/ So, if no one who participated in the PEP 453 decision had time to look at distil in that four month period, fair enough. It doesn't mean that I'm somehow trying to bring something *new* to the discussion *now*. I certainly made my reservations about the pip- bundling route at the time it was being discussed here, but if nobody was willing to look at alternatives, it's not something I could help. Of course, once the decision is made, what can you do but defend it? I understand your position, and I'm not trying to change anything. If anyone wants to provide specific, technical feedback about any problems with distil's approach in any area, I'll welcome that feedback - but I've already expressed my view on non-technical battles. Regards, Vinay Sajip