[Distutils] restart-in-venv

[Nick Coghlan]

On 5 February 2014 00:00, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> On Tue, 4/2/14, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> It's a higher level of isolation than that offered by *not* installing
>> pip into the virtual environments - the two approaches are
>> *not* equivalent, and one is *not* clearly superior to the other,
>> but rather there are pros and cons to both.
>
> Obviously the two approaches aren't equivalent,  or there wouldn't
> be anything to discuss. What you haven't yet done is answered
> my question about what the cons are of the not-in-venv approach
> in the practical "you can't do X, because of Y", sense. What
> *practical* problems are caused by a lower level of isolation
> exactly, which make the higher level of isolation necessary?
>
> ISTM the pip developers tried to implement -E because it
> was (independently of me) seen as a desirable feature, and
> Paul said as much, too. If it didn't work for pip, that could well
> be because of the specific implementation approach used,
> as I've said in my response to Carl.
>
> Rather than talk in abstractions like "higher levels of isolation",
> which have an implied sense of goodness, it would be helpful
> to spell out specific scenarios (in the X/Y sense I mentioned
> before) that illustrate that goodness versus the badness of
> other approaches.

I'm not sure how I can explain it more clearly, but I'll try.

Scenario A: one tool to rule them all. Not versioned per virtualenv.
If you want multiple versions on a system, you must manage that
independently of both virtualenv and the system package manager. You
may end up manipulating a virtualenv with version X.Y one day and
version X.Z the next day, and then go back to version X.Y.

Scenario B: the installation tool is installed into the virtual
environment when it is created. No external event will impact the
behaviour of installation commands in that virtual environment - you
must upgrade the tool within the environment. There are only two
packaging systems in use: the system package manager and virtualenv.
There is no third mechanism needed to version the package management
tool, because that is handled using virtualenv. If one client is using
pip 1.3, and the other pip 1.5 (with it's much stricter default
security settings), then having separate virtual environments for
separate projects will deal with that automatically.

I'm not saying what you want to do *can't* be done (it obviously can).
I'm saying it adds additional complexity, because you're adding a
second isolation mechanism into the mix, with a completelely
independent set of ways of interacting with it.

For an example of that consider that "pip install" and "python -m pip
install" are essentially equivalent commands. The status quo preserves
that equivalence even inside a virtual environment, because there is
no separate mechanism to be considered. If the system Python
site-packages is disabled in an active virtual environment, you don't
need to carefully ensure that doesn't affect your install tool,
because it's right there in the virtual environment with you.

Is it possible that a tool could be written that *did* adequately
address all of these cases, with the test suite to prove it? Yes, I'm
sure that's possible. What I'm saying is that I *don't really see the
point*, since it doesn't achieve anything that couldn't be achieved
with a utility script that simply iterates over all defined virtual
environments and execute "pip install --upgrade" in each of them (with
the wheel cache, every venv after the first should be quick).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia