[Distutils] Finally making distlib handle spaces

[Radon Rosborough]

> The current behaviour fails, as you note, but it does so in a
> "standard" way - shebang behaviour (and its limitations) is
> well-known.

I agree with this, but in my opinion the shebang is simply an
implementation detail of virtualenv. I would like to quote @JDLH from
[1]: "There is nothing about the value provided by virtualenv that
demands it use the shebang." If the shebang were fundamentally
necessary to provide the functionality of virtualenv, it would make
sense for virtualenv to inherit the shebang's restrictions. But this
is not the case, so I think that the shebang should be considered an
implementation detail that the end user should not need to be aware
of.

> At the moment, your proposal is just to use "an alternative" launch
> process. Without a specific proposal, it's impossible to judge
> whether the solution is better than the current situation.

We already have three specific patches which provide alternative
launch processes: [2], [3], and [4]. I feel like that should be
specific enough to start a discussion. In fact, Vinay specifically
requested a discussion about [2] be raised on distutils-sig [5]. The
only reason that no action has been taken is that nobody started that
discussion (until now).

> I would have thought that "#!/usr/bin/env sh" runs the risk of
> picking up a malicious sh executable injected into the user's PATH.

That's certainly a valid concern. Does this happen in the real world?
I feel like if you have a malicious sh executable on your PATH, you're
going to have a lot more problems than just from virtualenv. But this
is a good reason that we may want to restrict the patch to only take
effect when using the shebang directly would fail.

> Also, different systems use different sh implementations - so care
> would need to be taken to only code in the lowest common denominator
> syntax.

Can we assume POSIX compatibility? Even if not, we're not doing
anything complicated, only passing some arguments to a command. Surely
that can be done in pretty much any shell one can find.

> multiple proposals and bikeshedding

Although we have three implementations, my personal preference is for
[4]. This is because:

* It avoids the need for creating new files.
* It only takes effect when necessary (i.e. when the shebang won't
  work).
* The code is fairly clean.

> On Windows (where shebang processing is handled by the wrapper exe,
> and is well defined and robust) there should be no change to the
> current behaviour.

Agreed.

[1]: https://github.com/pypa/virtualenv/issues/53#issuecomment-302019457
[2]: https://bitbucket.org/pypa/distlib/pull-requests/31
[3]: https://bitbucket.org/pypa/distlib/pull-requests/32
[4]: https://bitbucket.org/pypa/distlib/pull-requests/33
[5]: https://bitbucket.org/pypa/distlib/pull-requests/31#comment-29795586