[pypy-dev] Virtualenv

Donald Stufft donald at stufft.io
Tue May 13 13:18:03 CEST 2014


On May 13, 2014, at 3:54 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Donald,
> 
> On 12 May 2014 18:39, Donald Stufft <donald at stufft.io> wrote:
>> I have a half written patch to backport ensurepip to PyPy, I just got confused
>> trying to test it and task switched to cover some other things.
>> 
>> As far as virtualenv goes, instead of virtualenv I’d much rather that you backport
>> the venv module. It does require some interpreter changes but I don’t think
>> they are particularly hard.
> 
> I am myself not following closely these various packaging schemes, but
> I do know that with any Python or PyPy installation, I need a working
> pip and often a virtualenv.  So from my own point of view, these two
> are the most used pieces of software I need from a PyPy installation
> --- the ones I have to worry about and reinstall every time, and every
> time I need to google around for how this is done exactly.  Anything
> else can be installed from there.  So that's why I'd suggest we
> include these two exactly.  In particular, I don't see "ensurepip" as
> intrinsically solving anything for PyPy (here talking about the 2.7
> version only) --- most people just need "pip".  Same about virtualenv
> --- sorry, it's the standard, and I don't see the point of adding a
> different one (again, talking about the 2.7 version only.  If CPython
> 3.x decides to include ensurepip or whatever, we can follow suit on
> PyPy3).

So FWIW CPython has already included ensurepip in 3.4.

So currently PyPy ships with a few preinstalled things (cffi, greenlet,
readline) which are shipped inside of lib_pypy. One of the key requirements of
PEP453 (which added the ensurepip module to 3.4) was that it remained possible
to independently upgrade pip. However if PyPy ships pip installed into lib_pypy
like the other things it ships with then pip will not be able to be
independently upgraded and people will be stuck with whatever version of pip
was available when PyPy shipped.

The idea behind ensurepip is that instead of shipping pip, you ship a bootstrap
which will install pip for you. This would mean that in order to install pip
into a PyPy site-packages all you would need to do is run ``pypy -m ensurepip``
and it would take care of the rest for you. CPython, by default, runs this
as part of ``make install`` and ``make altinstall`` and also as part of the
Windows and OSX installers. I'm not sure if PyPy has a similar place that
executes during the "install" phase as I generally use pyenv or homebrew or
a package manager to install PyPy. I would not be opposed to including pip
directly preinstalled into PyPy but it *must* preserve the ability to
independently upgrade pip however I think you still want ensurepip for the
venv module.

The reason I'm asking for a backport of the venv module instead of virtualenv
itself is simple. The venv module works way better and without massive platform
specific hacks. This is because it includes support within the intepreter for
a virtual environment and doesn't rely on monkeypatching and obscure Python
startup semantics in order to make it function. I've currently got a branch of
virtualenv that I'm working on where virtualenv itself will utilize the
isolation provided by the standardlib venv module (when it's available) and
only fall back to the hacks which work on the earlier versions. 

In Python 3.3 the venv module did not include pip inside of a freshly created
environment which limited it's usefulness to end users however if PyPy chooses
to ship pip (and virtualenv) directly then it doesn't specifically need
ensurepip and it's venv module could function similarly to Python 3.3's because
that is all the virtualenv code will need in order to support a much cleaner
isolation.

Part of the way that the virtualenv module works is that you don't need to have
it installed into a particular python in order to create an environment for it.
So if you have virtualenv installed with CPython 2.x or 3.x and you also have
PyPy installed on your system you can create a PyPy environment without
installing anything into it by running ``virtualenv -p pypy myenv``. However
in order to get the support for the more stable isolation provided by the venv
module, that will need to be part of PyPy.

I don't really feel strongly one way or the other if PyPy should or shouldn't
ship with virtualenv (given that it doesn't generally need installed) however
if it does it should also be independently upgradeable such as pip should be
and ideally PyPy will still have the venv module backported so that future
versions of virtualenv can utilize it for better isolation techniques.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20140513/aff49d2d/attachment.sig>


More information about the pypy-dev mailing list