[Python-ideas] Draft PEP for virtualenv in the stdlib

[Barry Warsaw]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Oct 25, 2011, at 11:37 AM, Carl Meyer wrote:

>Also, in earlier discussions on distutils-sig some people considered it
>a _feature_ to have the virtual environment's Python binary copied,
>making the virtual environment more isolated from system changes.
>Obviously, this is an area where programmers and sysadmins often don't
>see eye to eye ;)
>
>The technique in this PEP works just as well with a symlinked binary,
>though, and I don't see much reason not to provide a symlink option.
>Whether it is on by default where supported is something that may need
>more discussion (I don't personally have a strong opinion either way).

I definitely think folks will want copy and symlink options, for exactly the
reason you and Nick point out.  As for what the default should be, I think
maximal isolation probably wins, and thus copying should be the default.  I
wouldn't scream if it were the other way 'round though.

>> 'site' is *way* too overloaded already, let's not make it worse. I
>> suggest "sys.venv_prefix".
>
>My original thinking here was that sys.site_prefix is an attribute that
>should always exist, and always point to "where stuff should be
>installed to site-packages", whether or not you are in a venv (if you
>are not, it would have the same value as sys.prefix). It's a little odd
>to use an attribute named "sys.venv_prefix" in that way, even if your
>code doesn't know or care whether its actually in a venv (and in general
>we should be encouraging code that doesn't know or care). (The attribute
>doesn't currently always-exist in the reference implementation, but I'd
>like to change that).

That makes sense to me.  It would be nice to have the location of your
site-prefix always available for calculations regardless of your
"virtualness".  Note that in all likelihood, Debian will have to modify that
so that it points to dist-packages instead of site-packages.  Will there be an
easy hook/config file or some such that we could use to make that change?

>I agree that "site" is overloaded, though. Any ideas for a name that
>doesn't further overload that term, but still communicates "this
>attribute is a standard part of Python that always has the same meaning
>whether or not you are currently in a venv"?

sys.addons_directory, sys.plugins_directory, sys.external_packages or some
such?

>No, but in real-world usage, scripts from installed packages in the
>virtualenv will be installed there. Putting the Python binary in the
>same location as the destination for installed scripts is a pretty
>important convenience, as it means you only need to add a single
>directory to the beginning of your shell path to effectively "activate"
>the venv.

+1

>> I'm actually not seeing the rationale for the obfuscated FHS inspired
>> layout in the first place - why not dump the binaries adjacent to the
>> config file, with a simple "site-packages" directory immediately below
>> that? If there are reasons for a more complex default layout, they
>> need to be articulated in the PEP.
>
>The historical reason is that it emulates the layout found under
>sys.prefix in a regular Python installation (note that it's not actually
>Linux-specific, in virtualenv it matches the appropriate platform; i.e.
>on Windows it's "Lib\" rather than "lib\pythonX.X"). This was necessary
>for virtualenv because it couldn't make changes in distutils/sysconfig).
>I think there may be good reason to continue to follow this approach,
>simply because it makes the necessary changes to distutils/sysconfig
>less invasive, reducing the need for special-casing of the venv case.
>But I do need to look into this a bit more and update the PEP with
>further rationale in any case.
>
>Regardless, I would not be in favor of dumping binaries directly next to
>pyvenv.cfg. It feels cleaner to keep scripts and binaries in a directory
>specifically named and intended for that purpose, which can be added to
>the shell PATH.
>
>I also think there is some value, all else being roughly equal, in
>maintaining consistency with virtualenv's layout. This is not an
>overriding concern, but it will make a big difference in how much
>existing code that deals with virtual environments has to change.

All of that makes sense to me, so adding these rationales to the PEP would be
useful.

- -Barry
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBCAAGBQJOpx1XAAoJEBJutWOnSwa/I2YP/jXyJCu5uIVp20H+SjlqlnmB
y5tbzgSTPi3Yf1In1+bs1bPdNhqlZYi1WfmGRPA7QfXWK4oeSpPXTjs/bAaTYmPE
GjWDvQKue1hNv8qdFbo0IyI3u+2Zq0KB3sC+ACxB8WZ/Xc9wt4q47zMOTods4/Zj
ct5nvsib6xF/ZGk2BJopEQwFHhraxuNkrhSsgbUUsQJ3XEmVtLQZ4Dyr5gxRDmOF
/oShU556C3JjyPsXkNmURjQgXsbsWN/7j815yVpsLWNm5fjCPjqJAjY64RBFe9l/
EJqutLvqfQJJ1ie/R8xBBLF1LWNzzOeWgcgenbrdlgXa7xuGQE8psb1CSxdEUgec
9CUqsZbXDhQDtgdIcqsemhtGSIrl8UtCyrIDj7HUqK6ibOLTCa83AztpyupXpcki
ynTwJLdSYRc+8IH6kLQ5DpqYCYC+14jEoMrDSSRGB/EyAE1P0l3DE710JtYHV12R
6e0Ciw8qjuhUTnmAr6H5DnT8UaWXENBDkDbel6ABHxEKalavAnr3kzjUTLcnTwMJ
QfrbcVDBRnGV1swelLZ3oXitgofCCaz7Ca4kwqHikUQ5R/iPNl5AJyyJenFHsPrc
J0jfSQEqPP3FOO27atPlVy/9lE45Ac0D7UcMq5IrfruLd4Rw7P50sPIxa+anAfKK
GEuw81sCOyyJ4PU0cfxw
=KObT
-----END PGP SIGNATURE-----