[Python-ideas] Draft PEP for virtualenv in the stdlib

[Carl Meyer]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Nick,

On 10/25/2011 06:47 PM, Nick Coghlan wrote:
> I've been persauded that copying by default is reasonable - so long as
> the option to use symlinks is there, sysadmins can put a policy for
> their organisation in place that requires its use.

Sure - the symlink option is now there in both PEP and impl. Vinay
actually flipped it on by default for platforms where it's reliable, but
if you and Barry are both now leaning towards leaving it off by default
everywhere, that's equally fine by me. We'll flip it off by default.

>>> 'site' is *way* too overloaded already, let's not make it worse. I
>>> suggest "sys.venv_prefix".
>>
>> My original thinking here was that sys.site_prefix is an attribute that
>> should always exist, and always point to "where stuff should be
>> installed to site-packages", whether or not you are in a venv (if you
>> are not, it would have the same value as sys.prefix). It's a little odd
>> to use an attribute named "sys.venv_prefix" in that way, even if your
>> code doesn't know or care whether its actually in a venv (and in general
>> we should be encouraging code that doesn't know or care). (The attribute
>> doesn't currently always-exist in the reference implementation, but I'd
>> like to change that).
>>
>> I agree that "site" is overloaded, though. Any ideas for a name that
>> doesn't further overload that term, but still communicates "this
>> attribute is a standard part of Python that always has the same meaning
>> whether or not you are currently in a venv"?
> 
> I'd actually prefer that we use the explicit "sys.prefix" and
> "sys.venv_prefix" naming (with the latter set to None when not in a
> virtual env) and possibly offer a convenience API somewhere that hides
> the "sys.prefix if sys.venv_prefix is None else sys.venv_prefix"
> dance.
> 
> There's already a site.getsitepackage() API that gets the full list of
> site package directories (it's not a given that there's only one).

This is true. I guess it mostly boils down to making the code in
sysconfig less ugly; any external tool should be using sysconfig/site
APIs anyway, not doing custom stuff with sys.*_prefix attributes to find
site-packages. But a helper function can take care of that ugliness in
sysconfig, so I'm ok with your approach to the sys attributes. (Though
we still have to pick a name to refer to "sys.venv_prefix if
sys.venv_prefix else sys.prefix", it just pushes the naming question
onto the helper function :P).

>>> I'm actually not seeing the rationale for the obfuscated FHS inspired
>>> layout in the first place - why not dump the binaries adjacent to the
>>> config file, with a simple "site-packages" directory immediately below
>>> that? If there are reasons for a more complex default layout, they
>>> need to be articulated in the PEP.
>>
>> The historical reason is that it emulates the layout found under
>> sys.prefix in a regular Python installation (note that it's not actually
>> Linux-specific, in virtualenv it matches the appropriate platform; i.e.
>> on Windows it's "Lib\" rather than "lib\pythonX.X"). This was necessary
>> for virtualenv because it couldn't make changes in distutils/sysconfig).
>> I think there may be good reason to continue to follow this approach,
>> simply because it makes the necessary changes to distutils/sysconfig
>> less invasive, reducing the need for special-casing of the venv case.
>> But I do need to look into this a bit more and update the PEP with
>> further rationale in any case.
>>
>> Regardless, I would not be in favor of dumping binaries directly next to
>> pyvenv.cfg. It feels cleaner to keep scripts and binaries in a directory
>> specifically named and intended for that purpose, which can be added to
>> the shell PATH.
>>
>> I also think there is some value, all else being roughly equal, in
>> maintaining consistency with virtualenv's layout. This is not an
>> overriding concern, but it will make a big difference in how much
>> existing code that deals with virtual environments has to change.
>>
>>> If the problem is wanting to allow cross platform computation of
>>> things like the site-packages directory location and other paths, then
>>> the answer to that seems to lie in better helper methods (whether in
>>> sysconfig, site, venv or elsewhere) rather than Linux specific layouts
>>> inside language level virtual environments.
> 
> As Barry said, the rationale sounds reasonable, but the PEP needs to
> explain it in terms those of us not fully versed in the details of
> cross-platform virtual environments can understand (examples of both
> *nix and Windows layouts with a simple example package installed would
> probably be useful on that front)

Yep, adding more Windows examples and generally a fuller rationale for
the layout to the PEP are next on my list.

Carl
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6nX2YACgkQ8W4rlRKtE2dCWgCePm8irdm52Mz2klfBJBY/DwCt
0bEAn32tOMwjYQmrwZiDEtv2YSkE82Ff
=WKEK
-----END PGP SIGNATURE-----