draft PEP: virtual environments
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello python-dev,
As has been discussed here previously, Vinay Sajip and I are working on
a PEP for making "virtual Python environments" a la virtualenv [1] a
built-in feature of Python 3.3.
This idea was first proposed on python-dev by Ian Bicking in February
2010 [2]. It was revived at PyCon 2011 and has seen discussion on
distutils-sig [3], more recently again on python-dev [4] [5], and most
recently on python-ideas [6].
Full text of the draft PEP is pasted below, and also available on
Bitbucket [7]. The reference implementation is also on Bitbucket [8].
For known issues in the reference implementation and cases where it does
not yet match the PEP, see the open issues list [9].
In particular, please note the "Open Questions" section of the draft
PEP. These are areas where we are still unsure of the best approach, or
where we've received conflicting feedback and haven't reached consensus.
We welcome your thoughts on anything in the PEP, but feedback on the
open questions is especially useful.
We'd also especially like to hear from Windows and OSX users, from
authors of packaging-related tools (packaging/distutils2, zc.buildout)
and from Python implementors (PyPy, IronPython, Jython).
If it is easier to review and comment on the PEP after it is published
on python.org, I can submit it to the PEP editors anytime. Otherwise
I'll wait until we've resolved a few more of the open questions, as it's
easier for me to update the PEP on Bitbucket.
Thanks!
Carl
[1] http://virtualenv.org
[2] http://mail.python.org/pipermail/python-dev/2010-February/097787.html
[3] http://mail.python.org/pipermail/distutils-sig/2011-March/017498.html
[4] http://mail.python.org/pipermail/python-dev/2011-June/111903.html
[5] http://mail.python.org/pipermail/python-dev/2011-October/113883.html
[6] http://mail.python.org/pipermail/python-ideas/2011-October/012500.html
[7] https://bitbucket.org/carljm/pythonv-pep/src/
[8] https://bitbucket.org/vinay.sajip/pythonv/
[9] https://bitbucket.org/vinay.sajip/pythonv/issues?status=new&status=open
PEP: XXX
Title: Python Virtual Environments
Version: $Revision$
Last-Modified: $Date$
Author: Carl Meyer
This is really very comprehensive, thank you!
Why not modify sys.prefix? - --------------------------
As discussed above under `Backwards Compatibility`_, this PEP proposes to add ``sys.site_prefix`` as "the prefix relative to which site-package directories are found". This maintains compatibility with the documented meaning of ``sys.prefix`` (as the location relative to which the standard library can be found), but means that code assuming that site-packages directories are found relative to ``sys.prefix`` will not respect the virtual environment correctly.
Since it is unable to modify ``distutils``/``sysconfig``, `virtualenv`_ is forced to instead re-point ``sys.prefix`` at the virtual environment.
An argument could be made that this PEP should follow virtualenv's lead here (and introduce something like ``sys.base_prefix`` to point to the standard library and header files), since virtualenv already does this and it doesn't appear to have caused major problems with existing code.
Another argument in favor of this is that it would be preferable to err on the side of greater, rather than lesser, isolation. Changing ``sys.prefix`` to point to the virtual environment and introducing a new ``sys.base_prefix`` attribute would err on the side of greater isolation in the face of existing code's use of ``sys.prefix``.
It would seem to make sense to me to err on the side of greater isolation, introducing sys.base_prefix to indicate the base prefix (as opposed to sys.site_prefix indicating the venv prefix). Bugs introduced via a semi-isolated virtual environment are very difficult to troubleshoot. It would also make changes to existing code unnecessary. I have encountered no issues with virtualenv doing this so far. - C
On Sat, Oct 29, 2011 at 4:37 AM, Carl Meyer
If it is easier to review and comment on the PEP after it is published on python.org, I can submit it to the PEP editors anytime. Otherwise I'll wait until we've resolved a few more of the open questions, as it's easier for me to update the PEP on Bitbucket.
It's best to get it posted, firstly so it has an assigned PEP number (although some may argue having to call it "the virtualenv PEP" is a feature!), secondly so that it's easy for people to get hold of a formatted version. All the core committers can actually publish PEPs via the PEP hg repo, so Vinay could probably handle pushing the updates to python.org. Submission via the PEP editors is mainly there as a backstop for cases where there's no current core dev directly involved in the PEP. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Fri, 28 Oct 2011 12:37:35 -0600
Carl Meyer
What about include files? - -------------------------
For example, ZeroMQ installs zmq.h and zmq_utils.h in $VE/include, whereas SIP (part of PyQt4) installs sip.h by default in $VE/include/pythonX.Y. With virtualenv, everything works because the PythonX.Y include is symlinked, so everything that's needed is in $VE/include. At the moment the reference implementation doesn't do anything with include files, besides creating the include directory; this might need to change, to copy/symlink $VE/include/pythonX.Y.
As in Python there's no abstraction for a site-specific include directory, other than for platform-specific stuff, then the user expectation would seem to be that all include files anyone could ever want should be found in one of just two locations, with sysconfig labels "include" & "platinclude".
There's another issue: what if includes are Python-version-specific? For example, SIP installs by default into $VE/include/pythonX.Y rather than $VE/include, presumably because there's version-specific stuff in there - but even if that's not the case with SIP, it could be the case with some other package.
Why would that be a problem? Do you plan to install several versions of Python in a single VE?
Activation and Utility Scripts - ------------------------------
Virtualenv provides shell "activation" scripts as a user convenience, to put the virtual environment's Python binary first on the shell PATH. This is a maintenance burden, as separate activation scripts need to be provided and maintained for every supported shell.
We already have Unix shell scripts and BAT files in the source tree. Is it really complicated to maintain these additional shell scripts? Is there a lot of code in them? Regards Antoine.
Nick Coghlan
All the core committers can actually publish PEPs via the PEP hg repo, so Vinay could probably handle pushing the updates to python.org. Submission via the PEP editors is mainly there as a backstop for cases where there's no current core dev directly involved in the PEP.
Added as PEP 404 - hope y'all can find it ;-) Regards, Vinay Sajip
Antoine Pitrou
Why would that be a problem? Do you plan to install several versions of Python in a single VE?
No, but some packages might install headers in /include and others in /include/pythonX.Y. I wasn't sure whether this would cause a problem with files not being found during build, though I realise this can be worked around with specific -I flags to the compiler. At present, we only create a /include in the venv, but not /include/pythonX.Y.
We already have Unix shell scripts and BAT files in the source tree. Is it really complicated to maintain these additional shell scripts? Is there a lot of code in them?
No, they're pretty small: wc -l gives 76 posix/activate (Bash script, contains deactivate() function) 31 nt/activate.bat 17 nt/deactivate.bat The question is whether we should stop at that, or whether there should be support for tcsh, fish etc. such as virtualenv provides. IMO, if we provide the above as a bare minimum + an easy way for third-party tools to install replacements/additions, then we probably don't need to worry too much about an additional support burden in the stdlib - third parties can take up the responsibility for supporting additional shells or helper scripts. Regards, Vinay Sajip
On Sun, 30 Oct 2011 12:10:18 +0000 (UTC)
Vinay Sajip
We already have Unix shell scripts and BAT files in the source tree. Is it really complicated to maintain these additional shell scripts? Is there a lot of code in them?
No, they're pretty small: wc -l gives
76 posix/activate (Bash script, contains deactivate() function) 31 nt/activate.bat 17 nt/deactivate.bat
The question is whether we should stop at that, or whether there should be support for tcsh, fish etc. such as virtualenv provides.
I don't think we need additional support for more or less obscure shells. Also, if posix/activate is sufficiently well written (don't ask me how :-)), it should presumably be compatible with all Unix shells? Regards Antoine.
Antoine Pitrou
We already have Unix shell scripts and BAT files in the source tree.
Do we have a blessed location in the stdlib for data files in general? Although we're talking in this instance about scripts, they're just data as far as the venv module is concerned. While it's not uncommon for data which is included with packages to be installed in the source tree for that package(e.g. packaging's test data), I'm not sure what one would do with data which belongs to a top-level module. At the moment it's in the source as a base64-encoded string, but I'm not sure that's ideal - it's workable only because the data is so small. I don't really want to add a Lib/scripts.zip adjacent to venv.py, which venv accesses via os.path.dirname(__file__), because if every module did this, it would be a tad untidy. The other alternative would be to make venv a package with all its code in venv/__init__.py and a scripts.zip adjacent to that. Does that seem like a better solution? Can anyone suggest better alternatives? Sorry if this has come up before and I've missed something obvious. Regards, Vinay Sajip
On Sun, 30 Oct 2011 12:35:20 +0000 (UTC)
Vinay Sajip
The other alternative would be to make venv a package with all its code in venv/__init__.py and a scripts.zip adjacent to that. Does that seem like a better solution?
Please don't make it a zip file. We want code to be easily trackable and editable. Regards Antoine.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/30/2011 08:35 AM, Vinay Sajip wrote:
Antoine Pitrou
writes: We already have Unix shell scripts and BAT files in the source tree.
Do we have a blessed location in the stdlib for data files in general? Although we're talking in this instance about scripts, they're just data as far as the venv module is concerned. While it's not uncommon for data which is included with packages to be installed in the source tree for that package(e.g. packaging's test data), I'm not sure what one would do with data which belongs to a top-level module. At the moment it's in the source as a base64-encoded string, but I'm not sure that's ideal - it's workable only because the data is so small. I don't really want to add a Lib/scripts.zip adjacent to venv.py, which venv accesses via os.path.dirname(__file__), because if every module did this, it would be a tad untidy.
The other alternative would be to make venv a package with all its code in venv/__init__.py and a scripts.zip adjacent to that. Does that seem like a better solution? Can anyone suggest better alternatives? Sorry if this has come up before and I've missed something obvious.
+1 to making it a package and keeping the data in the package. - -1 to a zip file: each scripts should be a noraml, version-controlled entity. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6tb+8ACgkQ+gerLs4ltQ463wCfZoOOYK1c7XgAaihSdM9+0dxn /YgAoMVlq+ZRGA6xZUFNrajSbdr4aUQZ =P6zT -----END PGP SIGNATURE-----
Antoine Pitrou
Please don't make it a zip file. We want code to be easily trackable and editable.
Of course. I was thinking of a directory tree in the source, subject to our normal revision control, but processed during make or installation to be available as a zip file once deployed. It was a general point about data that I was making; in this particular case, that data just happens to be source code. Regards, Vinay Sajip
On Sun, 30 Oct 2011 15:42:11 +0000 (UTC)
Vinay Sajip
Antoine Pitrou
writes: Please don't make it a zip file. We want code to be easily trackable and editable.
Of course. I was thinking of a directory tree in the source, subject to our normal revision control, but processed during make or installation to be available as a zip file once deployed.
It would be even simpler not to process it at all, but install the scripts as-is (without the execute bit) :) Regards Antoine.
Antoine Pitrou
It would be even simpler not to process it at all, but install the scripts as-is (without the execute bit) :)
Sure, but such an approach makes it difficult to provide a mechanism which is easily extensible; for example, with the current approach, it is straightforward for third party tools to either easily replace completely, update selectively or augment simply the scripts provided by base classes. Regards, Vinay Sajip
On Sun, 30 Oct 2011 22:47:13 +0000 (UTC)
Vinay Sajip
Antoine Pitrou
writes: It would be even simpler not to process it at all, but install the scripts as-is (without the execute bit) :)
Sure, but such an approach makes it difficult to provide a mechanism which is easily extensible; for example, with the current approach, it is straightforward for third party tools to either easily replace completely, update selectively or augment simply the scripts provided by base classes.
I don't understand why a zip file makes this easier (especially the "update selectively" part). Regards Antoine.
Antoine Pitrou
I don't understand why a zip file makes this easier (especially the "update selectively" part).
Not a zip file specifically - just a binary stream which organises scripts to be installed. If each class in a hierarchy has access to a binary stream, then subclasses have access to the streams for base classes as well as their own stream, and can install selectively from base class streams and their own stream. class Base: scripts = ... # zip stream containing scripts A, B def install_scripts(self, stream): # ... def setup_scripts(self): self.install_scripts(self.scripts) class Derived: scripts = ... # zip stream containing modified script B, new script C def setup_scripts(self): self.install_scripts(Base.scripts) # adds A, B self.install_scripts(self.scripts) # adds C, overwrites B I'm not saying you couldn't do this with e.g. directory trees; it just seems neater to have the scripts in a black box once they're deployed, with a zip file representing that black box. Regards, Vinay Sajip
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/28/2011 05:10 PM, Chris McDonough wrote:
Why not modify sys.prefix? - --------------------------
As discussed above under `Backwards Compatibility`_, this PEP proposes to add ``sys.site_prefix`` as "the prefix relative to which site-package directories are found". This maintains compatibility with the documented meaning of ``sys.prefix`` (as the location relative to which the standard library can be found), but means that code assuming that site-packages directories are found relative to ``sys.prefix`` will not respect the virtual environment correctly.
Since it is unable to modify ``distutils``/``sysconfig``, `virtualenv`_ is forced to instead re-point ``sys.prefix`` at the virtual environment.
An argument could be made that this PEP should follow virtualenv's lead here (and introduce something like ``sys.base_prefix`` to point to the standard library and header files), since virtualenv already does this and it doesn't appear to have caused major problems with existing code.
Another argument in favor of this is that it would be preferable to err on the side of greater, rather than lesser, isolation. Changing ``sys.prefix`` to point to the virtual environment and introducing a new ``sys.base_prefix`` attribute would err on the side of greater isolation in the face of existing code's use of ``sys.prefix``.
It would seem to make sense to me to err on the side of greater isolation, introducing sys.base_prefix to indicate the base prefix (as opposed to sys.site_prefix indicating the venv prefix). Bugs introduced via a semi-isolated virtual environment are very difficult to troubleshoot. It would also make changes to existing code unnecessary. I have encountered no issues with virtualenv doing this so far.
I'm convinced that this is the better tradeoff. I'll begin working on a branch of the reference implementation that does things this way. Thanks for the feedback. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6uq2IACgkQ8W4rlRKtE2chHQCgik136LkoQ/JE6b3r4astWcog kYYAoN7ESaPlZOaYeok5t0i9hMkb2L4g =/Rn1 -----END PGP SIGNATURE-----
On Mon, 31 Oct 2011 07:50:24 +0000 (UTC)
Vinay Sajip
Antoine Pitrou
writes: I don't understand why a zip file makes this easier (especially the "update selectively" part).
Not a zip file specifically - just a binary stream which organises scripts to be installed. If each class in a hierarchy has access to a binary stream, then subclasses have access to the streams for base classes as well as their own stream, and can install selectively from base class streams and their own stream.
Isn't that overengineered? We're talking about a couple of files. It's not even obvious that third-party tools will want to modify them, instead of writing their own (if the venv API is stable, it should be relatively easy).
I'm not saying you couldn't do this with e.g. directory trees; it just seems neater to have the scripts in a black box once they're deployed, with a zip file representing that black box.
I don't know why it's neater. After all, we install .py files in their original form, not in a zipfile (even though Python supports the latter). Regards Antoine.
Antoine Pitrou
Isn't that overengineered? We're talking about a couple of files.
We're not talking about a lot of code to do this, either - just the interface to the existing code (which is needed anyway to install the minimal scripts in the venv).
It's not even obvious that third-party tools will want to modify them, instead of writing their own (if the venv API is stable, it should be relatively easy).
Well, virtualenvwrapper is pretty popular addon to virtualenv which delivers additional scripts, even though virtualenv already supplies more scripts than we're proposing to do in the stdlib. Example use cases for such scripts might be things like environment manipulation when environments are activated/deactivated (e.g. for LD_LIBRARY_PATH) - we can't always predict all the different needs that arise, so I'm just leaving the door open to third parties to be able to do what they need.
I don't know why it's neater. After all, we install .py files in their original form, not in a zipfile (even though Python supports the latter).
Perhaps it's a matter of taste. The files we're talking about are actually data in the context we're discussing. Regards, Vinay Sajip
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/30/2011 04:47 PM, Vinay Sajip wrote:
Antoine Pitrou
writes: It would be even simpler not to process it at all, but install the scripts as-is (without the execute bit) :) Sure, but such an approach makes it difficult to provide a mechanism which is easily extensible; for example, with the current approach, it is straightforward for third party tools to either easily replace completely, update selectively or augment simply the scripts provided by base classes.
I don't understand this point either. It seems to me too that having the scripts installed as plain data files inside a package is just as easy or easier for third-party tools to work with flexibly in all of the ways you mention, compared to having them available in any kind of zipped format. The current os.name-based directory structure can still be used, and we can still provide the helper to take such a directory structure and install the appropriate scripts based on os.name. I don't see any advantage to zipping. If done at install-time (which is necessary to make the scripts maintainable in the source tree) it also has the downside of introducing another difficulty in supporting source builds equivalently to installed builds. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6uvkYACgkQ8W4rlRKtE2ezZwCfUv80rp7Vg//zRA471R9JJDlj 83gAn0e9r76c9WkjutLcpbRjeopFkmew =Z0kj -----END PGP SIGNATURE-----
Carl Meyer
I don't see any advantage to zipping. If done at install-time (which is necessary to make the scripts maintainable in the source tree) it also has the downside of introducing another difficulty in supporting source builds equivalently to installed builds.
That's true, I hadn't thought of that. So then it sounds like the thing to do is make venv a package and have the code in venv/__init__.py, then have the scripts in a 'scripts' subdirectory below that. The API would then change to take the absolute pathname of the scripts directory to install from, right? Regards, Vinay Sajip
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/31/2011 09:35 AM, Vinay Sajip wrote:
That's true, I hadn't thought of that. So then it sounds like the thing to do is make venv a package and have the code in venv/__init__.py, then have the scripts in a 'scripts' subdirectory below that. The API would then change to take the absolute pathname of the scripts directory to install from, right?
That sounds right to me. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6uwXEACgkQ8W4rlRKtE2fUQgCfb1Cn7OYZzt3/xoKzmJuCmvbt B9sAn0kuBZzjVImIC1r8Jb506KbsRHBN =lgga -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/30/2011 06:28 AM, Antoine Pitrou wrote:
On Sun, 30 Oct 2011 12:10:18 +0000 (UTC) Vinay Sajip
wrote: We already have Unix shell scripts and BAT files in the source tree. Is it really complicated to maintain these additional shell scripts? Is there a lot of code in them?
No, they're pretty small: wc -l gives
76 posix/activate (Bash script, contains deactivate() function) 31 nt/activate.bat 17 nt/deactivate.bat
The question is whether we should stop at that, or whether there should be support for tcsh, fish etc. such as virtualenv provides.
I don't think we need additional support for more or less obscure shells. Also, if posix/activate is sufficiently well written (don't ask me how :-)), it should presumably be compatible with all Unix shells?
I have no problem including the basic posix/nt activate scripts if no one else is concerned about the added maintenance burden there. I'm not sure that my cross-shell-scripting fu is sufficient to write posix/activate in a cross-shell-compatible way; I use bash and am not very familiar with other shells. If it runs under /bin/sh is that sufficient to make it compatible with "all Unix shells" (for some definition of "all")? If so, I can work on this. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6uw80ACgkQ8W4rlRKtE2e0AACcCGqxp/HWxX0UAqtS9W5y+UDr weAAnjF4YdsCUvb4bXFloEGt1b7KlvWB =2bd+ -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/31/2011 11:50 AM, Carl Meyer wrote:
I have no problem including the basic posix/nt activate scripts if no one else is concerned about the added maintenance burden there.
I'm not sure that my cross-shell-scripting fu is sufficient to write posix/activate in a cross-shell-compatible way; I use bash and am not very familiar with other shells. If it runs under /bin/sh is that sufficient to make it compatible with "all Unix shells" (for some definition of "all")? If so, I can work on this.
I would say this is a perfect "opportunity to delegate," in this case to the devotees of other cults^Wshells than bash. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6ux/QACgkQ+gerLs4ltQ7j0wCffLICxbvo9ed0wMhEkn/iFzCj euEAnjvhPOAz09570Xh1PGBcksQ0De4n =YIG0 -----END PGP SIGNATURE-----
On 31 October 2011 16:08, Tres Seaver
On 10/31/2011 11:50 AM, Carl Meyer wrote:
I have no problem including the basic posix/nt activate scripts if no one else is concerned about the added maintenance burden there.
I'm not sure that my cross-shell-scripting fu is sufficient to write posix/activate in a cross-shell-compatible way; I use bash and am not very familiar with other shells. If it runs under /bin/sh is that sufficient to make it compatible with "all Unix shells" (for some definition of "all")? If so, I can work on this.
I would say this is a perfect "opportunity to delegate," in this case to the devotees of other cults^Wshells than bash.
For Windows, can you point me at the nt scripts? If they aren't too complex, I'd be willing to port to Powershell. Paul.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/31/2011 10:28 AM, Paul Moore wrote:
On 31 October 2011 16:08, Tres Seaver
wrote: On 10/31/2011 11:50 AM, Carl Meyer wrote:
I have no problem including the basic posix/nt activate scripts if no one else is concerned about the added maintenance burden there.
I'm not sure that my cross-shell-scripting fu is sufficient to write posix/activate in a cross-shell-compatible way; I use bash and am not very familiar with other shells. If it runs under /bin/sh is that sufficient to make it compatible with "all Unix shells" (for some definition of "all")? If so, I can work on this.
I would say this is a perfect "opportunity to delegate," in this case to the devotees of other cults^Wshells than bash.
Good call - we'll stick with what we've got until such devotees show up :-) Hey devotees, if you're listening, this is what you want to test/port: https://bitbucket.org/vinay.sajip/pythonv/src/6d057cfaaf53/Lib/venv/scripts/... For reference, here's what virtualenv ships with (includes a .fish and .csh script): https://github.com/pypa/virtualenv/tree/develop/virtualenv_support
For Windows, can you point me at the nt scripts? If they aren't too complex, I'd be willing to port to Powershell.
Thanks! They are here: https://bitbucket.org/vinay.sajip/pythonv/src/6d057cfaaf53/Lib/venv/scripts/... Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6vAKMACgkQ8W4rlRKtE2eEfwCgtpzQtUktUSU8ZyDDeqjD0yEe QXgAoLoCD8EQ74jHR1lWPFjgnwQFkM46 =6+Rn -----END PGP SIGNATURE-----
Carl Meyer writes:
On 31 October 2011 16:08, Tres Seaver
wrote:
I would say this is a perfect "opportunity to delegate," in this case to the devotees of other cults^Wshells than bash.
Good call - we'll stick with what we've got until such devotees show up :-)
That's fine, but either make sure it works with a POSIX-conformant /bin/sh, or make the shebang explicitly bash (bash is notoriously buggy in respect of being POSIX-compatible when named "sh").
Not a zip file specifically - just a binary stream which organises scripts to be installed. If each class in a hierarchy has access to a binary stream, then subclasses have access to the streams for base classes as well as their own stream, and can install selectively from base class streams and their own stream.
class Base: scripts = ... # zip stream containing scripts A, B
def install_scripts(self, stream): # ...
def setup_scripts(self): self.install_scripts(self.scripts)
class Derived: scripts = ... # zip stream containing modified script B, new script C
def setup_scripts(self): self.install_scripts(Base.scripts) # adds A, B self.install_scripts(self.scripts) # adds C, overwrites B
I'm not sure how many scripts you are talking about, and how long they are. Assuming there are free, and assuming they are short, I'd not make them separate source files again, but put them into string literals instead: scripts = { 'start':'''\ #!/bin/sh echo start ''', 'stop':'''\ #!/bin/sh echo stop ''' }}} Then, your install_scripts would take a dictionary filename:script contents. That's just as easily extensible. Regards, Martin
Martin v. Löwis
I'm not sure how many scripts you are talking about, and how long they are. Assuming there are free, and assuming they are short, I'd not make them separate source files again, but put them into string literals instead:
scripts = { 'start':'''\ #!/bin/sh echo start ''', 'stop':'''\ #!/bin/sh echo stop ''' }}}
Then, your install_scripts would take a dictionary filename:script contents. That's just as easily extensible.
True, but while the default scripts are not *too* long, third party scripts might be not amenable to this treatment. Plus, there can be binary executables in there too: at the moment, the pysetup3 script on Windows is shipped as a stub executable pysetup3.exe and a script pysetup3-script.py (since we can't rely on the PEP 397 launcher being available, this is the only way of being sure that the correct Python gets to run the script). I've changed the implementation now to use a directory tree, and the API takes the absolute pathname of the directory containing the scripts. Regards, Vinay Sajip
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/31/2011 09:57 PM, Stephen J. Turnbull wrote:
That's fine, but either make sure it works with a POSIX-conformant /bin/sh, or make the shebang explicitly bash (bash is notoriously buggy in respect of being POSIX-compatible when named "sh").
It has no shebang line, it must be sourced not run (its only purpose is to modify the current shell environment). Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6wEJYACgkQ8W4rlRKtE2dNGQCguHy8iYMgWIJyaQqABObt5ecv esIAnjmuHYH+G8JBGBzcwZzj8sofPinc =MR6D -----END PGP SIGNATURE-----
On 31 October 2011 20:10, Carl Meyer
For Windows, can you point me at the nt scripts? If they aren't too complex, I'd be willing to port to Powershell.
Thanks! They are here: https://bitbucket.org/vinay.sajip/pythonv/src/6d057cfaaf53/Lib/venv/scripts/...
The attached should work. Untested at the moment, I'm afraid, as I don't have access to a PC with the venv branch available. But they aren't complex, so they should be fine. Paul.
On 1 November 2011 16:29, Paul Moore
On 31 October 2011 20:10, Carl Meyer
wrote: For Windows, can you point me at the nt scripts? If they aren't too complex, I'd be willing to port to Powershell.
Thanks! They are here: https://bitbucket.org/vinay.sajip/pythonv/src/6d057cfaaf53/Lib/venv/scripts/...
The attached should work. Untested at the moment, I'm afraid, as I don't have access to a PC with the venv branch available. But they aren't complex, so they should be fine.
By the way, these do not need to be dot-sourced to activate/deactivate the venv, but they do need to be dot-sourced to enable the prompt change. As the prompt is more of a cosmetic thing, I'm not sure how crucial that is... Paul.
On 1 November 2011 16:40, Paul Moore
On 1 November 2011 16:29, Paul Moore
wrote: On 31 October 2011 20:10, Carl Meyer
wrote: For Windows, can you point me at the nt scripts? If they aren't too complex, I'd be willing to port to Powershell.
Thanks! They are here: https://bitbucket.org/vinay.sajip/pythonv/src/6d057cfaaf53/Lib/venv/scripts/...
The attached should work. Untested at the moment, I'm afraid, as I don't have access to a PC with the venv branch available. But they aren't complex, so they should be fine.
By the way, these do not need to be dot-sourced to activate/deactivate the venv, but they do need to be dot-sourced to enable the prompt change. As the prompt is more of a cosmetic thing, I'm not sure how crucial that is...
... and of course, to prove that anything untested is wrong, here's a minor fix to deactivate.ps1 :-) Paul.
For what its worth On 11/1/2011 11:43 AM, Paul Moore wrote:
On 1 November 2011 16:40, Paul Moore
wrote: On 1 November 2011 16:29, Paul Moore
wrote: On 31 October 2011 20:10, Carl Meyer
wrote: For Windows, can you point me at the nt scripts? If they aren't too complex, I'd be willing to port to Powershell.
For what its worth, there have been a number of efforts in this direction: https://bitbucket.org/guillermooo/virtualenvwrapper-powershell https://bitbucket.org/vanl/virtualenvwrapper-powershell (Both different implementations)
On Sat, Oct 29, 2011 at 4:37 AM, Carl Meyer
Why not modify sys.prefix? - --------------------------
As discussed above under `Backwards Compatibility`_, this PEP proposes to add ``sys.site_prefix`` as "the prefix relative to which site-package directories are found". This maintains compatibility with the documented meaning of ``sys.prefix`` (as the location relative to which the standard library can be found), but means that code assuming that site-packages directories are found relative to ``sys.prefix`` will not respect the virtual environment correctly.
Since it is unable to modify ``distutils``/``sysconfig``, `virtualenv`_ is forced to instead re-point ``sys.prefix`` at the virtual environment.
An argument could be made that this PEP should follow virtualenv's lead here (and introduce something like ``sys.base_prefix`` to point to the standard library and header files), since virtualenv already does this and it doesn't appear to have caused major problems with existing code.
Another argument in favor of this is that it would be preferable to err on the side of greater, rather than lesser, isolation. Changing ``sys.prefix`` to point to the virtual environment and introducing a new ``sys.base_prefix`` attribute would err on the side of greater isolation in the face of existing code's use of ``sys.prefix``.
I'm actually finding I quite like the virtualenv scheme of having "sys.prefix" refer to the virtual environment and "sys.real_prefix" refer to the interpeter's default environment. If pyvenv used the same naming scheme, then a lot of code designed to work with virtualenv would probably "just work" with pyvenv as well. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 11/08/2011 05:43 PM, Nick Coghlan wrote:
I'm actually finding I quite like the virtualenv scheme of having "sys.prefix" refer to the virtual environment and "sys.real_prefix" refer to the interpeter's default environment. If pyvenv used the same naming scheme, then a lot of code designed to work with virtualenv would probably "just work" with pyvenv as well.
Indeed. I've already been convinced (see my reply to Chris McDonough earlier) that this is the more practical approach. I've already updated my copy of the PEP on Bitbucket (https://bitbucket.org/carljm/python-peps/src/0936d8e00e5b/pep-0404.txt) to reflect this switch, working (slowly) on an update of the reference implementation. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk650A0ACgkQ8W4rlRKtE2cYuACgk5oRU54R+w4jHAynvW/QAxNU mQQAoI0zM4wzpPdOa0RIvEuAkUCmm+jT =RMyV -----END PGP SIGNATURE-----
participants (10)
-
"Martin v. Löwis"
-
Antoine Pitrou
-
Carl Meyer
-
Chris McDonough
-
Nick Coghlan
-
Paul Moore
-
Stephen J. Turnbull
-
Tres Seaver
-
VanL
-
Vinay Sajip