Hello all, Here at PyCon we've had some discussion about building a virtualenv-alike into Python core for Python 3.3. The goal is to improve on virtualenv by providing something that does what virtualenv does without requiring a copied Python binary, symlinked/copied parts of the standard library, or a forked site.py. (I'm not entirely sure that distutils-sig is the right venue for discussing this, but it's the closest I know of and was recommended by others in our conversations here; if there's a better place please let me know; maybe python-ideas? The patch as is stands does affect sysconfig.py, which is used more by distutils than anything else.) The idea we discussed is to add to Python's built-in site.py the ability to set paths up for a virtual environment, triggered by certain environment variables. Then, for convenience, there'll be a small executable which can be placed in the "bin/" directory of a virtual environment and knows how to set up these environment variables and then exec() the system Python binary. Larry Hastings had already created this wrapper executable at last year's PyCon, and tonight I made the necessary modifications to site.py and sysconfig.py to support it on the Python side. The early prototype is now working well (at least on Linux; I think it ought to work on OS X, and should work partially on Windows as well), and I'd welcome review and comment: https://bitbucket.org/carljm/cpythonv Look in Tools/pythonv/README.rst for instructions. This is an early prototype and will certainly require refinement (not to mention most likely a PEP, at some point). Please try it out and let me know if it works for you! Thanks, Carl
I forgot to mention - the code for this is in the "pythonv" named branch in the mercurial repo I linked (https://bitbucket.org/carljm/cpythonv), so you'll need to "hg update pythonv" after cloning to see it. Carl On 03/16/2011 04:31 AM, Carl Meyer wrote:
This is an early prototype and will certainly require refinement (not to mention most likely a PEP, at some point). Please try it out and let me know if it works for you!
On Wed, Mar 16, 2011 at 4:31 AM, Carl Meyer <carl@oddbird.net> wrote:
Hello all,
Here at PyCon we've had some discussion about building a virtualenv-alike into Python core for Python 3.3. The goal is to improve on virtualenv by providing something that does what virtualenv does without requiring a copied Python binary, symlinked/copied parts of the standard library, or a forked site.py.
(I'm not entirely sure that distutils-sig is the right venue for discussing this, but it's the closest I know of and was recommended by others in our conversations here; if there's a better place please let me know; maybe python-ideas? The patch as is stands does affect sysconfig.py, which is used more by distutils than anything else.)
The idea we discussed is to add to Python's built-in site.py the ability to set paths up for a virtual environment, triggered by certain environment variables. Then, for convenience, there'll be a small executable which can be placed in the "bin/" directory of a virtual environment and knows how to set up these environment variables and then exec() the system Python binary.
Larry Hastings had already created this wrapper executable at last year's PyCon, and tonight I made the necessary modifications to site.py and sysconfig.py to support it on the Python side. The early prototype is now working well (at least on Linux; I think it ought to work on OS X, and should work partially on Windows as well), and I'd welcome review and comment: https://bitbucket.org/carljm/cpythonv Look in Tools/pythonv/README.rst for instructions.
This is an early prototype and will certainly require refinement (not to mention most likely a PEP, at some point). Please try it out and let me know if it works for you!
Carl, Thanks for posting this. I'm very hopeful that buildout can use this same mechanism to get the isolation it needs. I would really appreciate it if buildout users who care about this would test it with buildout. In particular, I know of 2 basic use cases: - Get complete isolation from local additions relative to the standard Python distribution. - Have the ability to cherry pick some local additions while having isolation from the rest. (Gary and Ubuntu, I'm looking at you.) Please, let's make sure this mechanism is enough. To raise the stakes a bit, when this mechanism is available, presumably in Python 3.3, I plan to use it exclusively to provide isolation in buildout. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton
Jim, On 03/16/2011 08:00 AM, Jim Fulton wrote: [snip]
buildout. In particular, I know of 2 basic use cases:
- Get complete isolation from local additions relative to the standard Python distribution.
This is the primary design goal, and I believe it's currently achieved. To be clearer, we have isolation from the system Python's site-packages and site-python (site directories added to sys.path in site.py), where all non-stdlib modules should go. We are not isolated from any changes to the system Python's standard library (which should never be changed anyway); this is intentional and matches the current behavior of virtualenv, and avoids the necessity of copying the entire stdlib.
- Have the ability to cherry pick some local additions while having isolation from the rest. (Gary and Ubuntu, I'm looking at you.)
We don't currently have this. There's a question whether this behavior should be built in to pythonv, or provided by a tool like distutils2 or pip or buildout that could be virtual-aware. The latter seems like perhaps a more natural fit, since it's a persistent modification of the virtualenv's working set, parallel to installing or uninstalling packages within the virtualenv. I'm open to further thoughts on this question. Carl
Carl Meyer <carl <at> oddbird.net> writes:
This is an early prototype and will certainly require refinement (not to mention most likely a PEP, at some point). Please try it out and let me know if it works for you!
I haven't tried it yet, but I'm curious: was an approach using PYTHONHOME considered? Or is PYTHONHOME off-limits for an application like this? Regards, Vinay Sajip
participants (3)
-
Carl Meyer -
Jim Fulton -
Vinay Sajip