pythonv: let's also make sure the standard Python install includes an "isolated" python
Whatever mechanism we end up with, I suggest that a standard python install include an isolated configuration. This is a common use case and should be available without having to create a virtualenv (or whatever) for each project or working directory.
I'm very happy to see this work taking place.
Jim
On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote:
Whatever mechanism we end up with, I suggest that a standard python install include an isolated configuration. This is a common use case and should be available without having to create a virtualenv (or whatever) for each project or working directory.
Could you elaborate on what this means? I don't quite understand what you mean by "include an isolated configuration".
I do think that Python should include a script to create the small amount of set up needed to trigger a (built-in) virtual environment. E.g. create the bin and lib/... directories, populate bin, and drop a minimal pythonv.conf file.
-Barry
On 03/17/2011 10:59 AM, Barry Warsaw wrote:
On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote:
Whatever mechanism we end up with, I suggest that a standard python install include an isolated configuration. This is a common use case and should be available without having to create a virtualenv (or whatever) for each project or working directory.
Could you elaborate on what this means? I don't quite understand what you mean by "include an isolated configuration".
I'm also not entirely clear what this means, but I think perhaps "python -S" already covers it? That will start up a python interpreter without importing site.py, so it will have no site-packages at all; nothing but the stdlib. Of course, then you'd have to take care of fixing up sys.path yourself to include your project and its dependencies: this may be reasonable for buildout.
I do think that Python should include a script to create the small amount of set up needed to trigger a (built-in) virtual environment. E.g. create the bin and lib/... directories, populate bin, and drop a minimal pythonv.conf file.
Definitely. It looks like this'll end up being the bulk of the added code; I'm planning to do it (borrowing liberally from virtualenv.py) as soon as I'm confident of the basic approach.
Any opinions on the commandline UI for this? I was thinking of just adding a "pythonv.py" to the stdlib that you could execute with "python -m pythonv path/to/new/env" (and would also export appropriate API to create environments programmatically).
Carl
On Thu, Mar 17, 2011 at 12:44 PM, Carl Meyer carl@oddbird.net wrote:
On 03/17/2011 10:59 AM, Barry Warsaw wrote:
On Mar 17, 2011, at 08:07 AM, Jim Fulton wrote:
Whatever mechanism we end up with, I suggest that a standard python install include an isolated configuration. This is a common use case and should be available without having to create a virtualenv (or whatever) for each project or working directory.
Could you elaborate on what this means? I don't quite understand what you mean by "include an isolated configuration".
As I mentioned in an earlier thread, I want to get complete isolation from local additions relative to the standard Python distribution. I want python executable I can use to bootstrap a buildout that doesn't include site packages or it's equivalent.
My understanding of how this will work was that I could created this myself by creating some sort of configuration file, say clean.cfg and then link a Python executable to the name "clean". Reading "pythonv, take two" more carefully, I see that it is a lot more virtualenv-oriented than I'd hoped. <shrug>
I'm also not entirely clear what this means, but I think perhaps "python -S" already covers it?
I thought so too, but people keep telling me that's not enough. One issue was that maybe site.py might be imported accidentally, for example to get at one of the helper functions it contains. I don't know if this is a significant danger.
I'd like to have some defined way to express my need for isolation that's more rubust that simply not importing site.py.
That will start up a python interpreter without importing site.py, so it will have no site-packages at all; nothing but the stdlib. Of course, then you'd have to take care of fixing up sys.path yourself to include your project and its dependencies: this may be reasonable for buildout.
Yup.
Jim
-- Jim Fulton http://www.linkedin.com/in/jimfulton
On 03/17/2011 01:23 PM, Jim Fulton wrote:
My understanding of how this will work was that I could created this myself by creating some sort of configuration file, say clean.cfg and then link a Python executable to the name "clean". Reading "pythonv, take two" more carefully, I see that it is a lot more virtualenv-oriented than I'd hoped. <shrug>
I'm also not entirely clear what this means, but I think perhaps "python -S" already covers it?
I thought so too, but people keep telling me that's not enough. One issue was that maybe site.py might be imported accidentally, for example to get at one of the helper functions it contains. I don't know if this is a significant danger.
I'd like to have some defined way to express my need for isolation that's more rubust that simply not importing site.py.
Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up, and you simply _don't_ create any lib/python3.3/site-packages relative to pythonv.conf, what you'll end up with is identical to "python -S" but robust against import of site.py (it will already have been imported, but it won't have found any site-packages directories).
If the possibility of someone accidentally creating lib/python3.3/site-packages is an issue, we could easily add a "no-site=True" option to pythonv.conf that would prevent it from even checking for the existence of that directory.
And in either case, we could add a --no-site option to "python -m pythonv" that would create a virtualenv without the site-packages directory (and with no-site=True, if we decide that's worth having).
Carl
On Thu, Mar 17, 2011 at 1:41 PM, Carl Meyer carl@oddbird.net wrote:
Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up, and you simply _don't_ create any lib/python3.3/site-packages relative to pythonv.conf, what you'll end up with is identical to "python -S" but robust against import of site.py (it will already have been imported, but it won't have found any site-packages directories).
Cool.
If the possibility of someone accidentally creating lib/python3.3/site-packages is an issue, we could easily add a "no-site=True" option to pythonv.conf that would prevent it from even checking for the existence of that directory.
I don't think this is an issue.
And in either case, we could add a --no-site option to "python -m pythonv" that would create a virtualenv without the site-packages directory (and with no-site=True, if we decide that's worth having).
Cool (for virtualenv).
It occurs to me that it would be nice if site.py could grow knowledge of whether -S was used and not automatically mutate the path if -S was used. That would allow -S to work robustly without having to link anything or create a config file.
Jim
-- Jim Fulton http://www.linkedin.com/in/jimfulton
On 03/17/2011 01:53 PM, Jim Fulton wrote:
It occurs to me that it would be nice if site.py could grow knowledge of whether -S was used and not automatically mutate the path if -S was used. That would allow -S to work robustly without having to link anything or create a config file.
This seems entirely sensible to me (and quite easy to implement). I can't imagine why anyone would be relying on being able to do "python -S" and later import site and get the path modifications automatically, so it seems pretty much like a straightforward bugfix. And the fix is a one-liner. Issue filed with patch: http://bugs.python.org/issue11591
Carl
On Thu, Mar 17, 2011 at 3:46 PM, Carl Meyer carl@oddbird.net wrote:
On 03/17/2011 01:53 PM, Jim Fulton wrote:
It occurs to me that it would be nice if site.py could grow knowledge of whether -S was used and not automatically mutate the path if -S was used. That would allow -S to work robustly without having to link anything or create a config file.
This seems entirely sensible to me (and quite easy to implement). I can't imagine why anyone would be relying on being able to do "python -S" and later import site and get the path modifications automatically, so it seems pretty much like a straightforward bugfix. And the fix is a one-liner. Issue filed with patch: http://bugs.python.org/issue11591
Thanks!
Jim
At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote:
Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up,
Is there any reason why the configuration file has to be one directory up, intead of adjacent to the executable?
On 03/17/2011 03:00 PM, P.J. Eby wrote:
At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote:
Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up,
Is there any reason why the configuration file has to be one directory up, intead of adjacent to the executable?
Not a particularly good one. I inherited this from the standard layout of virtualenv, which includes a bin/ directory for the python binary and scripts, and it made more sense to me in that layout for pythonv.conf to live at the root of the "virtualenv" rather than in bin/.
I'd like to continue to support it being one level up, but I can't think of any reason not to first check adjacent, and then one level up. I suppose it could just keep checking up the tree, but that introduces a bunch more filesystem checks and could give surprising results. I won't do that unless someone steps up to give convincing arguments for it.
Carl
On Thu, Mar 17, 2011 at 3:04 PM, Carl Meyer carl@oddbird.net wrote:
On 03/17/2011 03:00 PM, P.J. Eby wrote:
At 01:41 PM 3/17/2011 -0400, Carl Meyer wrote:
Actually, now that I come to think of it, pythonv (take two) does already cover your requirement. If you have a symlinked or copied python binary, and an empty pythonv.conf one directory up,
Is there any reason why the configuration file has to be one directory up, intead of adjacent to the executable?
Not a particularly good one. I inherited this from the standard layout of virtualenv, which includes a bin/ directory for the python binary and scripts, and it made more sense to me in that layout for pythonv.conf to live at the root of the "virtualenv" rather than in bin/.
I'd like to continue to support it being one level up, but I can't think of any reason not to first check adjacent, and then one level up. I suppose it could just keep checking up the tree, but that introduces a bunch more filesystem checks and could give surprising results. I won't do that unless someone steps up to give convincing arguments for it.
I suggest the following:
Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg.
Jim
On 03/17/2011 05:13 PM, Jim Fulton wrote:
I suggest the following:
Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg.
Nice - I like the ability to have multiple interpreters side-by-side with different pythonv configurations.
Is ".cfg" generally preferred to ".conf" for some good reason? I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-)
And I kind of dislike the inconsistency in extension; would "clean.pythonv.cfg" be acceptable?
To simplify documentation and allow more flexibility, I might just check for all four: first the executable-specific one in both directories, then the general one in both directories.
Carl
On Mar 17, 2011, at 05:33 PM, Carl Meyer wrote:
On 03/17/2011 05:13 PM, Jim Fulton wrote:
I suggest the following:
Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg.
Nice - I like the ability to have multiple interpreters side-by-side with different pythonv configurations.
Indeed. In that case though, wouldn't ./bin/clean.cfg be fine? Even if the executable were named ./bin/python a sibling ./bin/python.cfg would be fine too I think. I agree with Carl that I dislike the inconsistency in the extension, but I think argv[0] + '.cfg' would be fine.
Having a non-.cfg or -.conf extension makes it harder to do stuff like automatically set the major mode in Emacs (although I guess it wouldn't be that hard to add a mapping for *.pythonv).
Is ".cfg" generally preferred to ".conf" for some good reason? I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-)
I guess I'm on the .cfg side, but wouldn't care too much if .conf is the consensus.
And I kind of dislike the inconsistency in extension; would "clean.pythonv.cfg" be acceptable?
I'm not sure you need the .pythonv. part.
To simplify documentation and allow more flexibility, I might just check for all four: first the executable-specific one in both directories, then the general one in both directories.
Definitely add a --verbose option to print out exactly which one got chosen. :)
-Barry
On Thu, Mar 17, 2011 at 5:33 PM, Carl Meyer carl@oddbird.net wrote:
On 03/17/2011 05:13 PM, Jim Fulton wrote:
I suggest the following:
Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg.
Nice - I like the ability to have multiple interpreters side-by-side with different pythonv configurations.
Is ".cfg" generally preferred to ".conf" for some good reason? I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-)
And I kind of dislike the inconsistency in extension; would "clean.pythonv.cfg" be acceptable?
To simplify documentation and allow more flexibility, I might just check for all four: first the executable-specific one in both directories, then the general one in both directories.
Too many variants will get annoying. Along those lines, if it can be in multiple places please make having more than one an error.
On Thu, Mar 17, 2011 at 5:33 PM, Carl Meyer carl@oddbird.net wrote:
Is ".cfg" generally preferred to ".conf" for some good reason? I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-)
That all depends on who you ask; I tend to prefer .conf myself (but only by a hair). Jim likely has his own opinion.
And I kind of dislike the inconsistency in extension; would "clean.pythonv.cfg" be acceptable?
I'd accept that, so long as we pick only one of .conf or .cfg.
-Fred
-- Fred L. Drake, Jr. <fdrake at acm.org> "A storm broke loose in my mind." --Albert Einstein
On Thu, Mar 17, 2011 at 5:33 PM, Carl Meyer carl@oddbird.net wrote:
On 03/17/2011 05:13 PM, Jim Fulton wrote:
I suggest the following:
Look for argv[0]+'.pythonv' and then for '../pythonv.cfg'.
So if I've linked the Python executable to ./bin/clean, look for ./bin/clean.pythonv and ./pythonv.cfg.
Nice - I like the ability to have multiple interpreters side-by-side with different pythonv configurations.
Is ".cfg" generally preferred to ".conf" for some good reason?
I dunno.
I don't personally care too much; the former is shorter but the latter looks less ugly to me ;-)
I don't care either.
And I kind of dislike the inconsistency in extension; would "clean.pythonv.cfg" be acceptable?
Anything's acceptable. :) The main idea is to base the config file name on the executable name.
To simplify documentation and allow more flexibility, I might just check for all four: first the executable-specific one in both directories, then the general one in both directories.
<shrug> I don't think that's necessary.
Jim
On Sat, Mar 19, 2011 at 8:54 AM, Éric Araujo merwok@netwok.org wrote:
it’s the form used by modules in the standard library: distutils, idle, unittest in a near future.
Which is a good reason to continue using the less-readable .cfg extension.
(One exception is .pypirc.)
Traditional dotfiles don't have extensions, and Greg was definitely a traditionalist.
-Fred
-- Fred L. Drake, Jr. <fdrake at acm.org> "A storm broke loose in my mind." --Albert Einstein
On Sat, Mar 19, 2011 at 11:20 AM, Éric Araujo merwok@netwok.org wrote:
Except that Greg’s code used .pydistutils.cfg and pydistutils.cfg; .pypirc was a later addition by amk (PEP 301).
--sigh--
Getting up on Saturday is always a mistake.
whippersnapperly yours
Yes, indeed.
-Fred
-- Fred L. Drake, Jr. <fdrake at acm.org> "A storm broke loose in my mind." --Albert Einstein
On Mar 17, 2011, at 12:44 PM, Carl Meyer wrote:
Any opinions on the commandline UI for this? I was thinking of just adding a "pythonv.py" to the stdlib that you could execute with "python -m pythonv path/to/new/env" (and would also export appropriate API to create environments programmatically).
Bikeshedding time: how about something a little more descriptive?
$ python -m virtualize path/to/new/env
-Barry
participants (7)
-
Barry Warsaw -
Benji York -
Carl Meyer -
Fred Drake -
Jim Fulton -
P.J. Eby -
Éric Araujo