On 03/30/2011 07:18 PM, Vinay Sajip wrote:
So... it seems to me that we're likely to break _some_ third-party code using sys.prefix regardless of what we do here. My instinct says adding sys.virtual_prefix and leaving sys.prefix alone is the better approach, but I'm not very firmly entrenched in that position.
That's problematic, of course (third-party breakage, I mean). It may be that despite using virtual_prefix being a cleaner approach, it may be necessary to alter sys.prefix to ensure compatibility, a la virtualenv.
Well, as I said, I think there will be compatibility problems either way. We are splitting site-packages from the base Python installation, and when third-party code assumes that site-packages and the rest of the base Python installation are both found relative to sys.prefix, neither choice we make can possibly be right. Virtualenv tries to avoid this problem by copying or symlinking in enough stuff from the base Python install in order to convince third parties using sys.prefix that they really are looking at a full Python installation. But that approach is hacky and far from problem-free, and requires keeping up with various things that third-party code might try to look for under sys.prefix (there is at least one current open bug on virtualenv, Tkinter failing in a Windows virtualenv, that is the result of this problem). The logical conclusion of that approach, if you want full compatibility with third-party code, is to just copy the entire Python installation to the new location. So three options as I see it: 1) Add sys.virtual_prefix (or perhaps sys.site_prefix would be a better name), and require third-party code to use that (or sysconfig) to find site-packages, if it wants to be virtual-compatible. 2) Repoint sys.prefix, and require third-party code to understand some new sys.system_prefix or sys.installed_prefix or equivalent to find the rest of the base Python installation. 3) Attempt the virtualenv approach of trying to fool third-party code into thinking the virtualenv IS a full Python installation, even though it's not. I think #1 is far preferable to #2, and in the long run preferable to #3 even if it involves some short-term pain. But maybe I'm being too idealistic, and #3 is the pragmatic choice even though it's "wrong."
It may be that the thing isn't working properly, but it's masked on my system because I have an installed 3.3 in /usr/local (from when I was doing my pip/virtualenv tests). This means that there is a Makefile in a location which would not be there if I'd never installed 3.3 using sudo make install.
However, the getpath.c stuff should work - to test this, insert an "import pdb; pdb.set_trace()" in site.main, and check sys.path. This should contain entries from the virtualenv. The problems are more likely to be in the Python code: the C code always sets sys.prefix and sys.exec_prefix to /usr/local (or whatever configure was invoked with) and the sys.prefix / sys.exec_prefix stuff is done in site.virtualize, as before.
No, if I break early in site.py (or run python -S), sys.path doesn't contain any virtualenv paths.
Sorry - I got in a muddle because I've got too many clones around and ran into free disk space issues, and goofed somewhere. Don't bother to do a full merge for now, as I'm not sure if my code's working yet. Since I don't expect to change more than 4 files (getpath.c, site.py, distutils/sysconfig.py and sysconfig.py) I don't think we'll have too much trouble merging in due course, and if/when we can agree that my experiment in pythonv is worth taking further, I'll do the work of merging into cpythonv. For now, though, my changes might be treading on your toes, making merging needlessly annoying.
I'll try to test on a Python-clean machine in a day or two, will report back then.
Ok, thanks! Carl