Hi Vinay, On 03/29/2011 07:00 PM, Vinay Sajip wrote:
If we switch to sys.virtual_prefix and leave sys.prefix alone, the danger would be that some installer that's using sys.prefix directly, and ignoring sysconfig.py, would try to install stuff to the global Python installation. But I'm not aware of any such installer, and I think we could safely call any installer of Python code that ignores the layout schemes in sysconfig thoroughly broken anyway.
Your thoughts? (Or anyone else?)
What about the possibility of third-party code which uses sys.prefix/sys.exec_prefix, though? (I mean apart from the installer code.)
Right. The question is, for third-party code using sys.prefix/sys.exec_prefix, which value would be the right one, the installation prefix or the virtual prefix? It depends on what they're using it for. I did a quick Google Code Search for "sys.prefix," and unfortunately right on the front page I can find examples where either approach we take would be "wrong." The majority of uses I found are people copying code from trace.py which is intended to exclude the standard library and system-wide packages from coverage tracing. In this case, leaving sys.prefix pointing to the installation prefix would give the desired result. Also, for instance, here where matplotlib's setup looks for DLLs on Windows, the system prefix would be correct: http://bit.ly/eQSVYc On the other hand, here scons uses sys.prefix to try to find site-packages: http://bit.ly/ik0PwJ If we leave sys.prefix pointing to the installation directory, this would leave scons unable to find its config file in the virtualenv's site-packages directory. Unless they fix their code to use the get_python_lib() function in sysconfig, which is what they ought to be doing instead of duplicating all that logic. So... it seems to me that we're likely to break _some_ third-party code using sys.prefix regardless of what we do here. My instinct says adding sys.virtual_prefix and leaving sys.prefix alone is the better approach, but I'm not very firmly entrenched in that position.
On the question of a C-readable configuration file, I've started work on this on the assumption that it's a UTF8-encoded file which contains lines of the form key = value. Having thought about it, since this file is only for bootstrapping the virtualenv anyway, there's no strong case for it to be ConfigParser-compatible.
Agreed. I only used ConfigParser initially because that was easiest from Python code :-)
I've modified calculate_path in getpath.c to look for the file adjacent to the executable or one level up. If found, we look for a line "home = /path/to/source/of/copied/python" and, if found, replace argv0_path with the value, then let the existing code take its course. I've modified your virtualize() in site.py to cater for the new config file format: we default the prefix to the parent of the directory containing the executable (hence, the virtualenv root), then set sys.virtual_prefix to this prefix, and optionally disallow the system site-packages from the system path. For now (since [distutils.]sysconfig is not sys.virtual_prefix aware) I continue to set sys.prefix and sys.exec_prefix to sys.virtual_prefix.
This sounds good.
So as not to cause confusion with existing config files you might have, I changed the config file name to 'env.cfg'. This can be changed to the appropriate bikeshed colour in two places - site.py and getpath.c.
Progress seems good. To test:
1. Clone the repo at https://bitbucket.org/vinay.sajip/pythonv 2. In that repo, run ./configure and make 3. Download the "pmv.py" script at https://gist.github.com/893507 4. Run pmv.py with the python you just built - see the header comment in that script.
Note that the pmv.py (Poor Man's virtualize.py) script deliberately runs with a copied rather than a symlinked Python.
Hmm, it doesn't seem to be working for me. sys.prefix and sys.exec_prefix are still /usr/local using the python binary in my "pmv", and sys.virtual_prefix is not set at all, despite env.cfg being present and looking as it should. Oddly when I run the compiled python binary directly from the checkout, it does set sys.virtual_prefix to point one level above the checkout, despite there being no env.cfg in the vicinity at all. You can see both of these things (in reverse order) here: http://paste.pocoo.org/show/362798/ I'll dig in a bit more and see if I can figure out what's happening. Is there a reason you didn't base your changesets on mine (in the cpythonv repo), and instead apparently copied over a bunch of the changes manually as a diff? I can transplant or copy your work back, or try to do a full merge, but in the long run if we'll both be working on this it seems best if we are easily able to merge our work back and forth. Carl