reservations about pythonv
I suspect that the current "pythonv" proposal should not become how virtualenv is implemented in the future. The proposal would add another exec() to every invocation of Python in the virtualenv. Not only is exec() among the most costly system calls on Unix systems, but it is even slower on Windows. Furthermore, the re-exec() will generate significant noise and annoyance - if not outright breakage - when people use binary tools like gdb and valgrind, because MYENV/bin/python will become a binary that looks nothing like the Python binary (none of the usual symbols, binary dependencies, and so forth) and that disappears almost immediately after being run and is replaced with a completely different binary. Do all popular binary debugging tools handle an exec() cleanly? At the very least everyone's debugging practices and scripts would have to be re- worked to ignore the stats and results that came back from that first ephemeral "pythonv" process image. I certainly agree with everyone that Python 3.3 should natively support a virtualenv mechanism, but I think that it should be the full "python" that gets copied. I realize that this means that the "python" binary will not get updates when the system Python does; but isn't the whole idea of a virtualenv that it insulates me from change in a way that using the system Python does not? I would be quite happy for either the "activate" script (or Python itself?) to issue a warning if the binary in the virtualenv has fallen behind the image in the system Python. -- Brandon Craig Rhodes brandon@rhodesmill.org http://rhodesmill.org/brandon
On Mar 16, 2011, at 12:21 PM, Brandon Craig Rhodes wrote:
I certainly agree with everyone that Python 3.3 should natively support a virtualenv mechanism, but I think that it should be the full "python" that gets copied. I realize that this means that the "python" binary will not get updates when the system Python does; but isn't the whole idea of a virtualenv that it insulates me from change in a way that using the system Python does not? I would be quite happy for either the "activate" script (or Python itself?) to issue a warning if the binary in the virtualenv has fallen behind the image in the system Python.
I agree that a separate pythonv binary that execs the real Python binary is suboptimal. I understand that to set environment variables such as $LD_LIBRARY_PATH (on *nix), it might be inevitable that a re-exec is necessary, but if so, I think it should be the python binary itself that does it. Here at the sprints we talked about several possible options, the details of which of course will have to be hashed out. There will also be cross platform considerations. I think on *nix at least, it would be nice if a symlink and configuration file were enough to trigger the virtualenv behavior. For example, if I have a local 'python' symlinked to the real executable, with a pythonv.conf file next to it, the virtual environment would be enabled. The real Python binary would adjust its behavior in that case to know where the standard library was, but also use the locally installed packages. Anyway, that's how I'd *like* it to work on *nix, but it may have to work differently on Windows, and it may have to work differently if environment variables have to be set. -Barry
Barry Warsaw <barry <at> python.org> writes:
Here at the sprints we talked about several possible options, the details of which of course will have to be hashed out. There will also be cross platform considerations. I think on *nix at least, it would be nice if a symlink and configuration file were enough to trigger the virtualenv behavior.
I feel that this approach definitely has potential to be the most useful and practical. There's no reason for a separate executable if Python itself incorporates the virtual environment functionality, which is mostly about setting paths and environment variables.
For example, if I have a local 'python' symlinked to the real executable, with a pythonv.conf file next to it, the virtual environment would be enabled. The real Python binary would adjust its behavior in that case to know where the standard library was, but also use the locally installed packages. Anyway, that's how I'd *like* it to work on *nix, but it may have to work differently on Windows, and it may have to work differently if environment variables have to be set.
Even if on Windows you have to copy rather than symlink, that could just be the main Python executable, which is not prohibitively large since the core of Python is in pythonX.Y.dll. Regards, Vinay Sajip
participants (3)
-
Barry Warsaw -
Brandon Craig Rhodes -
Vinay Sajip