Re: [Distutils] virtual env with exec prefix
Hi Ian, recently I reported this problem to you:
[...] here at our university, python (as well as hundreds of other software packages) is installed in a shared NFS tree with a clear separation of --prefix and --exec-prefix, i.e. all platform-specific stuff goes into the according subdirectories. [...]
[2] meine@kogspc33:~ -> virtualenv enthought-inst New python executable in enthought-inst/bin/python Installing setuptools.... Complete output from command enthought-inst/bin/python -c "#!python \"\"\"Bootstrap setuptoo...
" /software/python-2.4.4/SuSE-10...4.egg: error: invalid Python installation: unable to open /software/python-2.4.4/lib/python2.4/config/Makefile (No such file or directory)
[The message] is right, because it should be /software/python-2.4.4/SuSE-10.2/lib/python2.4/config/Makefile which exists (in the SuSE-10.2 directory).
I have now debugged this a little bit (using virtualenv from SVN), and it seems to be a problem with the virtualenv'd python and setuptools. I have written the EZ_SETUP_PY to disk, and when I mimick virtualenv's command for installing setuptools, I get the error: [2] meine@kogspc33:~/Programming/enthought -> inst/bin/python ../ez_setup.py -v /home/meine/Programming/virtualenv-svn/support-files/setuptools-0.6c8-py2.4.egg error: invalid Python installation: unable to open /software/python-2.4.4/lib/python2.4/config/Makefile (No such file or directory) (NB: inst is the virtual env) OTOH, when I run this with the python binary from its original location, it works fine. Since the error comes from deep within setuptools (AFAICS), should this be discussed on distutils-sig? Ciao, / / /--/ / / ANS
Am Montag, 14. April 2008 18:32:39 schrieb Hans Meine:
error: invalid Python installation: unable to open /software/python-2.4.4/lib/python2.4/config/Makefile (No such file or directory)
I have found and fixed this bug; it's because virtualenv overwrites distutils.sysconfig.get_python_lib and does not respect the 'plat_specific' parameter properly. I have modified virtualenv by adding sys.real_exec_prefix along sys.real_prefix and letting orig-prefix.txt be a file with two lines. Now, I can use virtualenv here, too. :-) @distutils-sig: Sorry for the noise, it turned out to be virtualenv's fault. -- Ciao, / / /--/ / / ANS
Hans Meine wrote:
Hi Ian,
recently I reported this problem to you:
[...] here at our university, python (as well as hundreds of other software packages) is installed in a shared NFS tree with a clear separation of --prefix and --exec-prefix, i.e. all platform-specific stuff goes into the according subdirectories. [...]
[2] meine@kogspc33:~ -> virtualenv enthought-inst New python executable in enthought-inst/bin/python Installing setuptools.... Complete output from command enthought-inst/bin/python -c "#!python \"\"\"Bootstrap setuptoo...
" /software/python-2.4.4/SuSE-10...4.egg: error: invalid Python installation: unable to open /software/python-2.4.4/lib/python2.4/config/Makefile (No such file or directory)
[The message] is right, because it should be /software/python-2.4.4/SuSE-10.2/lib/python2.4/config/Makefile which exists (in the SuSE-10.2 directory).
I have now debugged this a little bit (using virtualenv from SVN), and it seems to be a problem with the virtualenv'd python and setuptools. I have written the EZ_SETUP_PY to disk, and when I mimick virtualenv's command for installing setuptools, I get the error:
[2] meine@kogspc33:~/Programming/enthought -> inst/bin/python ../ez_setup.py -v /home/meine/Programming/virtualenv-svn/support-files/setuptools-0.6c8-py2.4.egg error: invalid Python installation: unable to open /software/python-2.4.4/lib/python2.4/config/Makefile (No such file or directory)
(NB: inst is the virtual env) OTOH, when I run this with the python binary from its original location, it works fine. Since the error comes from deep within setuptools (AFAICS), should this be discussed on distutils-sig?
Hmm... well, does this error only happen when you create a virtualenv from within a virtualenv? Otherwise, I think there's several things in distutils.sysconfig that need to be adjusted. That module isn't really documented, however, and I suspect there's just a bunch of things that will have to be added to over time, not a good general fix. Or... maybe there's some clever way to fix all these, but I'm not sure. -- Ian Bicking : ianb@colorstudy.com : http://blog.ianbicking.org
Hi Ian! Am Montag, 14. April 2008 20:37:10 schrieb Ian Bicking:
Hmm... well, does this error only happen when you create a virtualenv from within a virtualenv?
No, absolutely not. It arises whenever python is installed with --exec-prefix != --prefix.
Otherwise, I think there's several things in distutils.sysconfig that need to be adjusted. That module isn't really documented, however, and I suspect there's just a bunch of things that will have to be added to over time, not a good general fix. Or... maybe there's some clever way to fix all these, but I'm not sure.
AFAICS, my patch is quite clean - it just extends virtualenv slightly by handling exec_prefix and prefix separately. This affects the same entrypoints as before (get_python_inc / get_python_lib) and thus promotes to the same changes in higher level functions. Here's how I tested it: import distutils.sysconfig, os for p in range(2): for s in range(2): print distutils.sysconfig.get_python_lib(p, s) print "\nMakefile:", distutils.sysconfig.get_makefile_filename() print " exists:", os.path.exists(distutils.sysconfig.get_makefile_filename()) You can see the results before/after my virtualenv change here: *** Without virtualenv: *** [2] meine@kogspc33:~ -> python test_python_inst.py /software/python-2.4.4/lib/python2.4/site-packages /software/python-2.4.4/lib/python2.4 /software/python-2.4.4/SuSE-10.2/lib/python2.4/site-packages /software/python-2.4.4/SuSE-10.2/lib/python2.4 Makefile: /software/python-2.4.4/SuSE-10.2/lib/python2.4/config/Makefile exists: True *** Trying to create a virtualenv: *** [2] meine@kogspc33:~ -> virtualenv tmp/ve1 New python executable in tmp/ve1/bin/python Installing setuptools.... Complete output from command tmp/ve1/bin/python -c "#!python \"\"\"Bootstrap setuptoo... " /software/python-2.4.4/lib/pyt...4.egg: error: invalid Python installation: unable to open /software/python-2.4.4/lib/python2.4/config/Makefile (No such file or directory) ---------------------------------------- ...Installing setuptools...done. Traceback (most recent call last): File "/software/python-2.4.4/bin/virtualenv", line 7, in ? sys.exit( File "/software/python-2.4.4/lib/python2.4/site-packages/virtualenv-1.0-py2.4.egg/virtualenv.py", line 337, in main create_environment(home_dir, site_packages=not options.no_site_packages, clear=options.clear) File "/software/python-2.4.4/lib/python2.4/site-packages/virtualenv-1.0-py2.4.egg/virtualenv.py", line 536, in create_environment install_setuptools(py_executable) File "/software/python-2.4.4/lib/python2.4/site-packages/virtualenv-1.0-py2.4.egg/virtualenv.py", line 260, in install_setuptools extra_env=env) File "/software/python-2.4.4/lib/python2.4/site-packages/virtualenv-1.0-py2.4.egg/virtualenv.py", line 396, in call_subprocess raise OSError( OSError: Command tmp/ve1/bin/python -c "#!python \"\"\"Bootstrap setuptoo... " /software/python-2.4.4/lib/pyt...4.egg failed with error code 1 *** Unpatched virtualenv: *** As illustrated above, the installation of setuptools failed, but one can use the already-installed python interpreter to run my test program and gets: [2] meine@kogspc33:~ -> tmp/ve1/bin/python test_python_inst.py /home/meine/tmp/ve1/lib/python2.4/site-packages /software/python-2.4.4/lib/python2.4 /home/meine/tmp/ve1/lib/python2.4/site-packages /software/python-2.4.4/lib/python2.4 Makefile: /software/python-2.4.4/lib/python2.4/config/Makefile exists: False This explains why the setuptools installation failed. *** Patched virtualenv with real_exec_prefix *** [2] meine@kogspc33:~ -> . tmp/ve2/bin/activate (ve2)[2] meine@kogspc33:~ -> python test_python_inst.py /home/meine/tmp/ve2/lib/python2.4/site-packages /software/python-2.4.4/lib/python2.4 /home/meine/tmp/ve2/lib/python2.4/site-packages /software/python-2.4.4/SuSE-10.2/lib/python2.4 Makefile: /software/python-2.4.4/SuSE-10.2/lib/python2.4/config/Makefile exists: True I hope I could convince you to include this crucial fix in virtualenv. -- Ciao, / / /--/ / / ANS
participants (2)
-
Hans Meine -
Ian Bicking