On Thu, Feb 28, 2013 at 3:26 PM, Paul Moore wrote:

For some scripts I am writing, I want to be able to run a package's setup.py using subprocess.check_call(). However, the python installation I'm using may not have setuptools/distribute installed, and yet I want to correctly process setup.py files which rely on setuptools. Is there any way of setting things up so that the Python subprocess can use a local, non-installed, copy of setuptools or distribute for the setup.py run? I've tried setting PYTHONPATH, but that doesn't seem to work - probably because directories on PYTHONPATH do not have .pth files processed.

To clarify, what I'm doing is:

here = os.path.abspath(os.path.dirname(__file__)) # setuptools is available in os.path.join(here, 'setuptools') # setuptools.pth and easy-install.pth are in here

os.environ[PYTHONPATH] = here # this doesn't work setup_py = <the setup.py file> setup_dir = os.path.abspath(os.path.dirname(setup_py)) subprocess.check_call([sys.executable, 'setup.py', 'install'] + special_install_options, cwd=setup_dir)

The point of this is that I want to write a script to build wheels from sdists, but I don't want to require the user to install setuptools just for the build step.

Does anyone have any suggestions? Paul

Look at site.py, specifically site.addsitedir(sitedir, known_paths=None)

Sorry, I wasn't clear. I know about that, but I don't know how to set up a site directory *before* starting the Python process. Thinking about it, I guess I could do the addsitedir thing and then execfile setup.py. Put all of that into a -c script. That's probably OK, just a bit messy. Particularly as execfile is no longer a builtin.

It is messy but you will probably wind up doing what pip does in several places, for example: python -c "import setuptools; __file__=%r; exec(compile(open(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))" % self.setup_py https://github.com/pypa/pip/blob/develop/pip/req.py#L655 Why not just reserve a separate virtualenv for building?

Paul Moore writes:

distribute for the setup.py run? I've tried setting PYTHONPATH, but that doesn't seem to work

Here's what I tried. I set up the following work area: . ├── pkg_resources.py -> /path/to/distribute/pkg_resources.py ├── runit.py ├── setuptools -> /path/to/distribute/setuptools/ └── work ├── mymodule.py └── setup.py Where /path/to/distribute is pointing into my clone of Distribute (i.e. an uninstalled copy). Then, runit.py is analogous to your script: import os import subprocess import sys here = os.path.abspath(os.path.dirname(__file__)) # setuptools is available in os.path.join(here, 'setuptools') # setuptools.pth and easy-install.pth are in here os.environ['PYTHONPATH'] = here # this doesn't work setup_py = 'work/setup.py' setup_dir = os.path.abspath(os.path.dirname(setup_py)) subprocess.check_call([sys.executable, 'setup.py', 'sdist'], cwd=setup_dir) Of course, I'm doing sdist rather than install, but IMO it should work in the same way for install. mymodule.py is an empty file, and setup.py is: import setuptools print('using setuptools from %s' % setuptools.__file__) setuptools.setup(name='dummy', version='0.1', py_modules=['mymodule']) Now, when in the top of my work area I run "python runit.py", I get: $ python runit.py using setuptools from /home/vinay/projects/scratch/junk/setuptools/__init__.pyc running sdist running egg_info creating dummy.egg-info writing dummy.egg-info/PKG-INFO writing top-level names to dummy.egg-info/top_level.txt writing dependency_links to dummy.egg-info/dependency_links.txt writing manifest file 'dummy.egg-info/SOURCES.txt' reading manifest file 'dummy.egg-info/SOURCES.txt' writing manifest file 'dummy.egg-info/SOURCES.txt' warning: sdist: standard file not found: should have one of README, README.rst, README.txt running check warning: check: missing required meta-data: url warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied creating dummy-0.1 creating dummy-0.1/dummy.egg-info making hard links in dummy-0.1... hard linking mymodule.py -> dummy-0.1 hard linking setup.py -> dummy-0.1 hard linking dummy.egg-info/PKG-INFO -> dummy-0.1/dummy.egg-info hard linking dummy.egg-info/SOURCES.txt -> dummy-0.1/dummy.egg-info hard linking dummy.egg-info/dependency_links.txt -> dummy-0.1/dummy.egg-info hard linking dummy.egg-info/top_level.txt -> dummy-0.1/dummy.egg-info Writing dummy-0.1/setup.cfg creating dist Creating tar archive removing 'dummy-0.1' (and everything under it) and my work area now looks like this: . ├── pkg_resources.py -> /path/to/distribute/pkg_resources.py ├── pkg_resources.pyc ├── runit.py ├── setuptools -> /path/to/distribute/setuptools/ └── work ├── dist │ └── dummy-0.1.tar.gz ├── dummy.egg-info │ ├── dependency_links.txt │ ├── PKG-INFO │ ├── SOURCES.txt │ └── top_level.txt ├── mymodule.py └── setup.py Note the pkg_resources.pyc file, plus the "using setuptools from ..." message, indicating that it's not running some other setuptools, This is on Linux and I used symlinks for this quick test, but I can't see why it would be different for a copy. Regards, Vinay Sajip