Installing header files with setuptools
Perhaps this is a bug or perhaps I'm just doing it wrong, but I can't figure out a good way to install C header files as part of a setuptools project. With distutils, there's a "headers" argument to setup() that installs the header files in their proper spot, but setuptools ignores this. It looks like setuptools is hijacking the install command and manually running install_lib, install_data, etc, but never install_headers. Does setuptools expect me to do this in a different way, or is this something that should be fixed? - Jim
At 11:30 PM 3/2/2010, James Porter wrote:
Perhaps this is a bug or perhaps I'm just doing it wrong, but I can't figure out a good way to install C header files as part of a setuptools project.
With distutils, there's a "headers" argument to setup() that installs the header files in their proper spot, but setuptools ignores this. It looks like setuptools is hijacking the install command and manually running install_lib, install_data, etc, but never install_headers.
Egg-based installs don't include headers. However, if you use the --root or --single-version-externally-managed options to "setup.py install", a backwards-compatible, non-egg-based installation is done, and the headers should be included in that case. This installation mode is used by default when building other types of binary distribution, such as RPMs.
On 3/3/2010 12:02 AM, Phillip J. Eby wrote:
Egg-based installs don't include headers. However, if you use the --root or --single-version-externally-managed options to "setup.py install", a backwards-compatible, non-egg-based installation is done, and the headers should be included in that case. This installation mode is used by default when building other types of binary distribution, such as RPMs.
Hm. In that case, is there an easy way to get install_headers to run when installing via easy_install (or pip) *without* passing special arguments to it? Ideally I'd like to use the module as the basis for other C extension modules and have source compilation via easy_install/pip "just work". Right now I just have a duplicate of the headers in the dependent project, but that seems somewhat hackish. Nevertheless, at least it's encouraging to know what's really going on. Thanks. - Jim
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 James Porter wrote:
Hm. In that case, is there an easy way to get install_headers to run when installing via easy_install (or pip) *without* passing special arguments to it?
Have you tried pip? Pip always uses --single-version-externally-managed, so if what Phillip says is correct, I would expect it to already just work. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEUEARECAAYFAkuOZyYACgkQ1j/fhc23WEBAdwCgqYGL947f8oziQxeN7TuT0ITg o1cAmJyYT8iYFd/5Ktpp6vAvJqc0LVo= =9JBi -----END PGP SIGNATURE-----
On Wed, 2010-03-03 at 08:41 -0500, Carl Meyer wrote:
James Porter wrote:
Hm. In that case, is there an easy way to get install_headers to run when installing via easy_install (or pip) *without* passing special arguments to it?
Have you tried pip? Pip always uses --single-version-externally-managed, so if what Phillip says is correct, I would expect it to already just work.
This works, but it installs the headers in a surprising place. With the "prefix" scheme, the headers end up in `pwd`/lib/include. With the "home" scheme, the headers end up in <home>/lib/python/lib/include. In either case I'd expect them to end up in <home-or-prefix>/include (or something like that anyway). I'd *certainly* expect the prefix scheme not to install the headers outside of the prefix. The issue, it seems, is a workaround for virtualenv in pip/req.py: <http://hg.notalon.org/ianb/pip/src/tip/pip/req.py#cl-515>. Since I'm not actually using virtualenv, I don't know exactly what would happen with that, but if there's a workaround for virtualenv, shouldn't it only be activated when virtualenv is in use? Not that I know how to detect that. :) - Jim
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 James Porter wrote:
This works, but it installs the headers in a surprising place. With the "prefix" scheme, the headers end up in `pwd`/lib/include. With the "home" scheme, the headers end up in <home>/lib/python/lib/include.
In either case I'd expect them to end up in <home-or-prefix>/include (or something like that anyway). I'd *certainly* expect the prefix scheme not to install the headers outside of the prefix.
The issue, it seems, is a workaround for virtualenv in pip/req.py: <http://hg.notalon.org/ianb/pip/src/tip/pip/req.py#cl-515>. Since I'm not actually using virtualenv, I don't know exactly what would happen with that, but if there's a workaround for virtualenv, shouldn't it only be activated when virtualenv is in use? Not that I know how to detect that. :)
Yeah, that does appear to be a bug in pip, I've reported it at http://bitbucket.org/ianb/pip/issue/73/header-files-installed-to-wrong-locat.... And I think you have the right fix, too; that header location hack should only happen if actually inside a virtualenv. Shouldn't be a hard fix. Is your package publicly available? I don't have anything on hand that installs header files to test the fix with. BTW, the best way I know of to check for "am I in a virtualenv?" is to check for the existence of sys.real_prefix. Carl -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkuO4usACgkQ1j/fhc23WEB74QCdGwJslblzsTKIjRrxQUZBL8tO BZYAni3OGpadrKi5nidV5zKZI1xZ/+A0 =Pn0u -----END PGP SIGNATURE-----
On Wed, 2010-03-03 at 17:30 -0500, Carl Meyer wrote:
Yeah, that does appear to be a bug in pip, I've reported it at http://bitbucket.org/ianb/pip/issue/73/header-files-installed-to-wrong-locat.... And I think you have the right fix, too; that header location hack should only happen if actually inside a virtualenv. Shouldn't be a hard fix. Is your package publicly available? I don't have anything on hand that installs header files to test the fix with.
My package is publically available here <http://pypi.python.org/pypi/PyTAPS/>, but that's probably not all that helpful in practice, since it has a fair number of C dependencies. For simplicity, I made an incredibly simple package (all it installs is a single header file) that reproduces the problem for me, which you can get via: pip install -f https://mywebspace.wisc.edu/jvporter/web/header_test-1.0.tar.gz header_test - Jim
participants (3)
-
Carl Meyer -
James Porter -
Phillip J. Eby