Hello,
I have a Python project that depends on an external C library (which is unrelated to Python and NOT part of the OS).
For an sdist, this is easy: my setup.py assumes that the library is pre-installed somewhere on the system where setuptools can find it.
However, is there a standard solution for packaging such a project as wheel? Ideally, the project should "just work" when doing pip install on it, which means that the external library should somehow be bundled in the wheel.
I know that for example numpy does that, but it also has a very complicated build system (containing a fork of distutils!).
Does anybody know any pointers for this? Or if you think that this is a minefield which is not really supported, feel free to say so.
Thanks, Jeroen.
It's necessary to bundle the external library with the wheel.
On linux, this is what the manylinux tag is all about (see PEP 513 & 571) and the auditwheel tool (https://github.com/pypa/auditwheel) will bundle the library into the wheel.
On OS X and Windows, I'm less familiar with the toolchain but the same principle applies.
-Robert
On Tue, Jan 22, 2019 at 6:41 AM Jeroen Demeyer J.Demeyer@ugent.be wrote:
Hello,
I have a Python project that depends on an external C library (which is unrelated to Python and NOT part of the OS).
For an sdist, this is easy: my setup.py assumes that the library is pre-installed somewhere on the system where setuptools can find it.
However, is there a standard solution for packaging such a project as wheel? Ideally, the project should "just work" when doing pip install on it, which means that the external library should somehow be bundled in the wheel.
I know that for example numpy does that, but it also has a very complicated build system (containing a fork of distutils!).
Does anybody know any pointers for this? Or if you think that this is a minefield which is not really supported, feel free to say so.
Thanks, Jeroen. -- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/6RZVK...
On 2019-01-22 13:04, Robert T. McGibbon wrote:
It's necessary to bundle the external library with the wheel.
On linux, this is what the manylinux tag is all about (see PEP 513 & 571) and the auditwheel tool (https://github.com/pypa/auditwheel) will bundle the library into the wheel.
On OS X and Windows, I'm less familiar with the toolchain but the same principle applies.
Thanks for your reply. Unfortunately as you say, it only applies to Linux.
So how do projects in practice deal with this in a cross-platform way? Does every package need to reinvent the "wheel"?
Hi,
On Fri, Feb 15, 2019 at 6:15 PM Jeroen Demeyer J.Demeyer@ugent.be wrote:
On 2019-01-22 13:04, Robert T. McGibbon wrote:
It's necessary to bundle the external library with the wheel.
On linux, this is what the manylinux tag is all about (see PEP 513 & 571) and the auditwheel tool (https://github.com/pypa/auditwheel) will bundle the library into the wheel.
On OS X and Windows, I'm less familiar with the toolchain but the same principle applies.
Thanks for your reply. Unfortunately as you say, it only applies to Linux.
So how do projects in practice deal with this in a cross-platform way? Does every package need to reinvent the "wheel"?
Delocate [1] deals with the same problem on macOS - see my Multibuild frameworks [2] for some shared tooling between the two. It's fairly widely used (numpy, scipy, matplotlib, etc).
I think the same general approach would be possible with Windows - see Nathaniel Smith's library at [3] - but I don't think anyone has done the work to package that into a wheel-building tool. Most people are building Windows wheels with static linking, I believe.
Nathaniel Smith proposed a specification for packaging binary libraries as wheels [4], but I think he isn't working on that now, and it hasn't progressed. Nathaniel - is that right?
Cheers,
Matthew
[1] https://github.com/matthew-brett/delocate [2] https://github.com/matthew-brett/multibuild [3] https://github.com/njsmith/machomachomangler [4] https://github.com/njsmith/wheel-builders/blob/pynativelib-proposal/pynative...
participants (3)
-
Jeroen Demeyer -
Matthew Brett -
Robert T. McGibbon