Exactly. Python actually specifies metadata around this (Requires-External https://packaging.python.org/specifications/core-metadata/#requires-external...), but I don’t believe pip implements it at all since there’re almost no sensible rules available on how the external libraries can be located in a cross-platform way. Conda is probably the best bet when you need to deal with tight cross-language package integration like this, by punting the whole idea of system libraries and installing a separate copy of everything you need. -- Tzu-ping Chung (@uranusjr) uranusjr@gmail.com https://uranusjr.com
On 06/8/2020, at 07:25, Jonathan DEKHTIAR
wrote: I like the general idea. But I feel it's not going to be doable in practice. Many of the C libraries are not necessarily installed in usual places like `/usr/shared/lib` (like drivers for instances) or you can't be 100% sure about it.
And that doesn't even account for Windows, which might behave quite a lot different. How about python package that comes with C libraries (compiled on install): numpy / tensorflow / torch / cupy / etc.
I'm not against the idea. I just don't see a good way of doing it. For example, do you want to check on the system libraries or also `LD_LIBRARY_PATH` and `LIBRARY_PATH`. Do you want to check inside the user .bashrc for some modification of env vars (what if he doesn't bash) ?
Sounds honestly difficult to design a feature like this,
Jonathan
---- Le mer., 05 août 2020 16:03:40 -0700 David Mathog
écrit ---- pip install package
often results in compiling (using gcc, g++, whatever) to produce a binary. Usually that proceeds without issue. However, there seems to be no checking that the libraries required to link that binary are already on the system. Or at least the message which results when they are not is not at all clear about what is missing.
I discovered that today by wasting several hours figuring out why scanpy-scripts was failing trying to build dependency "louvain", which would not install into a venv with pip. It had something to do with "igraph", but pip had downloaded python-igraph before it got to louvain. When louvain tried to build there was a mysterious message about pkgconfig and igraph
Cannot find the C core of igraph on this system using pkg-config.
(Note that when python-igraph installs it places an igraph directory in site-packages, so which it is referring to is fairly ambiguous.) Then it tried to install a different version number of igraph, failed, and the install failed. This was very confusing because the second igraph install was not (it turned out) a different version of python-igraph but a system level igraph library, which it could not install either because the process was not privileged and could not write to the target directories. Yet it tried to install anyway. This is discussed in the louvain documentation here (it turns out):
https://github.com/vtraag/louvain-igraph https://github.com/vtraag/louvain-igraph
but since I was actually trying to install a different package, of course I had not read the louvain documentation.
In short form the problem was "cannot build a binary because required library libigraph.so is not present in the operating system" but that was less than obvious in the barrage of warnings and error messages.
Is it possible to tell pip or setup.py to fail immediately when a required system library like this is not found, here presumably after that "C core" message, rather than confusing the matter further with a failed partial build and install of the same component?
More generally, is there anything in the python installation methods which could list system libraries as dependencies and give a more informative error message when they are missing?
Thanks,
David Mathog -- Distutils-SIG mailing list -- distutils-sig@python.org mailto:distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org mailto:distutils-sig-leave@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/MSS42... https://mail.python.org/archives/list/distutils-sig@python.org/message/MSS42...
-- Distutils-SIG mailing list -- distutils-sig@python.org To unsubscribe send an email to distutils-sig-leave@python.org https://mail.python.org/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/LSR26...