Use of objdump within ctypes _get_soname()
Hi, We ran into an issue on the Anaconda Distribution recently where we added libarchive-c to conda-build (so we can un-compress more source archive formats than tarfile supports) and everything was good a few hours, until it hit various CI systems where objdump is not installed. I was a bit surprised by this dependency and wondered if there'd be interest in a fallback path that inspects the elf with some fairly simple python code to determine the soname instead? I have code that works already - though it could do with and a tidy up - and has been tested on a variety of architectures. Would CPython be interested in an attempt to upstream this? Is it documented anywhere that objdump is needed to load some extension modules on Linux? Best regards, Ray Donnelly, Anaconda Inc,
On Mon, Oct 8, 2018 at 11:59 PM Ray Donnelly <mingw.android@gmail.com> wrote:
Hi,
We ran into an issue on the Anaconda Distribution recently where we added libarchive-c to conda-build (so we can un-compress more source archive formats than tarfile supports) and everything was good a few hours, until it hit various CI systems where objdump is not installed.
I was a bit surprised by this dependency and wondered if there'd be interest in a fallback path that inspects the elf with some fairly simple python code to determine the soname instead? I have code that works already - though it could do with and a tidy up - and has been tested on a variety of architectures. Would CPython be interested in an attempt to upstream this?
Is it documented anywhere that objdump is needed to load some extension modules on Linux?
Wow, that looks like gross code buried within ctypes.util (which libarchive-c uses) that calls various platforms versions of objdump or equivalent. File a bugs.python.org issue and submit a PR, native ELF header reading code for this makes sense. -G
Best regards,
Ray Donnelly, Anaconda Inc, _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
I've looked into this more and it is more subtle than my initial report suggests (as usual). We have some custom code in AD around find_library() so it looks in sys.prefix/lib (on Unix) and I had a bug in that code when _get_soname returned None. I've fixed that now. I believe there is a condition under which this could go wrong which is if the soname returned by objdump differs from the filename and then the filename is not found. This would happen to people with objdump installed but not to people without it installed, so I feel the behaviour here should be unified. If you think we should unify it by implementing a python replacement for this functionality then I'll get on with a PR (and file that bug report too), the other option is to just remove this part and have just the fallbacks happen instead? On Tue, Oct 9, 2018 at 8:13 AM Gregory P. Smith <greg@krypto.org> wrote:
On Mon, Oct 8, 2018 at 11:59 PM Ray Donnelly <mingw.android@gmail.com> wrote:
Hi,
We ran into an issue on the Anaconda Distribution recently where we added libarchive-c to conda-build (so we can un-compress more source archive formats than tarfile supports) and everything was good a few hours, until it hit various CI systems where objdump is not installed.
I was a bit surprised by this dependency and wondered if there'd be interest in a fallback path that inspects the elf with some fairly simple python code to determine the soname instead? I have code that works already - though it could do with and a tidy up - and has been tested on a variety of architectures. Would CPython be interested in an attempt to upstream this?
Is it documented anywhere that objdump is needed to load some extension modules on Linux?
Wow, that looks like gross code buried within ctypes.util (which libarchive-c uses) that calls various platforms versions of objdump or equivalent. File a bugs.python.org issue and submit a PR, native ELF header reading code for this makes sense.
-G
Best regards,
Ray Donnelly, Anaconda Inc, _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
participants (2)
-
Gregory P. Smith -
Ray Donnelly