[Matplotlib-devel] [Wheel-builders] Manylinux external library test case - matplotlib, tk

[Matthew Brett]

On Tue, May 10, 2016 at 5:26 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> Hi,
>
> On Tue, May 10, 2016 at 4:13 PM, Nathaniel Smith <njs at pobox.com> wrote:
>> On May 10, 2016 09:00, "Matthew Brett" <matthew.brett at gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> On Tue, May 3, 2016 at 4:47 PM, Matthew Brett <matthew.brett at gmail.com>
>>> wrote:
>>> > On Tue, May 3, 2016 at 5:35 AM, Olivier Grisel
>>> > <olivier.grisel at ensta.org> wrote:
>>> >>> I tested it with:
>>> >>>
>>> >>> import matplotlib
>>> >>> matplotlib.use('PyQt5')
>>> >>
>>> >> Typo, this was supposed to read:
>>> >>
>>> >> matplotlib.use('Qt5Agg')
>>> >
>>> > Meanwhile, I tried removing (patchelf --remove-needed) the
>>> > requirements of _tkagg.so on the vendored libtk, libtcl, and added
>>> > back the requirement (patch --add-needed) on general `libtk.so' and
>>> > 'libtcl.so'.  This removed the segfault and allowed me to display
>>> > `plt.range(10)'.  I suppose then, that the tk / tcl ABI that we are
>>> > using is relatively stable across versions 8.4 (on the docker image)
>>> > and 8.6 (on Debian sid).
>>> >
>>> > Worth experimenting with this - or too much of a hack?
>>>
>>> I have experimented with this.  It does seem that we can get away with
>>> using the system tcl / tk libraries as installed, at least on a couple
>>> of Debian systems I have.
>>>
>>> Now there's a new problem, to do with linux library namespaces.  We
>>> need the _tkagg.so file to use the same libraries as the Python
>>> Tkinter package at run time.    There can be more than one tcl / tk
>>> version installed on a given system, so, as things stand, we'd need to
>>> inspect the Tkinter extension modules to see what they are using, and
>>> then work out a way to use those libraries at run time.  Here are the
>>> questions for the experts out there:
>>>
>>> * Can anyone think of a way of using the symbol namespace of the
>>> tkinter extension modules directly, to pick up the symbols we need
>>> rather than re-importing the libraries?
>>
>> It would be a bit of a weird hack, but we could have _tkagg.so "link"
>> against the tkinter extension module. The idea would be that we don't care
>> about the symbols that the tkinter extension exports, but we'd take
>> advantage on the fact the on ELF, linking against one .so automatically
>> gives you not just its symbols, but also the symbols of all the libraries
>> that it links against, transitively.
>>
>> This would probably look like:
>>
>> _tkagg.so has a DT_NEEDED entry naming tkinter.so (or whatever Python calls
>> this module)
>>
>> Before loading _tkagg.so, we use Python level introspection figure out where
>> tkinter.so lives
>>
>> We add its directory to LD_LIBRARY_PATH
>>
>> we import _tkagg.so
>>
>> We take its directory back off of LD_LIBRARY_PATH
>>
>> Very weird, but I can't see why it wouldn't work, and probably more reliable
>> than anything where we try to reimplement the dynamic loader's search logic
>> ourselves.
>
> Nice - yes - it does work in a first-pass test - I'll look into automating that.

Ah - except I do not believe it is possible to change the linker path
for Python modules, within the Python process:

http://www.gossamer-threads.com/lists/python/python/393770#393770

Ugh.

Matthew