[Matplotlib-devel] [Wheel-builders] Manylinux external library test case - matplotlib, tk

[Matthew Brett]

Hi,

On Tue, May 10, 2016 at 4:13 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On May 10, 2016 09:00, "Matthew Brett" <matthew.brett at gmail.com> wrote:
>>
>> Hi,
>>
>> On Tue, May 3, 2016 at 4:47 PM, Matthew Brett <matthew.brett at gmail.com>
>> wrote:
>> > On Tue, May 3, 2016 at 5:35 AM, Olivier Grisel
>> > <olivier.grisel at ensta.org> wrote:
>> >>> I tested it with:
>> >>>
>> >>> import matplotlib
>> >>> matplotlib.use('PyQt5')
>> >>
>> >> Typo, this was supposed to read:
>> >>
>> >> matplotlib.use('Qt5Agg')
>> >
>> > Meanwhile, I tried removing (patchelf --remove-needed) the
>> > requirements of _tkagg.so on the vendored libtk, libtcl, and added
>> > back the requirement (patch --add-needed) on general `libtk.so' and
>> > 'libtcl.so'.  This removed the segfault and allowed me to display
>> > `plt.range(10)'.  I suppose then, that the tk / tcl ABI that we are
>> > using is relatively stable across versions 8.4 (on the docker image)
>> > and 8.6 (on Debian sid).
>> >
>> > Worth experimenting with this - or too much of a hack?
>>
>> I have experimented with this.  It does seem that we can get away with
>> using the system tcl / tk libraries as installed, at least on a couple
>> of Debian systems I have.
>>
>> Now there's a new problem, to do with linux library namespaces.  We
>> need the _tkagg.so file to use the same libraries as the Python
>> Tkinter package at run time.    There can be more than one tcl / tk
>> version installed on a given system, so, as things stand, we'd need to
>> inspect the Tkinter extension modules to see what they are using, and
>> then work out a way to use those libraries at run time.  Here are the
>> questions for the experts out there:
>>
>> * Can anyone think of a way of using the symbol namespace of the
>> tkinter extension modules directly, to pick up the symbols we need
>> rather than re-importing the libraries?
>
> It would be a bit of a weird hack, but we could have _tkagg.so "link"
> against the tkinter extension module. The idea would be that we don't care
> about the symbols that the tkinter extension exports, but we'd take
> advantage on the fact the on ELF, linking against one .so automatically
> gives you not just its symbols, but also the symbols of all the libraries
> that it links against, transitively.
>
> This would probably look like:
>
> _tkagg.so has a DT_NEEDED entry naming tkinter.so (or whatever Python calls
> this module)
>
> Before loading _tkagg.so, we use Python level introspection figure out where
> tkinter.so lives
>
> We add its directory to LD_LIBRARY_PATH
>
> we import _tkagg.so
>
> We take its directory back off of LD_LIBRARY_PATH
>
> Very weird, but I can't see why it wouldn't work, and probably more reliable
> than anything where we try to reimplement the dynamic loader's search logic
> ourselves.

Nice - yes - it does work in a first-pass test - I'll look into automating that.

Cheers,

Matthew