[Distutils] PEP for dependencies on libraries like BLAS (was: Re: Working toward Linux wheel support)

[Robert Collins]

On 13 August 2015 at 12:51, Nathaniel Smith <njs at pobox.com> wrote:
> On Aug 12, 2015 16:49, "Robert Collins" <robertc at robertcollins.net> wrote:
>>
>> I'm not sure what will be needed to get the PR accepted; At PyCon AU
>> Tennessee Leuwenberg started drafting a PEP for the expression of
>> dependencies on e.g. BLAS - its been given number 497, and is in the
>> packaging-peps repo; I'm working on updating it now.
>
> I wanted to take a look at this PEP, but I can't seem to find it. PEP 497:
>   https://www.python.org/dev/peps/pep-0497/
> appears to be something else entirely?
>
> I'm a bit surprised to hear that such a PEP is needed. We (= numpy devs)
> have actively been making plans to ship a BLAS wheel on windows, and AFAICT
> this is totally doable now -- the blocker is windows toolchain issues, not
> pypa-related infrastructure.
>
> Specifically the idea is to have a wheel that contains the shared library as
> a regular old data file, plus a stub python package that knows how to find
> this data file and how to make it accessible to the linker. So
> numpy/__init__.py would start by calling:
>
> import pyopenblas1
> # on Linux modifies LD_LIBRARY_PATH,
> # on Windows uses ctypes to preload... whatever
> pyopenblas1.enable()
>
> and then get on with things, or the build system might do:
>
> import pyopenblas1
> pyopenblas1.get_header_directories()
> pyopenblas1.get_linker_directories()
>
> This doesn't help if you want to declare dependencies on external, system
> managed libraries and have those be automatically somehow provided or
> checked for, but to me that sounds like an impossible boil-the-ocean project
> anyway, while the above is trivial and should just work.

Well, have a read of the draft.

Its a solved problem by e.g. conda, apt, yum, nix and many others.

Uploading system .so's is certainly also an option, and I see no
reason why we can't do both.

I do know that distribution vendors are likely to be highly allergic
to the idea of having regular shared libraries present as binaries,
but thats a different discussion :)

-Rob


-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud