[Distutils] Handling the binary dependency management problem

[Paul Moore]

On 1 December 2013 22:17, Nick Coghlan <ncoghlan at gmail.com> wrote:

> For example, I installed Nikola into a virtualenv last night. That required
> installing the development headers for libxml2 and libxslt, but the error
> that tells you that is a C compiler one.
>
> I've been a C programmer longer than I have been a Python one, but I still
> had to resort to Google to try to figure out what dev libraries I needed.

But that's a *build* issue, surely? How does that relate to installing
Nikola from a set of binary wheels?

I understand you are thinking about non-Python libraries, but all I
can say is that this has *never* been an issue to my knowledge in the
Windows world. People either ship DLLs with the Python extension, or
build statically. I understand that things are different in the Unix
world, but to be blunt why should Windows users care?

> Outside the scientific space, crypto libraries are also notoriously hard to
> build, as are game engines and GUI toolkits. (I guess database bindings
> could also be a problem in some cases)

Build issues again...

> We have the option to leave handling the arbitrary binary dependency problem
> to platforms, and I think we should take it.

Again, can we please be clear here? On Windows, there is no issue that
I am aware of. Wheels solve the binary distribution issue fine in that
environment (I know this is true, I've been using wheels for months
now - sure there may be specialist areas that need some further work
because they haven't had as much use yet, but that's details)

> This is why I suspect there will be a better near term effort/reward
> trade-off in helping the conda folks improve the usability of their platform
> than there is in trying to expand the wheel format to cover arbitrary binary
> dependencies.

Excuse me if I'm feeling a bit negative towards this announcement.
I've spent many months working on, and promoting, the wheel + pip
solution, to the point where it is now part of Python 3.4. And now
you're saying that you expect us to abandon that effort and work on
conda instead? I never saw wheel as a pure-Python solution, installs
from source were fine for me in that area. The only reason I worked so
hard on wheel was to solve the Windows binary distribution issue. If
the new message is that people should not distribute wheels for (for
example) lxml, pyyaml, pymzq, numpy, scipy, pandas, gmpy, and pyside
(to name a few that I use in wheel format relatively often) then
effectively the work I've put in has been wasted.

I'm hoping I've misunderstood here. Please clarify. Preferably with
specifics for Windows (as "conda is a known stable platform" simply
isn't true for me...) - I accept you're not a Windows user, so a
pointer to already-existing documentation is fine (I couldn't find any
myself).

Paul.