[Numpy-discussion] Issue with setup_requires and 1.16 release candidates

[Ralf Gommers]

On Sun, Jan 6, 2019 at 12:45 PM Thomas Robitaille <
thomas.robitaille at gmail.com> wrote:

> Hi all,
>
> Back in December I started getting failures in continuous integration
> as well as reports of failures from users of installation issues for a
> couple of packages. The problem can be easily reproduced in a Docker
> container with:
>
>    FROM ubuntu:16.04
>    RUN apt-get update
>    RUN apt-get install -y python3 python3-dev python3-pip python3-wheel
>    RUN pip3 install fast-histogram
>    RUN python3 -c 'import fast_histogram'
>
> Doing this results in the following traceback:
>
>    ImportError: No module named 'numpy.core._multiarray_umath'
>    Traceback (most recent call last):
>      File "<string>", line 1, in <module>
>      File
> "/usr/local/lib/python3.5/dist-packages/fast_histogram/__init__.py",
> line 1, in <module>
>        from .histogram import *
>      File
> "/usr/local/lib/python3.5/dist-packages/fast_histogram/histogram.py",
> line 7, in <module>
>        from ._histogram_core import (_histogram1d,
>    ImportError: numpy.core.multiarray failed to import
>
> I've seen similar issues with other packages too. The key is that the
> fast-histogram package defines:
>
>    setup_requires=['numpy']
>
> in the setup.py (since the package has a C extension that uses the
> Numpy C API) and numpy is needed before the install_requires
> dependencies are installed:
>
>    https://github.com/astrofrog/fast-histogram/blob/master/setup.py
>
> Now this normally works fine, but the issues I saw started when the
> first 1.16 RC was made available, when installing into an environment
> in which numpy is not already installed.
>
> My understanding is that setup_requires is honored by easy_install
> (even if installing the main package with pip), which doesn't ignore
> pre-releases. Thus, the package is built against the 1.16 RC but then
> 1.15 is installed due to:
>
>    install_requires=['numpy']
>
> which is honored by pip.
>

Oh fun. Thanks for explaining this!


> I think that the correct solution is to make sure that:
>
>    [build-system]
>    requires = ["setuptools", "wheel", "numpy"]
>
> is added to the pyproject.toml file (as per PEP 518).


To clarify: you're talking about fast-histogram's pyproject.toml file right?

This then works
> properly with recent versions of pip (>10 I think). I think removing
> setup_requires then makes sense because it'd be better to have an
> error with older versions of pip that numpy is not installed rather
> than having the package built against the wrong version.
>

This is a pretty hacky solution, needed because pip doesn't support
setup_requires, but yes it should work and I can't think of a better way.

Cheers,
Ralf



> I know this is a temporary issue in the sense that it will go away
> once the final version of 1.16 is out, but I just wanted to share this
> as a heads-up in case you get reports of issues from other people, and
> also to check whether there are any other solutions/workarounds to be
> aware of? (to avoid a similar situation in future).
>
> Thanks,
> Tom
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190106/449bb4f8/attachment.html>