[Numpy-discussion] linux wheels coming soon

[Matthew Brett]

On Thu, Apr 14, 2016 at 12:57 PM, Matthew Brett <matthew.brett at gmail.com> wrote:
> On Thu, Apr 14, 2016 at 12:25 PM, Jonathan Helmus <jjhelmus at gmail.com> wrote:
>>
>>
>> On 4/14/16 1:26 PM, Matthew Brett wrote:
>>>
>>> Hi,
>>>
>>> On Thu, Apr 14, 2016 at 11:11 AM, Benjamin Root <ben.v.root at gmail.com>
>>> wrote:
>>>>
>>>> Are we going to have to have documentation somewhere making it clear that
>>>> the numpy wheel shouldn't be used in a conda environment? Not that I
>>>> would
>>>> expect this issue to come up all that often, but I could imagine a
>>>> scenario
>>>> where a non-scientist is simply using a base conda distribution because
>>>> that
>>>> is what IT put on their system. Then they do "pip install ipython" that
>>>> indirectly brings in numpy (through the matplotlib dependency), and end
>>>> up
>>>> with an incompatible numpy because they would have been linked against
>>>> different pythons?
>>>>
>>>> Or is this not an issue?
>>>
>>> I'm afraid I don't know conda at all, but I'm guessing that pip will
>>> not install numpy when it is installed via conda.
>>
>> Correct, pip will not (or at least should not, and did not in my tests)
>> install numpy over top of an existing conda installed numpy. Unfortunately
>> from my testing, conda will install a conda version of numpy over top of a
>> pip installed version.  This may be the expected behavior as conda maintains
>> its own list of installed packages.
>>>
>>> So the potential difference is that, pre-wheel, if numpy was not
>>> installed in your conda environment, then pip would build numpy from
>>> source, whereas now you'll get a binary install.
>>>
>>> I _think_ that Python's binary API specification
>>> (pip.pep425tags.get_abi_tag()) should prevent pip from installing an
>>> incompatible wheel.  Are there any conda experts out there who can
>>> give more detail, or more convincing assurance?
>>
>> I tested "pip install numpy" in conda environments (conda's equivalent to
>> virtualenvs) which did not have numpy installed previously for Python 2.7,
>> 3.4 and 3.5 in a Ubuntu 14.04 Docker container.  In all cases numpy was
>> installed from the whl file and appeared to be functional.  Running the
>> numpy test suite found three failing tests for Python 2.7 and 3.5 and 21
>> errors in Python 3.4. The 2.7 and 3.5 failures do not look concerning but
>> the 3.4 errors are a bit strange.
>> Logs are in
>> https://gist.github.com/jjhelmus/a433a66d56fb0e39b8ebde248ad3fe36
>
> Thanks for testing.  For:
>
> docker run -ti --rm ubuntu:14.04 /bin/bash
>
> apt-get update && apt-get install -y curl
> curl -LO https://bootstrap.pypa.io/get-pip.py
> python3 get-pip.py
> pip install numpy nose
> python3 -c "import numpy; numpy.test()"
>
> I get:
>
> FAILED (KNOWNFAIL=7, SKIP=17, errors=21)
>
> This is stock Python 3.4 - so not a conda issue.  It is definitely a
> problem with the wheel because a compiled numpy wheel on the same
> docker image:
>
> apt-get update && apt-get install -y curl python3-dev
> curl -LO https://bootstrap.pypa.io/get-pip.py
> python3 get-pip.py
> pip install --no-binary=:all: numpy nose
> python3 -c "import numpy; numpy.test()"
>
> gives no test errors.
>
> It looks like we have some more work to do...

Actually, I can solve these errors by first doing:

apt-get install gcc

I think these must be bugs in the numpy tests where numpy is assuming
a functional compiler.

Does the conda numpy give test errors when there is no compiler?

Cheers,

Matthew