[Numpy-discussion] Debugging NumPy development build in conda environment

[Hameer Abbasi]

> On Friday, Jan 04, 2019 at 2:44 PM, Lars Grueter <lagru at mailbox.org (mailto:lagru at mailbox.org)> wrote:
> Dear devs,
>
> If this is the wrong place for this kind of question I apologize and
> feel free to ignore this or point me to the right place.
>
> I'm using the conda package manager for my development environment which
> used to work just fine with these steps:
>
> > conda create -n dev-numpy python=3.7 numpy numpydoc cython pytest ipython coverage
> > source activate dev-numpy
> > conda uninstall --force numpy numpy-base
> > pip install -e .
>
> This would yield me a nice isolated playground and I could use
>
> > python setup.py build_ext --incplace -j 4
>
> at anytime to rebuild the binaries. Unfortunately the last two commands
> stopped working out of the blue with my existing environment and fail with
>
> > RuntimeError: Broken toolchain: cannot link a simple C program
>
> (full output in [1]) on a clean repository. No recent changes to my tool
> chain come to mind which could help me pinpoint the problem. Because it
> used to work just fine I have the annoying feeling that the fix is
> something small and trivial and have the hope that it's immediately
> obvious to someone here or that you can suggest a way to debug this. In
> any case your help would be appreciated.
>
> I'm aware that this may be an exotic dev setup. Using a virtual
> environment instead of the conda one works just fine. The build script
> seems to use a different linear algebra library in that case: blas from
> my OS as opposed to mkl in the conda environment. This is not my
> preferred solution but it would do if nothing comes of this.
>
> Some additional facts and research I've done:
>
> - Using conda environments with asv has stopped working as well due to
> not finding any LA library at all.
> - If I'm understanding [1] correctly the build script finds the mkl
> library provided by the conda environment.
> - Trying earlier versions of NumPy shows the same behavior. I tested the
> tags v1.14.6, v1.15.4.
> - However creating a development environment for SciPy and scikit-image
> using this approach still works fine.
> - Searching the internet or looking at NumPy's build guide didn't yield
> me a solution.
> - I'm on linux (Manjaro) if that is of any relevance.
> - Miniconda was reinstalled in my home directory with the installation
> script provided through Anaconda's website.
> - Haven't yet tried this approach in a fresh virtual machine.
>
> I hope that I haven't overlooked something obvious. In any case thank
> you for your time and attention.
>
> Cheers, Lars
>
> [1] https://gist.github.com/lagru/c6ac16b9984492850fe1174d6418b7b5
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion

Hi Lars,

What I’m used to doing is simply the following (inside the NumPy root):

> conda create -n numpy-dev python=3[.x] pytest
> conda activate numpy-dev
>
>
> python setup.py build_ext --inplace -j 4
>
>
> pip install -e .
>
>

Often, when I run into issues with the build, I simply do this:

> [backup site.cfg]
> git clean -xfd
>
>
> [restore site.cfg]
>
> python setup.py build_ext --inplace -j 4
>
> pip install -e .
>
>

In your case, it might be the second that helps. If it’s not finding any LA library, you can try compiling with the site.cfg that points to MKL.
Cheers and Best Regards,
Hameer Abbasi

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190104/53baa833/attachment.html>