[Numpy-discussion] Should I use pip install numpy in linux?

[Benjamin Root]

Travis -

I will preface the following by pointing out how valuable miniconda and
anaconda has been for our workplace because we were running into issues
with ensuring that everyone in our mixed platform office had access to all
the same tools, particularly GDAL, NetCDF4 and such. For the longest time,
we were all stuck on an ancient "Corporate Python" that our IT staff
managed to put together, but never had the time to update. So, I do
absolutely love conda for the problems that it solved for us.

That being said... I take exception to your assertion that anaconda is
*the* solution to the packaging problem. I still have a number of issues,
particularly with the interactions of GDAL, shapely, and Basemap (they all
seek out libgeos_c differently), and I have to use my own build of GDAL to
enable many of the features that we use (the vanilla GDAL put out by
Continuum just has the default options, and is quite limited). If I don't
set up my environment *just right* one of those packages will fail to
import in some way due to being unable to find their particular version of
libgeos_c. I haven't figure it out exactly why this happens, but it is very
easy to break such an environment this way after an update.

But that problem is a solvable problem within the framework of conda, so I
am not too concerned about that. The bigger problem is Apache. In
particular, mod_wsgi. About a year ago, one of our developers was happily
developing a webtool that utilized numpy, NetCDF4 and libxml via conda
environments. All of the testing was done in flask and everything was
peachy. We figured that final deployment out to the Apache server would be
a cinch, right? Wrong. Because our mod_wsgi was built against the system's
python because it came in through RPMs, it was completely incompatible with
the compiled numpy because of the differences in the python compile
options. In a clutch, we had our IT staff manually build mod_wsgi against
anaconda's python, but they weren't too happy about that, due to mod_wsgi
no longer getting updated via yum.

If anaconda was the end-all, be-all solution, then it should just be a
simple matter to do "conda install mod_wsgi". But the design of conda is
that it is intended to be a user-space package-manager. mod_wsgi is
installed via root/apache user, which is siloed off from the user. I would
have to (in theory) go and install conda for the apache user and likely
have to install a conda "apache" package and mod_wsgi package. I seriously
doubt this would be an acceptable solution for many IT administrators who
would rather depend upon the upstream distributions who are going to be
very quick about getting updates out the door and are updated automatically
through yum or apt.

So, again, I love conda for what it can do when it works well. I only take
exception to the notion that it can address *all* problems, because there
are some problems that it just simply isn't properly situated for.

Cheers!
Ben Root


On Fri, Jan 15, 2016 at 3:20 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Fri, Jan 15, 2016 at 12:09 PM, Chris Barker <chris.barker at noaa.gov>
> wrote:
> > On Fri, Jan 15, 2016 at 11:21 AM, Nathaniel Smith <njs at pobox.com> wrote:
> >>
> >> Sure. Someone's already packaged those for conda, and no one has
> packaged
> >> them for pypi, so it makes sense that conda is more convenient for you.
> If
> >> someone does the work of packaging them for pypi, then that difference
> goes
> >> away.
> >
> > This is what I meant by "cultural" issues :-)
> >
> > but pypi has been an option for Windows and OS-X for ages and those
> > platforms are the bigger problem anyway -- and no one has done it for
> those
> > platforms
>
> I think what's going on here is that Windows *hasn't* been an option
> for numpy/scipy due to the toolchain issues, and on OS-X we've just
> been using the platform BLAS (Accelerate), so the core packages
> haven't had any motivation to sort out the whole library dependency
> issue, and no-one else is motivated enough to do it. My prediction is
> that after the core packages sort it out in order to solve their own
> problems then we might see others picking it up.
>
> -n
>
> --
> Nathaniel J. Smith -- http://vorpus.org
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20160115/d6e2be11/attachment.html>