[Distutils] Current Python packaging status (from my point of view)

Chris Barker chris.barker at noaa.gov
Thu Nov 3 18:15:30 EDT 2016


On Wed, Nov 2, 2016 at 5:02 PM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Anaconda has an overwhelming advantage on Windows, in that Continuum
> can bear the licensing liabilities enforced by the Intel Fortran
> compiler, and we can not.


Technically, that's an advantage that a commercial distribution has --
really nothing to do with the conda technology per se. But yes, from a
practical perspective Continuum's support for conda an Anaconda is a major
bonus in many ways.

Note that continuum in now (I think) default ing to MKL also -- they seem
to have solved the res-distribution issues.

But the conda-forge project has its own builds of lots of stuff --
including the core scipy stack, based on OpenBLAS. There is a scipy build
there:

https://github.com/conda-forge/scipy-feedstock/tree/master/recipe

but alas, still stuck:

```
build:
    # We lack openblas on Windows, and therefore can't build scipy there
either currently.
    skip: true # [win or np!=111]
```

>   I'm sure you know, but the only
> practical open-source option is mingw-w64, that does not work with the
> Microsoft runtime used by Python 3.5 [1].


OT -- but is it stable for Python2.7 now?


> > But not pyHDF, netCDF5, gdal, shapely, ... (to name a few that I need to
> > work with). And these are ugly: which means very hard for end-users to
> > build, and very hard for people to package up into wheels (is it even
> > possible?)
>
> I'd be surprised if these packages were very hard to build OSX and
> Linux wheels for.  We're already building hard packages like Pillow
>

Pilllow is relatively easy :-) -- I was doing that years ago.


> and Matplotlib and h5py and pytables.


Do h5py and pytables share libhdf5 somehow? Or is the whole mess statically
linked into each?

  If you mean netCDF4 - there are
> already OSX and Windows wheels for that [2].
>

God bless Chris Gohlke  --- I have no idea how he does all that!

So take all the hassle of those -- and multiply by about ten to get what
the GIS stack needs: GDAL/OGR, Shapely, Fiona, etc....

Which doesn't mean it couldn't be done -- just that it would be a pain, and
you'd end up with some pretty bloated wheels -- in the packages above, how
many copies of libpng will there be? how many of hdf5? proj.4? geos?

Maybe that simply doesn't matter -- computers have a lot of disk space and
memory these days.

But there is one more use-case -- folks that need to build their own stuff
against some of those libs (netcdf4 in my case). The nice thing about conda
is that it can provide the libs for me to use in my own custom built
projects, or for other folks to use to build conda packages with.

Maybe pip could be used to distribute those libs, too -- I know folks are
working on that.

Then there are non-python stuff that you need to work with that would be
nice to mange in environments, too -- R, MySQL, who knows?

As someone else on this thread noted -- it's jamming a square peg into a
round hole -- and why do that when there is a square hole you can use
instead?

At least that's what I decided.

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20161103/4c3d8a48/attachment.html>


More information about the Distutils-SIG mailing list