[Distutils] Dynamic linking between Python modules (was: Beyond wheels 1.0: helping downstream, FHS and more)

[Chris Barker]

On Fri, May 15, 2015 at 1:44 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> > Is there any point? or is the current approach of statically linking all
> > third party libs the way to go?
>
> If someone can make it work, that would be good. But (a) nobody is
> actually offering to develop and maintain such a solution,


well, it's on my list -- but it has been for a while, so I'm trying to
gauge whether it's worth putting at the top of my "things to do for python"
list. It's not at the top now ;-)


> and (b)
> it's not particularly clear how *much* of a benefit there would be
> (space savings aren't that important, ease of upgrade is fine as long
> as everything can be upgraded at once, etc...)
>

hmm -- that may be a trick, though not a uncommon one in python package
dependencies -- it maybe hard to have more than one version of a given lib
installed....

> If so, then is there any chance of getting folks to conform to this
> standard
> > for PyPi hosted binary packages anyway? i.e. the curation problem.
>
> If it exists, and if there's a benefit, people will use it.
>

OK -- that's encouraging...


> > Personally, I'm on the fence here -- I really want newbies to be able to
> > simply "pip install" as many packages as possible and get a good result
> when
> > they do it.
>
> Static linking gives that on Windows FWIW. (And maybe also on OSX?)
> This is a key point, though - the goal shouldn't be "use dynamic
> linking" but rather "make the user experience as easy as possible". It
> may even be that the best approach (dynamic or static) differs
> depending on platform.
>

true -- though we also have another problem -- that static linking solution
is actually a big pain for package maintainers -- building and linking the
dependencies the right way is a pain -- and now everyone that uses a given
lib has to figure out how to do it. Giving folks a dynamic lib they can use
would mie it easier for tehm to build their packages -- a nice benifit
there.

Though it's a lot harder to provide a build environment than just the lib
to link too .. I"m going to have to think more about that...


> > On the other hand, I've found that conda better supports this right now,
> so
> > it's easier for me to simply use that for my tools.
>
> And that's an entirely reasonable position. The only "problem" (if
> indeed it is a problem) is that by having two different solutions
> (pip/wheel and conda) splits the developer resource, which means that
> neither approach moves forward as fast as a combined approach does.
>

That's not the only problem -- the current split between the (more than
one) scientifc python distributions, and the community of folks using
python.org and pypi creates a bit of a mess for newbies.

I'm reviving this conversation because i just spent a class lecture in a
python class on numpy/scipy -- these students have been using a python
install for months, using virtualenv, ip installing whatever they need, et.

and now, to use another lib, they have to go through machination, maybe
even installing a entire additional python. This is not good. And I've had
to help more than one student untangle a mess of Apple Python python.org
python, homebrew, and/or Anaconda -- for someone that doesn't really get
python pacakging, never mond PATHS, and .bashrc vs .bash_profile, etc, it's
an unholy mess.

"There should be one-- and preferably only one --obvious way to do it." --
HA!


But that's OK if the two solutions are addressing different needs
>

The needs aren't really that different, however. Oh well.

Anyway, it seems like if I can find some time to prototype what I have in
mind, there may be some room to make it official if it works out. If anyone
else want to help -- let me know!

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150515/024a543d/attachment.html>