On 3 Dec 2013 00:19, "Paul Moore" <p.f.moore@gmail.com> wrote:
>
> On 2 December 2013 13:38, Tres Seaver <tseaver@palladion.com> wrote:
> > On 12/01/2013 06:38 PM, Paul Moore wrote:
> >> I understand that things are different in the Unix world, but to be
> >> blunt why should Windows users care?
> >
> > You're kidding, right?  90% or more of the reason for wheels in the first
> > place is because Windows users can't build their own software from
> > source.  The amount of effort put in by non-Windows package owners to
> > support them dwarfs whatever is bothering you here.
>
> My point is that most of the complex binary compatibility problems
> seem to be Unix-related, and as you imply, Unix users don't seem to
> have much interest in using wheels except for local caching. So why
> build that complexity into the spec if the main users (Windows, and
> Unix users who won't ever publish wheels outside their own systems)
> don't have a need for it? Let's just stick with something simple that
> has limitations but works (practicality beats purity). My original
> bdist_simple proposal was a pure-Windows replacement for wininst.
> Daniel developed that into wheels which cater for non-Windows systems
> (I believe, precisely because he had an interest in the local cache
> use case). We're now seeing the complexities of the Unix world affect
> the design of wheels, and it's turning out to be a hard problem. All
> I'm trying to say is let's not give up on binary wheels for Windows,
> just because we have unsolved issues on Unix.

Huh? This is *exactly* what I am saying we should do - wheels *already* work so long as they're self-contained.

They *don't* work (automatically) when they have an external dependency: users have to obtain the external dependency by other means, and ensure that everything is properly configured to find it, and that everything is compatible with the retrieved version.

You're right that Christoph is doing two different things, though, so our advice to him (or anyone that wanted to provide the cross-platform equivalent of his current Windows-only stack) would be split:

- for all self-contained installers, also publish a wheel file on a custom index server (although having a "builder" role on PyPI where project owners can grant someone permission to upload binaries after the sdist is published could be interesting)
- for those installers which actually form an integrated stack with shared external binary dependencies, use the mechanisms provided by conda rather than getting users to manage the external dependencies by hand (as licensing permits, anyway)

>Whether solving the Unix
> issues is worth it is the Unix users' call - I'll help solve the
> issues, if they choose to, but I won't support abandoning the existing
> Windows solution just because it can't be extended to cater for Unix
> as well.

You appear to still be misunderstanding my proposal, as we're actually in violent agreement. All that extra complexity you're worrying about is precisely what I'm saying we should *leave out* of the wheel spec. In most cases of accelerator and wrapper modules, the static linking and/or bundling solutions will work fine, and that's the domain I believe we should *deliberately* restrict wheels to, so we don't get distracted trying to solve an incredibly hard external dependency management problem that we don't actually need to solve at the wheel level, since anyone that actually needs it solved can just bootstrap conda instead.

> I'm immensely grateful for the amount of work projects which are
> developed on Unix (and 3rd parties like Cristoph) put into supporting
> Windows. Far from dismissing that, I want to avoid making things any
> harder than they already are for such people - current wheels are no
> more complex to distribute than wininst installers, and I want to keep
> the impact on non-Windows projects at that level. If I come across as
> ungrateful, I apologise.

The only problem I want to explicitly declare out of scope for wheel files is the one the wininst installers can't handle cleanly either: the subset of Christoph's installers which need a shared external binary dependency, and any other components in a similar situation.

Using wheels or native Windows installers can get you in trouble in that case, since you may accidentally set up conflicts in your environment. The solution is curation of a software stack built around that external dependency (or dependencies), backed up by a packaging system that prevents conflicts within a given local installation.

The mainstream Linux distros approach this problem by mapping everything to platform-specific packages and trying to get parallel installation working cleanly (a part of the problem I plan to work on improving post Python 3.4), but that approach doesn't scale well and is one of the factors responsible for the notorious time lags between software being released on PyPI and it being available in the Linux system package managers (streamlining that conversion is one of my main goals for metadata 2.0).

The conda folks approach it differently by using hashes to lock in *exact* versions of dependencies. From a compatibility management perspective, it's functionally equivalent to bundling a complete stack into a single monolithic installer, but has the benefit of still making it easy to only install the pieces that you actually need locally.

Regards,
Nick.

>
> Paul
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig