[Distutils] distlib and wheel metadata

[Freddy Rietdijk]

> Here's an example I've just run into that involves a == dependency on
a public package: I have a library that needs to access some C API
calls on Windows, but not on other platforms. The natural way to do
this is to split out the CFFI code into its own package,
_mylib_windows_helper or whatever, that has zero public interface, and
have mylib v1.2.3 require "_mylib_windows_helper==1.2.3; os_name ==
'nt'".

You have a public library, that, depending on the platform, depends on a
public (helper) library that has no public interface? That doesn't sound
good to me. If you don't want to implement a public interface then it
should just be included in the main library because it is in the end a
requirement of the library. It's a pity you can't have a universal wheel
but so be it. Choosing to depend on an exact version of a package that has
no public interfance is in my opinion the wrong solution.

As I stated before, though perhaps not explicitly, I cannot think of *any*
good reason that one uses == in `install_requires`. Something like `>= 1.7,
< 1.8` should be sufficient. In the CFFI case that should be sufficient
unless you change your function signatures in a maintenance release (which
is bad). And in case of a metapackage like PyObjC this should also be
sufficient because it will downgrade dependencies when downgrading the
metapackage while still giving you the latest maintenance releases of the
dependencies.

Regarding 'application', 'library', and 'metapackage'. In Nixpkgs we
distinguish Python libraries and applications. Applications are available
for 1 version of the interpreter, whereas libraries are available for all
(supported) interpreter versions. It's nice if it were more explicit on say
PyPI whether a package is a library or an application. There are difficult
cases though, e.g., `ipython`. Is that an application or a library? As user
I would argue that it is an application, however, it should be available
for each version of the interpreter and that's why we branded it a library.

Metapackages. `jupyter` is a metapackage. We had to put it with the rest of
the Python libraries for the same reason as we put `ipython` there.
>From a distributions'
point of view I don't see why you would want to have them mentioned
separately.

On Thu, Feb 23, 2017 at 12:49 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Thu, Feb 23, 2017 at 12:44 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> > On 23 February 2017 at 18:37, Paul Moore <p.f.moore at gmail.com> wrote:
> >>
> >> On 23 February 2017 at 08:18, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> > I'm not a huge fan of having simple boolean toggles in metadata
> >> > definitions
> >> > (hence the more elaborate idea of two different kinds of dependency
> >> > declaration), but this may be a case where that's a good way to go,
> >> > since it
> >> > would mean that services and tools that care can check it (with a
> >> > recommendation in the spec saying that public index servers SHOULD
> check
> >> > it), while those that don't care would continue to have a single
> unified
> >> > set
> >> > of dependency declarations to work with.
> >>
> >> While boolean metadata may not be ideal in the general case, I think
> >> it makes sense here. If you want to make it more acceptable, maybe
> >> make it Package-Type, with values "application" or "library".
> >
> >
> > That gets us back into the world of defining what the various package
> types
> > mean, and I really don't want to go there :)
> >
> > Instead, I'm thinking in terms of a purely capability based field:
> > "allow_pinned_dependencies", with the default being "False", but actually
> > checking the field also only being a SHOULD for public index servers and
> a
> > MAY for everything else.
> >
> > That would be enough for downstream tooling to pick up and say "I should
> > treat this as a multi-component module rather than as an individual
> > standalone component", *without* having to inflict the task of
> understanding
> > the complexities of multi-tier distribution systems onto all component
> > publishers :)
>
> I'm still not sure I understand what you're trying to do, but this
> feels like you're trying to have it both ways... if you don't want to
> define what the different package types mean, and it's purely a
> capability-based field, then surely that means that downstream tooling
> *can't* make assumptions about what kind of package type it is based
> on the field? ISTM that from the point of view of downstream tooling,
> "allow_pinned_dependencies" carries literally no information, because
> all it means is "this package is on a public server and its
> Requires-Dist field has an == in it", which are things we already
> know. I can see how this would help your goal of educating uploaders
> about good package hygiene, but not how it helps downstream
> distributors.
>
> (Here's an example I've just run into that involves a == dependency on
> a public package: I have a library that needs to access some C API
> calls on Windows, but not on other platforms. The natural way to do
> this is to split out the CFFI code into its own package,
> _mylib_windows_helper or whatever, that has zero public interface, and
> have mylib v1.2.3 require "_mylib_windows_helper==1.2.3; os_name ==
> 'nt'". That way I can distribute one pure-Python wheel + one binary
> wheel and everything just works. But there's no sense in which this is
> an "integrated application" or anything, it's just a single library
> that usually ships in one .whl but sometimes ships in 2 .whls.)
>
> ((In actual fact I'm currently not building the package this way
> because setuptools makes it extremely painful to actually maintain
> that setup. Really I need the ability to build two wheels out of a
> single source package. Since we don't have that, I'm instead using
> CFFI's slow and semi-deprecated ABI mode, which lets me call C
> functions from a pure Python package. But what I described above is
> really the "right" solution, it's just tooling limitations that make
> it painful.))
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20170223/f1f8efa5/attachment.html>