[Distutils] Towards a simple and standard sdist format that isn't intertwined with distutils

[Oscar Benjamin]

On Wed, 7 Oct 2015 18:51 Donald Stufft <donald at stufft.io> wrote:


On October 7, 2015 at 1:27:31 PM, Nathaniel Smith (njs at pobox.com) wrote:
> On Mon, Oct 5, 2015 at 6:51 AM, Donald Stufft wrote:
> [...]
> > I also don't think it will be confusing. They'll associate the VCS
thing (a source release)
> as something focused on development for most everyone. Most people won't
explicitly
> make one and nobody will be uploading it to PyPI. The end goal in my mind
is someone produces
> a source wheel and uploads that to PyPI and PyPI takes it from there.
Mucking around with
> manually producing binary wheels or producing source releases other than
what's checked
> into vcs will be something that I suspect only advanced users will do.
>
> Of course people will make source releases, and should be able to
> upload them to PyPI. The end goal is that *pip* will not use source
> releases, but PyPI is not just there for pip. If it was, it wouldn't
> even show package descriptions :-).
>
> There are projects on PyPI right now, today, that have no way to
> generate sdists and will never have any need for "source wheels"
> (because they don't use distutils and they build "none-any" wheels
> directly from their source). It should still be possible for them to
> upload source releases for all the other reasons that having source
> releases is useful: they form a permanent record of the whole project
> state (including potentially docs, tests, working notes, etc. that
> don't make it into the wheels), human users may well want to download
> those archives, Debian may prefer to use that as their orig.tar.gz,
> etc. etc.
>
> And on the other end of the complexity scale, there are projects like
> numpy where it's not clear to me whether they'll ever be able to
> support "source wheels", and even if they do they'll still need source
> releases to support user configuration at build time.

We must have different ideas of what a source release vs source wheel would
look like, because I'm having a hard time squaring what you've said here
with
what it looks like in my head. In my head, source releases (outside of the
VCS
use case) will be rare and only for very complex packages that are doing
very
complex things. Source wheels will be something that will be semi mandatory
to
being a well behaved citizen (for Debian and such to download) and binary
wheels will be something that you'll want to have but aren't required. I
don't
see any reason why source wheels wouldn't include docs, tests, and other
misc
files.

I picture building a binary wheel directly being something similar to using
fpm
to build binary .deb packages directly, totally possible but unadvised.

Having talked to folks who deal with Debian/Fedora packages, they won't
accept
a binary wheel as the input source and (given how I explained it to them)
they
are excited about the concept of source wheels and moving away from dynamic
metadata and towards static metadata.


Your idea of an sdist as something that has fully static build/runtime
dependency metadata and a one to one correspondence with binary wheels is
not a usable format when releasing the code for e.g. numpy 1.10. It's fine
to say that pip/PyPI should work with the source in some other distribution
format and numpy could produce that but it means that the standard tarball
release needs to be supported some how separately. Numpy should be able to
use PyPI in order to host the tarball even if pip ignores the file.

If numpy released only source wheels then there would be more than one
source wheel for each release corresponding to e.g. the different ways that
numpy is linked. There still needs to be a way to release a single file
representing the code for the release as a whole.

--
Oscar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20151007/716ed5ea/attachment-0001.html>