[Distutils] it's happened - wheels without sdists (flit)

[Nick Coghlan]

On 31 Mar 2015 01:17, "Xavier Fernandez" <xav.fernandez at gmail.com> wrote:
>
> Fair enough, I didn't think of compiled wheels :)
> And having a clean way to run tests for the provided wheel is indeed an
other good point.

To elaborate on Donald's answer, one of our general requirements downstream
in distro land is the ability to rebuild the entire distro from source
without human intervention (for example:
https://beaker-project.org/docs-develop/user-guide/beaker-provided-tasks.html#distribution-rebuild
for Fedora & derivatives)

We need to build from source not just to ensure our binaries match the
published source code, but also because our build systems are designed to
let us *patch* the packages before we build them. This is what lets us
backport security updates, bug fixes, and sometimes even entire features
without needing to rebase a package on a new upstream release of a project.

We also like to be able to automatically *generate* package build
instructions from upstream metadata. Those tools aren't perfect today (too
much info is missing from current generation upstream metadata), but
they're a lot better than nothing. Historically those tools have relied on
"setup.py install" working on an unpacked sdist.

These days in Fedora we aim to rely on "pip install" working instead of
invoking setup.py directly, but there are over 15000 Python packages in
Fedora (and a similar number in Debian), and Fedora's policy only switched
to preferring indirection through pip for Python package builds last year
(AFAIK Debian still favours invoking setup.py directly, hopefully Barry
will correct me if I'm wrong).

But as long as those source package builds keep working downstream, we
don't really care what's happening behind the scenes in the upstream build
systems. It's mildly irritating when packaging a new release of an already
included project turns into a yak shaving expedition to package a whole new
build system as a dependency, but that's just one of the consequences of
running an open source integration project rather than writing everything
from scratch yourself.

It's also one of the reasons why d2to1 is such a neat hack - that presents
as plain distutils at sdist build time, while allowing the use of
distutils2 features at package definition time upstream.

Cheers,
Nick.

>
> On Mon, Mar 30, 2015 at 5:04 PM, Donald Stufft <donald at stufft.io> wrote:
>>
>>
>>> On Mar 30, 2015, at 10:58 AM, Xavier Fernandez <xav.fernandez at gmail.com>
wrote:
>>>
>>>> Wheels without sdists are likely a generally bad idea, downstream
>>>> redistributors are not going to like them.
>>>
>>>
>>> Why do you think that ? Wheels seem way simpler/saner than all the
possible things setup.py can do.
>>
>>
>> Wheels are simpler than a setup.py, because wheels are a binary format
and Wheels don’t need to
>> handle things like build software because it’s already been built.
However downstream redistributors
>> will not accept a Wheel as the source for a package because it is a
binary format. It doesn’t matter
>> if you can unzip it and there is pure python there, it is still a binary
format. So if you release only Wheels
>> you’re essentially saying that downstream redistributors will never
package your software (or any
>> software that depends on it).
>>
>> A few issues that Wheel only has:
>>
>> * If your project has a C extension, downstream redistributes need
access to the source code not the
>>   compiled code (as does anyone wanting to use the project on platform
you didn’t release for).
>>
>> * If your project has tests that don’t get installed, they should get
shipped as part of the sdist so that
>>   downstream can run them as part of their packaging activities to
ensure they didn’t break anything
>>   in the test suite. However if you’re installing from Wheel you can’t
do that.
>>
>> ---
>> Donald Stufft
>> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>>
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150331/7d1cdab0/attachment.html>