[Distutils] Builders vs Installers

[Donald Stufft]

On Mar 27, 2013, at 11:44 PM, PJ Eby <pje at telecommunity.com> wrote:

> On Wed, Mar 27, 2013 at 10:09 PM, Donald Stufft <donald at stufft.io> wrote:
>> Right, which is why I think the ability to install from a raw source is a good feature for an installer, but not for the dependency metadata.
> 
> Sure - I never said the dependency metadata had to be able to say
> *where* you get the raw source from, just that the tool for resolving
> dependencies needed to be able to process raw source into something
> installable.
> 
>> Following that we just need a standard way for a raw source tarball to declare what it's builder is, either via some sort of file that tells you that, or a build script , or something along those lines.
> 
> Yep.  Static configuration is a *must*, here, though, as we want to
> move away from arbitrary setup script writing by package authors: in
> general they are really bad at it.  A lot of setuptools' odd
> build-time features (like sandboxing) exist specifically because
> people write whatever crap they want in setup.py and have zero idea
> how to actually use/integrate with distutils.
> 
> One interesting feature that would be possible under a
> configuration-based system is that you could actually have an
> installer with a whitelist or blacklist for build tools and
> setup-requires, in order to prevent or limit untrusted code execution
> by the overall build system.  This would make it slightly more
> practical to have, say, servers that build wheels, such that only
> tools the servers' owners know won't import or run arbitrary code are
> allowed to do the compiling.  (Not that that should be the only
> security involved, but it'd be a cool first-tier sanity check.)
> 
> (Interestingly, this is also an argument for having a separate
> "tests-require-dist" in metadata 2.0, since testing tools *have* to
> run arbitrary code from the package, but archivers and builders do
> not.)

catalog-sig without an argument? Is this a first? ;)

> 
> 
> Nick wrote:
>>> However, as I've said elsewhere, for metadata 2.0, I *do not* plan to
>>> migrate the archiving or build steps away from setup.py. So "give me
>>> an sdist" will be spelled "python setup.py sdist" and "give me a wheel
>>> file"  will be spelled "python setup.py bdist_wheel".
> 
> Works for me.  Well, sort of.  In principle, it means you can grow
> next generation build systems that use a dummy setup.py.
> 
> In practice, it means you're still gonna be relying on setuptools.
> (Presumably 0.7 post-merge, w/bdist_wheel support baked in.)  At some
> point, there has to be a new way to do it, because the pain of
> creating a functional dummy setup.py is a really high barrier to entry
> for a build tool to meet, until all the current tools that run
> setup.py files go away.
> 
> IMO it'd be better to standardize this bit *now*, so that it'd be
> practical to start shipping projects without a setup.py, or perhaps
> make a "one dummy setup.py to rule them all" implementation that
> delegates everything to the new build interface.
> 
> I can certainly understand that there are more urgent priorities in
> the short run; I just hope that a standard for this part lands
> concurrent with, say, PEP 439 and distlib ending up in the stdlib, so
> we don't have to wait another couple years to begin phasing out
> setuptools/distutils as the only build game in town.
> 
> I mean, it basically amounts to defining some parameters to
> programmatically call a pair of sdist() and bdist_wheel() functions
> with, and a configuration syntax to say what distributions and modules
> to import those functions from.  So it's not like it's going to be a
> huge time drain.  (Maybe not even as much has already been consumed by
> this thread so far.  ;-) )
> 
> 
> Nick also wrote:
>>> There's also an interesting migration problem for pre-2.0 sdists,
>>> where we can't assume that "python setup.py bdist_wheel && pip install
>>> <created wheel>" is equivalent to "python setup.py install": projects
>>> like Twisted that run a post-install hook won't install properly if
>>> you build a wheel first, since the existing post-install hook won't
>>> run.
>>> 
>>> It's an interesting problem, but one where my near term plans amount
>>> to "document the status quo".
> 
> Yeah, it's already broken and the new world order isn't going to break
> it any further.  Same goes for allowing pip to convert eggs; the ones
> that don't work right due to bad platform tags, etc. *already* don't
> work, so documenting the status quo as a transitional measure is
> sufficient.  Heck, in general, supporting backward compatible stuff
> that suffers from the same problems as the stuff it's being backward
> compatible with is a no-brainer if it lets people get on the new so we
> can phase out the old.
> 
> (Which is why I love that Vinay is looking into how to make wheels
> more usable for some of eggs less-frequent but still important use
> cases: it makes it that much easier to tell someone they don't need to
> stay on setuptools to do the same stuff.)


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130327/01893ff7/attachment-0001.pgp>