[Distutils] What is the official position on distutils?

Brett Cannon brett at python.org
Mon Aug 29 15:52:09 EDT 2016


On Sun, 28 Aug 2016 at 16:55 Sylvain Corlay <sylvain.corlay at gmail.com>
wrote:

> Distutils seems to be the de-facto standard for building extension modules
> for Python and it is used for most of the third-party extensions out there.
> I don’t think that it is reasonable to declare that it is now only meant
> for Python itself.
>

Sorry, I didn't mean to suggest distutils was suddenly going to change or
disappear, but I did want a direction/focus for what we want distutils to
become long-term as we have this discussion of what to do with distutils
any time a change is proposed. At this point distutils is practically a
black box of code as no one wants to touch out of fear of breaking the
world and that's not sustainable.


> I actually see a contradiction in pointing out some lack of involvement in
> the library to oppose well needed improvements, especially for a tool that
> has so much adoption.
>

It's not a contradiction because new features != maintenance. And the usage
of distutils makes its lack of maintenance even more scary. I guess my key
point from what I said in my previous email is that we shouldn't
perpetually ignore the fact that distutils is not to a level of support or
quality that any of us are happy with and yet so many people rely on.

For me this means we should prioritize getting our build API defined and
sdists standardized so that distutils (or a replacement) can be in the
stdlib to build Python itself only, and after that you get your build tool
from PyPI. And if that means setuptools swallows/subsumes distutils then
that's fine with me, but my key point is we are not maintaining distutils
in a way I think any of us would choose to now and so I'm trying to prod
this question to start thinking about how to fix the maintenance problem
long-term.


>
> As we discussed earlier, even though it is not a concern with C, checking
> for availability of a compiler flag becomes crucial when building
> extensions with C++ since new flavors of the language emerge every couple
> of years now. It is important to be able to output meaningful error
> messages when the compiler does not support C++[11/14/17] features if they
> are needed for a given extension. This is a new aspect of the landscape in
> this area.
>
> Finally, adding this method is a very straightforward change. `has_flag`
> simply comes aside `has_function` as a method of ccompiler. I don't see a
> more natural place for it. It would be a very weird design decision in my
> opinion to not add it there, and instead to add it to distutils.ccompiler
> by monkeypatching it in setuptools.
>

Honestly I have no comment on your feature, Sylvain, and I'm sorry your
proposal happens to be the catalyst to this discussion. I'm just trying to
get a general alignment from the PyPA group as the "distutils problem"
comes up and has the same points made every time with no general decision
on how to handle it long-term.

-Brett


>
> Thanks,
>
> Sylvain
>
> On Mon, Aug 29, 2016 at 12:36 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>
>> On 28 August 2016 at 18:05, Brett Cannon <brett at python.org> wrote:
>> > The discussion of Sylvain's proposed changes to distutils suggests that
>> > there isn't a clear-cut agreement or position of this SIG -- and thus
>> Python
>> > -- on changes to distutils, its future, etc. Is there an official
>> position
>> > I'm not aware of? If not, could we get one so we know how to handle any
>> more
>> > proposed changes to distutils?
>> >
>> > For me personally (and to start this conversation if necessary), I view
>> > distutils as the package in the stdlib that handles building extension
>> > modules in the stdlib for Python. That view means that if something
>> doesn't
>> > service that explicit goal then it shouldn't go into distutils. Now
>> that we
>> > are moving down the road towards making the build step configurable I'm
>> fine
>> > with saying that distutils is not expected to work for people other than
>> > Python itself and others can use setuptools, enscons, or other projects
>> > while we continue to improve the situation to where the build system is
>> just
>> > something you specify in pyproject.toml to generate your wheel. IOW
>> > distutils is only exposed publicly because Python doesn't hide
>> anything, but
>> > making it useful for the general case of direct usage by people is a
>> > non-goal of the package.
>>
>> FWIW, my view is:
>>
>> * distutils handles building of Python packages, both in the stdlib and
>> outside
>> * in practice, setuptools is almost always used with distutils, and
>> any proposed change to distutils could be made to setuptools just as
>> easily
>> * patches for distutils typically suffer from a lack of maintainers
>> motivated to work on them
>> * there are a few developers with a strong investment in distutils,
>> who have code that is deeply entwined with the internals of distutils,
>> and changes to distutils come with a high risk of causing issues for
>> those people
>> * these developers *don't* use setuptools, so changes in setuptools
>> don't tend to suffer from the same level of backward compatibility
>> constraints
>> * implementing changes in setuptools is *far* easier than doing so in
>> distutils, for the above reasons
>> * changes in setuptools can be used with any version of Python,
>> distutils changes are only available to people with no need to support
>> anything but the latest version (which in practice is a vanishingly
>> small group)
>> * as a result, there would have to be an *extremely* compelling reason
>> to make a change in distutils rather than in setuptools - sufficient
>> to justify the additional risk, the extra developer effort needed, and
>> the fact that any such change is only going to benefit users of newer
>> versions of Python
>>
>> The reality is simply that distutils is in a somewhat unhappy place,
>> with too few interested maintainers and serious compatibility problems
>> to deal with, but that's not exactly news to anyone. I don't think
>> that a formal statement will help much. Past history has shown that
>> making pronouncements like "distutils is frozen" has not helped anyone
>> - so we're just trying to work with the status quo.
>>
>> With regard to Sylvain's proposed changes, I haven't really seen any
>> arguments justifying making the change in distutils rather than in
>> setuptools - the last suggestion I saw was to make the change in
>> *both* places, but I don't see any particular reason (beyond abstract
>> "it makes more sense to have the change in distutils" arguments) for
>> not just targeting setuptools. I may be missing something (as I'm not
>> personally in a position where I'd be comfortable reviewing the
>> change) but it seems to me that it's a bit of a non-issue - just
>> implement in setuptools and move on. The rhetoric about "this means
>> that distutils is dead" doesn't feel particularly helpful.
>>
>> Paul
>>
> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20160829/9b2e7236/attachment-0001.html>


More information about the Distutils-SIG mailing list