[Distutils] Extracting distutils into setuptools

[Paul Moore]

I use distutils to build a C program that embeds Python, for making
standalone launchers. I'm not at all clear how I'd get all the
information I needed to build a compatible Python (including the "find
the appropriate C compiler" dance) manually - I'd probably end up with
something that didn't work except with specific Python versions or
setups.

Having said that, I assume that the relevant APIs or equivalents would
be exposed via a future setuptools that included distutils (as it
would need them internally to build extensions). Alternatively, I'd
support refactoring these aspects of distutils out into a standalone
library.

Paul

On 28 September 2017 at 15:55, Daniel Holth <dholth at gmail.com> wrote:
> Enscons uses parts of distutils to get compiler flags and so on but does not
> use Extension() to do the actual compiling. There might be a cleaner way to
> do it that I was not able to find. There could be a cleaner separation
> between parts of distutils related to how Python itself was compiled and the
> part that does the actual compiling. The sysconfig module attempts to do at
> least part of this.
>
> distutils also knows about some compiler tricks like generating symbol
> tables for Windows, that are not easy to re-use or invoke separately.
> Someone could catalog these tricks and techniques and refactor the useful
> ones into a reusable library. I don't have a great sense of how to do it.
>
> https://bitbucket.org/dholth/enscons/src/tip/enscons/cpyext.py?at=default&fileviewer=file-view-default
>
> On Wed, Sep 27, 2017 at 11:36 PM Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>> On 28 September 2017 at 06:00, xoviat <xoviat at gmail.com> wrote:
>> > That's actually an interesting idea though: for Python 3.7 distutils ->
>> > _distutils (and then setuptools is required for building). For/against?
>>
>> distutils works fine for its original purpose (building components for
>> the system Python in Linux distros), so we still need to avoid
>> breaking that. setuptools is only essential if you want full support
>> for modern *Python* level packaging features (PEP 376 install
>> metadata, venv compatibility, wheel files, etc), and a lot of Linux
>> system components simply don't worry about those things, and rely on
>> their system level equivalents instead (e.g. the RPM/deb databases,
>> chroots and containers, RPM/deb files)
>>
>> However, what *could* be interesting is a proposal to move distutils
>> to the "ensurepip" model, where rather than maintaining distutils
>> directly as part of CPython, the CPython build process instead runs
>> setuptools/distutils from a bundled wheel file. Doing that would
>> entail having setuptools actually start installing a copy of distutils
>> into site-packages: older CPython releases would ignore it by default
>> (since the stdlib version would shadow it), while 3.7+ would offer
>> either "python3 -m ensuredistutils" or "python3 -m ensuresetuptools"
>> (bikeshed to be painted via the PEP process) to install the
>> setuptools-provided version.
>>
>> I'm not claiming actually doing that would be particularly easy - I
>> just think it's the most viable path to get us away from the current
>> version coupling between the build infrastructure in distutils and the
>> runtime support infrastructure in the rest of the standard library,
>> and to avoid maintaining two distinct copies of distutils indefinitely
>> (one in the stdlib, one in setuptools).
>>
>> That approach wouldn't even entail any *new* bundling at the CPython
>> level, as while it's currently formally an implementation detail
>> (pending potential removal in a post-PEP-517 world), setuptools is
>> already bundled as part of the support infrastructure for ensurepip:
>> https://github.com/python/cpython/tree/master/Lib/ensurepip/_bundled
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>