[Distutils] Comments on PEP 426

[Nick Coghlan]

On 1 Sep 2013 01:04, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> Oscar Benjamin <oscar.j.benjamin <at> gmail.com> writes:
> >
> > The difference between this
> >
> >     # setup.py
> >     if sys.argv[1] == 'install':
> >         from myproj.build import build
> >         build()
> >
> > and something like this
> >
> >     # setup.cfg
> >     [install]
> >     command = "from myproj.build import build; build()"
>
> Well, sure, this is a rather silly way to use a ini-style declarative
> format. I agree we wouldn't gain anything if setup.cfg files ended up
> written like this.
>
> > > I tend to disagree. Such bugs are not fixed, not because they
shouldn't /
> > > can't be fixed, but because distutils isn't really competently
maintained
> > > (or not maintained at all, actually; Éric sometimes replies on bug
entries
> > > but he doesn't commit anything these days).
> >
> > So is that particular issue a lost cause?
>
> Why would it be?
>
> > > The idea that "distutils shouldn't change" was more of a
widely-promoted
> > > propaganda item than a rational decision, IMO. Most setup scripts
wouldn't
> > > suffer from distutils changes or improvements; the few that *may*
suffer
> > > belong to large projects which probably have other items to solve
when a
> > > new Python comes out, anyway.
> >
> > It's not just the setup script for a particular project. It's the
> > particular combination of compilers and setup.py invocations used by
> > any given user for any given setup.py from each of the thousands of
> > projects that do anything non-trivial in their setup.py.
>
> I don't know what those "thousands of projects" are. Most Python projects
> don't even need a compiler, except Python itself.
>
> > For example
> > in the issue I mentioned above the spanner in the works came from PJE
> > who wanted to use --compiler=mingw32 while surreptitiously placing
> > Cygwin's gcc on PATH:
> > http://bugs.python.org/issue12641#msg161514
> > It's hard for distutils to react to outside changes in e.g. external
> > compilers because of the need to try and prevent breaking countless
> > unknown and obscure setups for each end user.
>
> This sounds like a deformation of reality. Most users don't have
> "unknown and obscure setups", they actually have quite standardized
> and well-known ones (think Windows, OS X, mainstream Linux distros).
>
> Sure, in some communities (scientific programming, I suppose) there
> may be obscure setups, but those communities have already grown their
> own bag of tips and tricks, AFAIK.

This perception is just wrong. Mac OS X is a mess due to clang vs gcc vs
homebrew vs macports vs Xcode, Windows is a mess due to mingw and cygwin
and platform SDKs and visual studio (Express or otherwise). Linux distros
are also ridiculous, with different gcc variants, library locations and
various other things, depending on flavour and version.

distutils itself is nigh impossible to work on, since it is underspecified,
and the fact it was added to the standard two years before unittest still
shows in its test suite. (This lack of test coverage is also a large part
of the reason setuptools *isn't* quite a drop-in replacement).

The key reason working on Python packaging has a tendency to chew
developers up and spit them back out isn't because the people trying to
work on it are incompetent, but because it's a genuinely hard problem where
the already formidable technical issues are dwarfed by even more complex
social ones. One of the most demotivating aspects comes from core
developers without extensive experience of the existing packaging ecosystem
failing to perceive the complexity of the problems to be addressed, and
insulting the competence of the people working on packaging tools as a
result. This is an aspect of the problem I was guilty of contributing to
myself until a year or so ago, and after a change in role at Red Hat led to
me learning the error of my ways, reducing it was one of my main
motivations for cutting python-dev out of the approval process for the
packaging related PEPs that don't immediately affect CPython or the
standard library.

While pure Python distributions are common, there's an awful lot of
"Python" code which includes C accelerator modules, wrappers around C or
C++ libraries, including Cython or SWIG generated versions of both. This
affects crypto, databases, networking, operating system interfaces,
graphics and other domains that rely heavily on non-Python code, not just
the scientific community.

setuptools definitely has its issues, but it's still substantially superior
to distutils, and has the critical virtue of behaving the *same* in all
currently supported versions of Python. Consistency across platform
versions is something you really want in a build tool, and is something a
standard library module like distutils can never provide. As one of the
most conservative Linux vendors, even Red Hat has acknowledged this key
point by creating the Red Hat Developer Toolset to provide a more
consistent build experience across different RHEL versions. Microsoft (with
Visual Studio) and Apple (with XCode) have long worked the same way.

Cheers,
Nick.

>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130901/0ef7a48d/attachment-0001.html>