[Distutils] PEP 517: Build system API

Nick Coghlan ncoghlan at gmail.com
Thu Nov 24 20:04:04 EST 2016

On 25 November 2016 at 09:10, Paul Moore <p.f.moore at gmail.com> wrote:
> Honestly, I don't see what's so bad about pth files. They are a
> standard supported Python approach. Maybe setuptools' use of them is
> messy? I recall it was possible to end up with a lot of clutter, but
> that was going back to the egg days, I'm not sure if it's like that
> any more. Why not just have a single pth file, maintained by the build
> tool, for all editable installs? Most users would then only have one
> or two pth files

The bad reputation of ".pth" doesn't generally stem from their normal
usage (i.e. just listing absolute or relative directory names that the
import system will then append to __path__).

Rather, it stems from this little aspect: "Lines starting with import
(followed by space or tab) are executed." (from
https://docs.python.org/3/library/site.html )

As such, rather than purely being a way of extending "__path__" with
additional directories, path files are *also* a way of
programmatically customising a particular Python installation, without
risking filename collisions the way sitecustomize and usercustomize

Since setuptools is the first or only encounter most folks have with
".pth" files, and it makes use of such lines to execute arbitrary code
at interpreter startup, the targeted criticism of "the 'import line'
feature of path files and the specific ways that setuptools uses that
feature are hard to debug when they break" morphed over time into the
blanket criticism of "path files are bad".

The main problem with fixing that is that folks really do rely on the
auto-execution feature, sometimes even without knowing it (e.g.
because setuptools uses it implicitly).

Therefore, in order to deprecate and remove the magic import lines,
we'd need to do something like adding an "__autorun__" directory to
site-packages and user site-packages as a long term replacement for:

- sitecustomize.py
- usercustomize.py
- executable import lines in path files

Supporting such a subdirectory in older Python versions could then be
bootstrapped via an "autorun.pth" file that provided the same start-up
logic that 3.7+ did.

I suspect this would actually be an improvement from a security
awareness perspective as well, as it would make it much clearer that
enabling implicit code execution at Python interpreter start-up is
trivial if you have write access to the site-packages or user
site-packages directory.


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Distutils-SIG mailing list