[Distutils] PEP 517: Build system API

Nick Coghlan ncoghlan at gmail.com
Sat Nov 26 02:52:11 EST 2016

On 25 November 2016 at 12:26, Robert Collins <robertc at robertcollins.net> wrote:
> On 25 November 2016 at 14:04, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> The bad reputation of ".pth" doesn't generally stem from their normal
>> usage (i.e. just listing absolute or relative directory names that the
>> import system will then append to __path__).
>> Rather, it stems from this little aspect: "Lines starting with import
>> (followed by space or tab) are executed." (from
>> https://docs.python.org/3/library/site.html )
> I think its also worth exploring a targeted, modern namespace aware replacement.

Right. For a long time I thought "existing .pth files, only with the
import line execution deprecated" was the only approach that stood any
chance whatsoever of being adopted in a reasonable time frame, but we
can technically use the "executable .pth file" trick ourselves to let
people opt in to a new sys.path extension format on earlier Python

> That is - there are two, related, use cases for .pth files vis-a-vis
> package installation. One is legacy namespace packages, which AFAICT
> are still in use - the migration is "complicated". The second is
> arguable a variant of that same thing: putting the current working dir
> into the PYTHONPATH without putting it in PYTHONPATH.

Third case: making zip archive contents available to applications
without unpacking the archive first.

> The former case may be sufficiently covered by (forget the pep #) that
> we don't need to do anything,

PEP 420, and I believe the only thing that can get tricky there now is
figuring out how to do things in such a way that they work with both
old-style namespace packages and the automatic Python 3 ones.

> and the latter is certainly something
> that we should be able to deliver without needing the turing complete
> capability that you're suggesting. Something data driven rather than
> code driven.

While I agree with this, I think there's two pieces to it:

- how to handle the data-driven use cases that we're entirely fine with
- how to handle the arbitrary-code-execution use cases that we'd like
to discourage, but don't want to make entirely impossible

The problem with proposing an entirely new implicit path manipulation
format is that if we deprecate ".pth" files entirely, then that hits
*both* of those groups equally, while if we don't deprecate them at
all, then neither group has an incentive to migrate to the new system.

By contrast, if we only propose deprecating "import" lines in ".pth"
files, and also propose a more explicit approach to automatic code
execution at interpreter startup, then it's only folks relying on the
arbitrary-code-execution feature that would incur any migration costs.
The language level pay-offs to justify that cost would be:

- simplification of the current systems for implicit code execution at start-up
- making ".pth" files less dangerous as a format


Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

More information about the Distutils-SIG mailing list