I guess this depends on how explicit you wan to be. PEP 517 is not enabled unconditionally, only when the project root contains pyproject.toml. The problem is that other projects (unrelated to packaging) take advantage of the file format’s existence, resulting in people “buy in” the situation unknowingly. Things are already explicit from pip’s point of view, but (unanticipated?) third-party usages of the same spec unrelated to PEP 517 muddles the situation.

Tzu-ping Chung (@uranusjr)
Sent from my iPhone

On 28 Jan 2019, at 04:37, Bernat Gabor <gaborjbernat@gmail.com> wrote:

This is more an argument to not use pep-517 unless people explicitly specify the backend, at which point they acknowledge to buy in that the cwd is not on sys path (and they need to alter their packaging code accordingly).

On Sun, 27 Jan 2019, 20:31 Tzu-ping Chung <uranusjr@gmail.com wrote:
Yes, of course people can. I think the problem is that common Python installations defaults to adding the cwd into sys.path, so people expect this to “just work”.  A PEP 517 not doing it is not intuitive unless you follow Python packaging closely, and most would simply assume pip is broken.

(Incidentally, the Windows embedded distribution also does not include the cwd in sys.path by default, and I do see confused people on discussion boards or issue trackers from time to time complaining things don’t work. The embedded distribution’s user base is quite small, however, so this is not really a problem.)

Tzu-ping Chung (@uranusjr)
Sent from my iPhone

On 28 Jan 2019, at 04:17, Bernat Gabor <gaborjbernat@gmail.com> wrote:

It feels to me that importing from setup.py level is a code smell and should be avoided (I mean what happens if you move the source into src layout, do you now want tools to also append the src folder additionally if exists?). Can't people just inject at the start of setup.py into sys path the cwd if they really want to go with it? (feels superior over pinning pip to 18.0).

On Sun, Jan 27, 2019 at 7:48 PM Paul Moore <p.f.moore@gmail.com> wrote:
On Sun, 27 Jan 2019 at 14:39, Thomas Kluyver <thomas@kluyver.me.uk> wrote:
> I think the rule about the CWD not being on sys.path only applies to loading a proper PEP 517 build backend when that's specified in pyproject.toml.

I'm not entirely sure what you're intending when you refer to a
"proper PEP 517 build backend". The setuptools backend is a perfectly
acceptable one, the only difference here is that we default to it in
pip when the project hasn't specified a backend. The reason we do that
is simple - if we don't, we're unlikely to be able to remove the
legacy setuptools code any time in the foreseeable future, as there
will be essentially no incentive for existing projects to specify a
setuptools backend. Also, we'd get basically no testing of the new PEP
517 code, for the same reason. (Yes, it would have been better to test
before release, but we've never had any success getting beta testers
for pip). We could have made PEP 517 entirely opt-in, but (a) that
would have drastically slowed down adoption, and (b) implicit in
everything we'd discussed was the assumption that there would be a
setuptools backend that was semantically equivalent to running
setup.py the way we currently did, so there was no need to. Clearly
that assumption turned out to be wrong, and that's where the issues
for our users arose.

Having said this, I don't have any problem with changing the default
backend used by pip. In fact, if the setuptools project don't view it
as a goal of their existing backend to replicate setup.py behaviour,
then I think it's pretty much essential that we change. Pip needs to
default to a PEP 517 backend that behaves the same as pip's legacy
code - that's essential for backward compatibility if we're to remove
or deprecate the legacy setup.py code path. The only problem is that
right now there doesn't appear to be a backend that suits our

Paul Ganssle suggested that setuptools could provide an alternative
("legacy") setuptools backend that preserved full setup.py
compatibility (including having the current directory on sys.path).
That seems to me to be a reasonable solution, and I appreciate the
offer. In practice, the legacy setuptools backend doesn't *have* to be
implemented by setuptools itself - it could be a separate project if
setuptools don't want to maintain it themselves. But IMO it would be
better if the setuptools team are OK with maintaining it.(At a pinch,
pip could even implement that backend internally, but adding pip to
every build environment seems a bit heavyweight).

So in terms of practical resolution of the problems people are hitting
since pip 19.0 was released, I don't think there's a significant issue
here. We need a new pip release, sure, and someone needs to write the
new backend, but I don't think either of those tasks are huge.

What is more difficult is the question of whether the PEP should
change. As Chris pointed out, the previous discussion ended up saying
that the build directory should not be on sys.path, but acknowledged
that mandating that might cause issues. So the question now is, are
the issues we've seen big enough that we want to change PEP 517 to say
that the build directory *should* be on sys.path? We don't have to
answer that question urgently (the urgent resolution is with a legacy
backend, as noted above) but we do have to decide one way or another.
And here the question is whether enforcing projects to adopt the "good
practice" of not relying on being able to import the project at build
time is worth the negative impact of requiring what appears to be a
non-trivial number of projects to change their build code. Or whether
there's an "intermediate" option (the discussions Chris linked to
included talk of adding another pyproject.toml value which would be a
list of directories the frontend should add to sys.path when calling
hooks, for example). But that discussion shouldn't be made under a
false impression of urgency - it's not on the critical path for fixing
pip's immediate issue.

Distutils-SIG mailing list -- distutils-sig@python.org
To unsubscribe send an email to distutils-sig-leave@python.org
Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/SG2VB3RFGHTUHYB2CMTS2WN3RUDTS3KB/