On Sun, 27 Jan 2019 at 14:39, Thomas Kluyver
I think the rule about the CWD not being on sys.path only applies to loading a proper PEP 517 build backend when that's specified in pyproject.toml.
I'm not entirely sure what you're intending when you refer to a "proper PEP 517 build backend". The setuptools backend is a perfectly acceptable one, the only difference here is that we default to it in pip when the project hasn't specified a backend. The reason we do that is simple - if we don't, we're unlikely to be able to remove the legacy setuptools code any time in the foreseeable future, as there will be essentially no incentive for existing projects to specify a setuptools backend. Also, we'd get basically no testing of the new PEP 517 code, for the same reason. (Yes, it would have been better to test before release, but we've never had any success getting beta testers for pip). We could have made PEP 517 entirely opt-in, but (a) that would have drastically slowed down adoption, and (b) implicit in everything we'd discussed was the assumption that there would be a setuptools backend that was semantically equivalent to running setup.py the way we currently did, so there was no need to. Clearly that assumption turned out to be wrong, and that's where the issues for our users arose. Having said this, I don't have any problem with changing the default backend used by pip. In fact, if the setuptools project don't view it as a goal of their existing backend to replicate setup.py behaviour, then I think it's pretty much essential that we change. Pip needs to default to a PEP 517 backend that behaves the same as pip's legacy code - that's essential for backward compatibility if we're to remove or deprecate the legacy setup.py code path. The only problem is that right now there doesn't appear to be a backend that suits our requirements. Paul Ganssle suggested that setuptools could provide an alternative ("legacy") setuptools backend that preserved full setup.py compatibility (including having the current directory on sys.path). That seems to me to be a reasonable solution, and I appreciate the offer. In practice, the legacy setuptools backend doesn't *have* to be implemented by setuptools itself - it could be a separate project if setuptools don't want to maintain it themselves. But IMO it would be better if the setuptools team are OK with maintaining it.(At a pinch, pip could even implement that backend internally, but adding pip to every build environment seems a bit heavyweight). So in terms of practical resolution of the problems people are hitting since pip 19.0 was released, I don't think there's a significant issue here. We need a new pip release, sure, and someone needs to write the new backend, but I don't think either of those tasks are huge. What is more difficult is the question of whether the PEP should change. As Chris pointed out, the previous discussion ended up saying that the build directory should not be on sys.path, but acknowledged that mandating that might cause issues. So the question now is, are the issues we've seen big enough that we want to change PEP 517 to say that the build directory *should* be on sys.path? We don't have to answer that question urgently (the urgent resolution is with a legacy backend, as noted above) but we do have to decide one way or another. And here the question is whether enforcing projects to adopt the "good practice" of not relying on being able to import the project at build time is worth the negative impact of requiring what appears to be a non-trivial number of projects to change their build code. Or whether there's an "intermediate" option (the discussions Chris linked to included talk of adding another pyproject.toml value which would be a list of directories the frontend should add to sys.path when calling hooks, for example). But that discussion shouldn't be made under a false impression of urgency - it's not on the critical path for fixing pip's immediate issue. Paul