On Fri, Aug 25, 2017 at 2:26 PM, xoviat <xoviat@gmail.com> wrote:
I'm more or less persuaded by Nathaniel's argument that the source directory shouldn't be on sys.path
I do too. There should be an option in pyproject.toml to disable this behavior though so that numpy can build itself.
My original proposal was to leave the srcdir off of sys.path, and then have a key in pyproject.toml like: [build-system] backend-python-path = ["."] Frontends would then do something like: os.chdir(srcdir) # This line is new to handle the above proposal: sys.path[:0] = [abspath(p) for p in config["build-system"].get("backend-python-path", [])] backend = resolve_entrypoint(config["build-system"]["build-backend"]) I don't have a strong opinion on whether we put this into PEP 517 (it's pretty light weight and doesn't interact with any other features AFAICT), or make it a followup PEP, or start out by deferring this option to a dedicated build backend, like: [build-system] requires = ["override_backend_path"] build-backend = "override_backend_path" [tool.override_backend_path] python-path = ["."] real-backend = "my_awesome:backend" These are all pretty similar. I think the big question for debate is: should sys.path be configurable, or not configurable? IIUC, the main argument for putting the source directory on the path was that extra configuration options are annoying, so we don't want one of those, but we do want to support in-tree backends (even though we expect that most projects won't use this), so we had better put the srcdir on sys.path. My feeling is that in practice, though, that the "no configuration, srcdir always on sys.path" approach is not going to hold up. So first, obviously, the hack above works just fine to make it configurable even if we don't put it in any PEP, so.... in fact it's configurable no matter what. Plus, sooner or later, someone's going to say "hey distutils-sig, I have a build backend package that I want to be able to bootstrap itself, AND I want to put my package inside src/ for reasons [1], wouldn't it be nice if I could put src/ on sys.path without jumping through hoops?". Or someone will release a new version of their build backend that adds a new dependency, or one of their transitive dependencies will release a new version that adds a new dependency, and it collides with some already-released package that uses that build-backend, and the project suffering the collision gets annoyed at being told they need to rearrange their source tree (retroactively, in already released versions!). And they'll come here and say "hey distutils-sig, can solve this problem once and for all?". And we'll be like... uh, fixing this would take what, <5 lines of new code in pip? Kinda hard to argue with that. So... it'll be configurable, one way or another. And if it's configurable... then the question is about whether the default configuration: is it srcdir on sys.path ("opt-out"), or srcdir not on sys.path ("opt-in"). And it seems to me that in this case, all the standard criteria say it should be opt-in. If it's opt-in, then everyone using build backends distributed on PyPI -- which we expect to be the vast majority of project -- never have to think about it and it does the right thing, with no risk of collisions or anything. In fact the only people who have to think about it are the ones implementing in-tree backends, and if you're already like, writing a whole backend and configuring your pyproject.toml to invoke it, then asking you to add one more line of configuration is really not a big deal. OTOH if it's opt-out, then it becomes Yet Another Bad Packaging Default, where conscientious package authors will fret about the risk of collisions and write blog posts about how every project needs to make sure to opt-out of this as a Best Practice, and I am so, so tired of those. -n [1] https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure -- Nathaniel J. Smith -- https://vorpus.org