Hi all,
I have been coaching a new contributor to SciPy and we are being routinely bitten by weird problems on import when trying to follow the development installation instructions. At this point, I'm not so concerned about getting things to work: I know that, with enough trial-and-error, we will manage at some point. Instead, I'd like to try to have a more solid mental model of why things fail (if that's even possible).
I am undecided and hesitant on how to proceed here since I feel I'm trying to solve too many problems at the same time. In no particular order:
1) I could try to get help on the specific errors we're getting. However, these are of the difficult-to-debug kind (undefined symbols and such) and I anticipate that we would end up in something like "you have a borked SciPy installation, please start from scratch" or "use conda instead".
2) I'd love to get a better understanding of what's the role of `conda develop .` in the development instructions, and how it's different from `pip install -e .`. I tried to look around and the only references I see about `conda develop` are the manual pages, and an old issue pointing to lack of documentation https://github.com/conda/conda-build/issues/1992
3) I've been asking myself whether SciPy would consider moving to an src-layout. I did a quick search on the issue tracker and the mailing list archives, and don't see any references. I know this is a contentious topic though, so it would be a pity to start a flame war about it, and that's definitely not my intention. I'm just wondering if at least it would help us avoid a family of problems: the ones appearing when folks do `import scipy` from the root directory.
4) We could continue the conversation we started in https://github.com/scipy/scipy/issues/12633 and try to see if there's a "happy path" that folks can follow given some assumptions (modern Linux distro/WSL2, compilers coming from either apt or conda-forge, no dangling Python installations) to reduce the installation instructions to the minimum (which could be `pip install -e . && pytest`).
Apologies if this rambling is not particularly helpful, hope I can get some help on these various points.
Best,
Juan Luis