But with your proposed change, wouldn’t this mean that building a project for system-wide installation would be broken by having a venv active?
At least for the use case where just the output of `python-config --libs` is used to determine the correct libpython version, nothing should break: `--libs` just returns the names of the library, and its up to the linker to actually find the lib. (Sidenote: the location of libpython is always the same. libpython is not copied to the venv, and my proposal here does not want to change that) For the other uses of `python-config`... I am not sure. Let's maybe ask: "What do we expect python-config to return in the venv for each of its options? and, assuming we get something path-like as response: "Is it bad if we get a response that points to something in that venv?". The command line switches of `python-config` are: - `--includes`: If we point to include files in the venv, this should not break anything, because the includes of the venv and system-wide should be the same. - `--prefix`: prints the prefix (base directory) under which python can be found. If I am in a venv, I actually would expect to get the python of the venv here. (So the current behavior is quite unexpected in my opionion). Is it bad if we get the venv python? That should be discussed. - `--exec-prefix`: print the prefix used for executable program directories. See `--prefix` - `--cflags`: prints the C compiler flags, so that is like `--includes` plus other compiler options. - `--abiflags`: Nothing path-like. No danger here. - `--extension-suffix`: Nothing path-like. No danger here. - `--libs`: Just returns linker flags, no absolute paths => nothing breaks - `--configdir`: path to the configuration directory under which the Makefile, etc. can be found. Since we don't link/copy that into the venv, I'd expect to get the system-wide configdir location that this venv was made from. - `--ldflags`: returns what `--libs` returns, plus what --configdir returns
In fact, looking at the virtualenv issue, it looks like that’s what people are actually asking for: they’re trying to use a package manager to, e.g., “brew install opencv”, and they want it to ignore the package-managed Python and instead build against the active virtualenv. That seems like a recipe for disaster (except for people who only have a single user and a single env that they keep activated 100% of the time, in which case, do they even need virtual environments?).
Can you elaborate on that? I must say I did not really understand why the user required a venv while brewing opencv, and hence did not give much thought about the consequences.
[...] But is there a design that would allow that to work for you, without silently breaking things for people mistakenly trying to run global installs against a venv? Maybe a way to explicitly ask for python-config as part of the venv creation, or a command to switch it on the fly, or even just a much simpler (and clearly documented) way to do the “magic” so you don’t have to waste a weekend trying to hack it into shape?
The workaround for cocotb consists of using `distutils.sysconfig` to obtain the essentials of what `python-config --libs` gives. However then we have to piece that together to a usable linker flag, whereas `python-config` returns it in a ready-to-use form for passing it to a compiler/linker. So maybe a solution would be to let distutils.sysconfig return information in the same form as python-config?
Also, it looks like the way virtualenv solved this was not to copy the script from the live Python, but to embed a custom script in virtualenv [...] did you check whether there was a reason they did it that way?
Good question. I have tagged Paul (madprog) in the comment under the virtualenv issue. If we're lucky he will see this and comment here. Unfortunatly github does't allow direct messages to users. I could also try and comment on the PR that added this to virtualenv aswell. FYI, I am out of office until Monday, so responses to this thread might be delayed.