Script bootstrapping executables on Windows
Hi, This is a Windows specific question, to give a bit of context I'm working in a studio where depending on the project we use different Python interpreters installed in different locations, e.g. Python 2.7.13, Python 2.7.14, Python 3.6. We set PATH, PYTHONHOME and PYTHONPATH accordingly depending the interpreter in use. Our Python packages are living atomically on the network and are added to the environment on a per project basis by extending PYTHONPATH. This is in contrast to using a monolithic virtual environment built with virtualenv or conda. Assuming it is compatible, a Python package might be used with any of the 3 aforementioned interpreters, e.g. yapf (a code formatter). Now, on Windows, if you for example *pip install yapf*, a yapf.exe boostrapping executable and a yapf-script.py file are being generated. The boostrapping executable seems to look for the yapf-script.py file and launch it using the absolute hardcoded interpreter path of the yapf-script.py shebang. Given the above we run into issues if for example yapf was deployed using Python 2.7.13 but the Python 2.7.14 interpreter is being used in the environment instead. We get a "failed to create process." error in that case. What we would like to do is not being tied to the absolute interpreter path but have it defined with a variable or just use #!python. I have tried to search for the above error in cpython source code and the installation directory without luck. I would like to know what module/package is responsible for generating the boostrapping executables to understand how it works and see if we can doctor it for our usage. Bests, Thomas
Hi, I hope that what follows will be useful for other people: after stepping through code for a few hours this morning, I ended up finding the location of the boostrapping executable. It (they) actually ship with setuptools, e.g. C:\Python27\Lib\site-packages\setuptools\cli-64.exe and get copied and renamed with the script name. The source code is here: https://github.com/pypa/setuptools/blob/master/launcher.c <https://github.com/pypa/setuptools/blob/master/launcher.c#L209>, and I was able to find the error mentioned in OP: https://github.com/pypa/setuptools/blob/master/launcher.c#L209 Cheers, Thomas On Wed, Nov 22, 2017 at 5:08 PM Thomas Mansencal <thomas.mansencal@gmail.com> wrote:
Hi,
This is a Windows specific question, to give a bit of context I'm working in a studio where depending on the project we use different Python interpreters installed in different locations, e.g. Python 2.7.13, Python 2.7.14, Python 3.6. We set PATH, PYTHONHOME and PYTHONPATH accordingly depending the interpreter in use.
Our Python packages are living atomically on the network and are added to the environment on a per project basis by extending PYTHONPATH. This is in contrast to using a monolithic virtual environment built with virtualenv or conda. Assuming it is compatible, a Python package might be used with any of the 3 aforementioned interpreters, e.g. yapf (a code formatter).
Now, on Windows, if you for example *pip install yapf*, a yapf.exe boostrapping executable and a yapf-script.py file are being generated. The boostrapping executable seems to look for the yapf-script.py file and launch it using the absolute hardcoded interpreter path of the yapf-script.py shebang.
Given the above we run into issues if for example yapf was deployed using Python 2.7.13 but the Python 2.7.14 interpreter is being used in the environment instead. We get a "failed to create process." error in that case.
What we would like to do is not being tied to the absolute interpreter path but have it defined with a variable or just use #!python. I have tried to search for the above error in cpython source code and the installation directory without luck. I would like to know what module/package is responsible for generating the boostrapping executables to understand how it works and see if we can doctor it for our usage.
Bests,
Thomas
participants (1)
-
Thomas Mansencal