[Python-Dev] Single-file Python executables (was: Computed Goto dispatch for Python 2)

[Paul Moore]

On 29 May 2015 at 21:49, Glenn Linderman <v+python at g.nevcal.com> wrote:
>
> That looks interesting, I wonder what compilation environment it would need?
> I don't think I've even installed a C compiler on my last couple boxes, and
> the only version of a C compiler I have is, umm... M$VC++6.0, since I've
> moved to using Python for anything a 5 line batch file can't do...
>
>> One mildly annoying thing is that python3.dll is only installed in
>> <python install dir>\DLLs, which typically isn't on PATH.
>
> Ah, linking.... so I guess if I figured out how to create this binary, it
> would contain a reference to python3.dll that would attempt to be resolved
> via the PATH, from what you say, and typically fail, due to PATH seldom
> containing python3.dll.  The python launcher gets around that by (1) being
> installed in %windir%, and going and finding the appropriate Python (per its
> own configuration file, and command line parameters), and setting up the
> path to that Python, which, when executed, knows its own directory structure
> and can thus find its own python3.dll.
>
> The launcher, of course, adds an extra layer of process between the shell
> and the program, because it launches the "real" Python executable.
>
>> So actually using the limited API from your own application fails by
>> default.
>> Fixing that's mostly a user admin issue, though (and you can just link
>> to the full API and avoid the whole problem).
>
>
> Do I understand correctly that the "user admin issue" means "add the
> appropriate <python install dir>\DLLs to the PATH"?
>
> What I don't understand here is how linking to the full API avoids the
> problem... it must put more python library code into the stub executable?
> Enough to know how to search the registry to find the <python install dir>
> for the version of Python from which the full API was obtained? Or something
> else?

Sorry, I assumed more Windows/C knowledge than you have.

I'll work on this and produce proper binaries in due course, so you
can always wait for them. But you can build the stub with pretty much
anything, I suspect - I managed with MSVC 2010 and mingw. I'll add
some build docs and get it on github.

Using mingw

    gcc -Wall -O2 -o stub.exe stub.c -I <python home>\Include
C:\Windows\system32\python34.dll
    strip -s stub.exe

Using MSVC

    cl /Festub.exe /O2 stub.c /I<python home>\Include <python
home>\libs\python34.lib

Regarding the DLLs, yes the "user admin issue" is adding the right
directory to PATH. I used the phrase "admin issue" as it's the aspect
that's likely to be far harder than any of the technical issues :-)
The reason using the full API helps is that the full API references
python34.dll rather than python3.dll. And the Python installer puts
python34.dll on PATH automatically, as it's what the "python" command
uses. (For the people with more Windows knowledge, I know this is a
simplification, but it's close enough for now).

So there are two  options with the code I posted.

1. Build an exe that uses a specific version of Python, but which will
"just work" in basically the same way that the "python" command works.
2. Build an exe that works with any version of Python, but requires
some setup from the user.

Either approach requires that the Python DLL is on PATH, but that's
far more likely with the version-specific one, just because of how the
installer does things.

With extra code, the stub could locate an appropriate Python DLL
dynamically, which would simplify usage at the cost of a bit of fiddly
code in the stub.

This might be a useful addition to the zipapp module for Python 3.6.

Paul

PS Current launchers (py.exe, the entry point launchers from
pip/setuptools, etc) tend to spawn the actual python program in a
subprocess. I believe there are *technically* some differences in the
runtime environment when you use an embedding approach like this, but
I don't know what they are, and they probably won't affect 99.9% of
users. Lack of support for binary extensions is likely to be way more
significant.