[Python-Dev] Single-file Python executables (was: Computed Goto dispatch for Python 2)

[Glenn Linderman]

On 5/29/2015 2:45 PM, Paul Moore wrote:
> On 29 May 2015 at 21:49, Glenn Linderman <v+python at g.nevcal.com> wrote:
>> That looks interesting, I wonder what compilation environment it would need?
>> I don't think I've even installed a C compiler on my last couple boxes, and
>> the only version of a C compiler I have is, umm... M$VC++6.0, since I've
>> moved to using Python for anything a 5 line batch file can't do...
>>
>>> One mildly annoying thing is that python3.dll is only installed in
>>> <python install dir>\DLLs, which typically isn't on PATH.
>> Ah, linking.... so I guess if I figured out how to create this binary, it
>> would contain a reference to python3.dll that would attempt to be resolved
>> via the PATH, from what you say, and typically fail, due to PATH seldom
>> containing python3.dll.  The python launcher gets around that by (1) being
>> installed in %windir%, and going and finding the appropriate Python (per its
>> own configuration file, and command line parameters), and setting up the
>> path to that Python, which, when executed, knows its own directory structure
>> and can thus find its own python3.dll.
>>
>> The launcher, of course, adds an extra layer of process between the shell
>> and the program, because it launches the "real" Python executable.
>>
>>> So actually using the limited API from your own application fails by
>>> default.
>>> Fixing that's mostly a user admin issue, though (and you can just link
>>> to the full API and avoid the whole problem).
>>
>> Do I understand correctly that the "user admin issue" means "add the
>> appropriate <python install dir>\DLLs to the PATH"?
>>
>> What I don't understand here is how linking to the full API avoids the
>> problem... it must put more python library code into the stub executable?
>> Enough to know how to search the registry to find the <python install dir>
>> for the version of Python from which the full API was obtained? Or something
>> else?
> Sorry, I assumed more Windows/C knowledge than you have.

It is mostly the C/Python interface knowledge that I lack... although my 
Windows/C knowledge is getting rusty.

> I'll work on this and produce proper binaries in due course, so you
> can always wait for them. But you can build the stub with pretty much
> anything, I suspect - I managed with MSVC 2010 and mingw. I'll add
> some build docs and get it on github.
>
> Using mingw
>
>      gcc -Wall -O2 -o stub.exe stub.c -I <python home>\Include
> C:\Windows\system32\python34.dll
>      strip -s stub.exe
>
> Using MSVC
>
>      cl /Festub.exe /O2 stub.c /I<python home>\Include <python
> home>\libs\python34.lib

Github sounds good.  Binaries sound good.  I would have to download the 
free MSVC10 or Ming and install and learn to use them, etc., to make 
progress... probably doable, but (1) I'm surviving at the moment with 
the launcher + zipapp, but it'd be nice for folks I code for to have 
.exe things, and (2) I'm backlogged in my other projects which don't 
need me to download a C compiler to make progress.

> Regarding the DLLs, yes the "user admin issue" is adding the right
> directory to PATH. I used the phrase "admin issue" as it's the aspect
> that's likely to be far harder than any of the technical issues :-)
> The reason using the full API helps is that the full API references
> python34.dll rather than python3.dll. And the Python installer puts
> python34.dll on PATH automatically, as it's what the "python" command
> uses. (For the people with more Windows knowledge, I know this is a
> simplification, but it's close enough for now).
>
> So there are two  options with the code I posted.
>
> 1. Build an exe that uses a specific version of Python, but which will
> "just work" in basically the same way that the "python" command works.
> 2. Build an exe that works with any version of Python, but requires
> some setup from the user.
>
> Either approach requires that the Python DLL is on PATH, but that's
> far more likely with the version-specific one, just because of how the
> installer does things.

I don't presently see any C:\Python34\DLLs or C:\Python34 on my path, 
but I didn't ask the installer to put it there either. So I'm guessing 
your option 1 assumes asking the Python installer to put it there? Not 
"automatically" but "on request", I think?

In my c:\Python34\DLLs, I don't see a python34.dll, only python3.dll... 
so I'm somewhat unclear on your simplified explanation.

>
> With extra code, the stub could locate an appropriate Python DLL
> dynamically, which would simplify usage at the cost of a bit of fiddly
> code in the stub.
>
> This might be a useful addition to the zipapp module for Python 3.6.

Indeed.  Especially with extra fiddly code if you or someone on Github 
has the time, it could be very useful for the zipapp module.

>
> Paul
>
> PS Current launchers (py.exe, the entry point launchers from
> pip/setuptools, etc) tend to spawn the actual python program in a
> subprocess. I believe there are *technically* some differences in the
> runtime environment when you use an embedding approach like this, but
> I don't know what they are, and they probably won't affect 99.9% of
> users. Lack of support for binary extensions is likely to be way more
> significant.

Lack of support for "zipped / bundled" binary extensions, I assume you 
mean. I have the perception this solution allows use of "normally 
installed" binary extensions.

I don't know the differences in the runtime either, would be good to 
know, if someone knows.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150529/4c6e5c36/attachment.html>