[Python-Dev] Single-file Python executables (was: Computed Goto dispatch for Python 2)
v+python at g.nevcal.com
Fri May 29 22:49:43 CEST 2015
On 5/29/2015 3:33 AM, Paul Moore wrote:
> On 28 May 2015 at 22:09, Glenn Linderman <v+python at g.nevcal.com> wrote:
>> This would be something I could use and benefit from immediately upon it
>> being available, so I laud your idea, and hope you have a successful
>> implementation, and look forward to using it. It would largely replace the
>> need for the py.exe launcher for some classes of applications.
> The following proof-of-concept works as is (based on my pretty minimal
> testing), and only uses the limited API, so it should work with any
> version of Python 3 (I've not tested it with Python 2, but I think the
> only "new" API is PySys_SetArgvEx, which could be replaced with
> PySys_SetArgv at a pinch). Excuse the dreadful coding style and lack
> of error handling, I hacked it up in about an hour :-)
I have no particular interest in Python 2, having started with Python 3,
and only used Python 2 for cases where dependent packages required it,
and I've now reached the nirvana of all my dependency packages being
ported to Python 3, although I have yet to port/test one remaining
application to prove that. So I only mentioned Python 2 because it still
could be useful for other people :)
> (Actually, I just tried building on Python 2 - guess what - Unicode
> :-) SetProgramName and SetArgvEx won't take Unicode values. The easy
> fix is just not to use Unicode, the hard one is to do the encoding
> dance, but I'm not going to bother...).
One approach would be to support Unicode arguments only for Python 3,
but that would really be only paying lip service to Python 2 support.
Another approach might be to not #define UNICODE for the Python 2
version, and use the 8-bit Windows APIs, allowing Windows and the C
runtime to do the encoding dance for you? Although I'm not sure what
Python 2 requires in that respect.
> #define UNICODE
> #define _UNICODE
> #include <Python.h>
> #include <windows.h>
> TCHAR program[MAX_PATH];
> LPWSTR *argv;
> int argc;
> PyObject *runpy;
> PyObject *ret;
> argv = CommandLineToArgvW(GetCommandLineW(), &argc);
> GetModuleFileName(NULL, program, MAX_PATH);
> Py_SetProgramName(program); /* optional but recommended */
> PySys_SetArgvEx(argc, argv, 0);
> runpy = PyImport_ImportModule("runpy");
> if (!runpy) PyErr_Print();
> ret = PyObject_CallMethod(runpy, "run_path", "u", program);
> if (!ret) PyErr_Print();
> return 0;
That looks interesting, I wonder what compilation environment it would
need? I don't think I've even installed a C compiler on my last couple
boxes, and the only version of a C compiler I have is, umm... M$VC++6.0,
since I've moved to using Python for anything a 5 line batch file can't
> One mildly annoying thing is that python3.dll is only installed in
> <python install dir>\DLLs, which typically isn't on PATH.
Ah, linking.... so I guess if I figured out how to create this binary,
it would contain a reference to python3.dll that would attempt to be
resolved via the PATH, from what you say, and typically fail, due to
PATH seldom containing python3.dll. The python launcher gets around
that by (1) being installed in %windir%, and going and finding the
appropriate Python (per its own configuration file, and command line
parameters), and setting up the path to that Python, which, when
executed, knows its own directory structure and can thus find its own
The launcher, of course, adds an extra layer of process between the
shell and the program, because it launches the "real" Python executable.
> So actually using the limited API from your own application fails by default.
> Fixing that's mostly a user admin issue, though (and you can just link
> to the full API and avoid the whole problem).
Do I understand correctly that the "user admin issue" means "add the
appropriate <python install dir>\DLLs to the PATH"?
What I don't understand here is how linking to the full API avoids the
problem... it must put more python library code into the stub
executable? Enough to know how to search the registry to find the
<python install dir> for the version of Python from which the full API
was obtained? Or something else?
Are there other alternatives? Assuming that the reference to the
missing DLL is not required until the point at which a symbol from it is
first referenced, so that the stub would have some ability to do
something before that first call, maybe...
1. The stub above could be enhanced to contained a "hard coded"
directory that it adds to the PATH itself?
2. The stub above could be enhanced to define that its first parameter
is the <python install dir>, and tweak its PATH.
3. These days, the Python installer does offer to optionally add itself
to the PATH. Is that sufficient to make the stub work?
4. The launcher could be used, assuming it is installed, but then you
don't need a stub, and you get the extra process layer.
5. stubpy.cmd could be created, a four line batch file below , which
wouldn't require the launcher or its extra process layer, but would
have to be placed on the PATH itself, or in the directory with the
stub Python programs.
Only #3 could be construed as "easy" for the "dumb user"... if the
Python installer offers to add itself to the PATH on "repair" installs,
particularly (I'm not sure if it does). Editing the System PATH through
the control panel is hard for the "dumb user", not made easier by the
squinchy text box M$ provides for the editing. Nor is editing the System
PATH made less error prone by the whole thing being available for
editing, rather than the "GUI promotors" providing an editing editing
interface such as displaying each item separately, with checkboxes to
delete items, or insert items at particular locations, and directory
selection dialogs rather than typing the desired new path as text. Hmm.
Sounds like a good program task for a stub Python program :) Except it
doesn't bootstrap, unless it lives in <python install dir>.
@PATH=<python install dir>;%PATH%
>> Of course, per other disccusions, this doesn't solve the problem for:
>> A) machine without Python installed
>> B) programs that need binary extensions
>> Other discussions have suggested:
>> 3) The stub could offer to download and install Python
>> A corollary:
>> 4) The stub could offer to download and install the needed binary extensions
>> as well as Python. This would require the installation uniformity of
>> something like pip, so perhaps would be restricted to extensions available
>> via pip. And it would be much enhanced by some technique where the zipapp
>> would contain metadata readable by the stub, that would declare the list of
>> binary extensions required. Or, of course, it could even declare non-binary
>> extension that are not packaged with the zipapp, if the process is smooth,
>> the modules available via pip, etc., as a tradeoff.
> I'm pretty strongly against downloading interpreters or extensions.
> Apart from the pretty huge added complexity, as a user I'm not sure
> I'd trust a supposedly simple application I'd received if it started
> asking to download stuff unexpectedly...
Yep. I mostly mentioned them for completeness.
I have no argument with this: installing Python can be a documentation
thing. "To use this program, you need to have Python installed, at
least version N.M." Maybe also : "and it should be installed on the
System PATH, or other methods of setting the PATH before running this
program must be used, or this program should be saved in the <python
If stub+zip programs need other extensions, it can be documented as a
batch file that calls pip a sufficient number of times with appropriate
parameters, or the stub+zip program itself could be written to detect
the needed but missing extensions, and invoke pip to get them before
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev