On 5/29/2015 3:33 AM, Paul Moore wrote:
On 28 May 2015 at 22:09, Glenn Linderman <v+python@g.nevcal.com> wrote:

This would be something I could use and benefit from immediately upon it
being available, so I laud your idea, and hope you have a successful
implementation, and look forward to using it.  It would largely replace the
need for the py.exe launcher for some classes of applications.

The following proof-of-concept works as is (based on my pretty minimal
testing), and only uses the limited API, so it should work with any
version of Python 3 (I've not tested it with Python 2, but I think the
only "new" API is PySys_SetArgvEx, which could be replaced with
PySys_SetArgv at a pinch). Excuse the dreadful coding style and lack
of error handling, I hacked it up in about an hour :-)

I have no particular interest in Python 2, having started with Python 3, and only used Python 2 for cases where dependent packages required it, and I've now reached the nirvana of all my dependency packages being ported to Python 3, although I have yet to port/test one remaining application to prove that. So I only mentioned Python 2 because it still could be useful for other people :)

(Actually, I just tried building on Python 2 - guess what - Unicode
:-) SetProgramName and SetArgvEx won't take Unicode values. The easy
fix is just not to use Unicode, the hard one is to do the encoding
dance, but I'm not going to bother...).

One approach would be to support Unicode arguments only for Python 3, but that would really be only paying lip service to Python 2 support.  Another approach might be to not #define UNICODE for the  Python 2 version, and use the 8-bit Windows APIs, allowing Windows and the C runtime to do the encoding dance for you? Although I'm not sure what Python 2 requires in that respect.

#define UNICODE
#define _UNICODE
#include <Python.h>
#include <windows.h>

int
main()
{
    TCHAR program[MAX_PATH];
    LPWSTR *argv;
    int argc;
    PyObject *runpy;
    PyObject *ret;

    argv = CommandLineToArgvW(GetCommandLineW(), &argc);
    GetModuleFileName(NULL, program, MAX_PATH);
    Py_SetProgramName(program);  /* optional but recommended */
    Py_Initialize();
    PySys_SetArgvEx(argc, argv, 0);
    runpy = PyImport_ImportModule("runpy");
    if (!runpy) PyErr_Print();
    ret = PyObject_CallMethod(runpy, "run_path", "u", program);
    if (!ret) PyErr_Print();
    Py_Finalize();
    return 0;
}

That looks interesting, I wonder what compilation environment it would need?  I don't think I've even installed a C compiler on my last couple boxes, and the only version of a C compiler I have is, umm... M$VC++6.0, since I've moved to using Python for anything a 5 line batch file can't do...

One mildly annoying thing is that python3.dll is only installed in
<python install dir>\DLLs, which typically isn't on PATH.
Ah, linking.... so I guess if I figured out how to create this binary, it would contain a reference to python3.dll that would attempt to be resolved via the PATH, from what you say, and typically fail, due to PATH seldom containing python3.dll.  The python launcher gets around that by (1) being installed in %windir%, and going and finding the appropriate Python (per its own configuration file, and command line parameters), and setting up the path to that Python, which, when executed, knows its own directory structure and can thus find its own python3.dll.

The launcher, of course, adds an extra layer of process between the shell and the program, because it launches the "real" Python executable.

So actually using the limited API from your own application fails by default.
Fixing that's mostly a user admin issue, though (and you can just link
to the full API and avoid the whole problem).

Do I understand correctly that the "user admin issue" means "add the appropriate <python install dir>\DLLs to the PATH"?

What I don't understand here is how linking to the full API avoids the problem... it must put more python library code into the stub executable? Enough to know how to search the registry to find the <python install dir> for the version of Python from which the full API was obtained? Or something else?

Are there other alternatives?  Assuming that the reference to the missing DLL is not required until the point at which a symbol from it is first referenced, so that the stub would have some ability to do something before that first call, maybe...
  1. The stub above could be enhanced to contained a "hard coded" directory that it adds to the PATH itself?
  2. The stub above could be enhanced to define that its first parameter is the <python install dir>, and tweak its PATH.
  3. These days, the Python installer does offer to optionally add itself to the PATH. Is that sufficient to make the stub work?
  4. The launcher could be used, assuming it is installed, but then you don't need a stub, and you get the extra process layer.
  5. stubpy.cmd could be created, a four line batch file below [1], which wouldn't require the launcher or its extra process layer, but would have to be placed on the PATH itself, or in the directory with the stub Python programs.

Only #3 could be construed as "easy" for the "dumb user"... if the Python installer offers to add itself to the PATH on "repair" installs, particularly (I'm not sure if it does).  Editing the System PATH through the  control panel is hard for the "dumb user", not made easier by the squinchy text box M$ provides for the editing. Nor is editing the System PATH made less error prone by the whole thing being available for editing, rather than the "GUI promotors" providing an editing editing interface such as displaying each item separately, with checkboxes to delete items, or insert items at particular locations, and directory selection dialogs rather than typing the desired new path as text.  Hmm. Sounds like a good program task for a stub Python program :) Except it doesn't bootstrap, unless it lives in <python install dir>.

[1] stubpy.cmd:
@setlocal
@PATH=<python install dir>;%PATH%
@shift
@%*



Of course, per other disccusions, this doesn't solve the problem for:

A) machine without Python installed
B) programs that need binary extensions

Other discussions have suggested:

3) The stub could offer to download and install Python

A corollary:

4) The stub could offer to download and install the needed binary extensions
as well as Python. This would require the installation uniformity of
something like pip, so perhaps would be restricted to extensions available
via pip.  And it would be much enhanced by some technique where the zipapp
would contain metadata readable by the stub, that would declare the list of
binary extensions required.  Or, of course, it could even declare non-binary
extension that are not packaged with the zipapp, if the process is smooth,
the modules available via pip, etc., as a tradeoff.

I'm pretty strongly against downloading interpreters or extensions.
Apart from the pretty huge added complexity, as a user I'm not sure
I'd trust a supposedly simple application I'd received if it started
asking to download stuff unexpectedly...

Paul

Yep. I mostly mentioned them for completeness.

I have no argument with this: installing Python can be a documentation thing.  "To use this program, you need to have Python installed, at least version N.M."  Maybe also : "and it should be installed on the System PATH, or other methods of setting the PATH before running this program must be used, or this program should be saved in the <python install dir>."

If stub+zip programs need other extensions, it can be documented as a batch file that calls pip a sufficient number of times with appropriate parameters, or the stub+zip program itself could be written to detect the needed but missing extensions, and invoke pip to get them before using them.