[Python-ideas] Calling python from C completely statically

[Paul Moore]

On 8 July 2018 at 09:01, Alberto Garcia <agarciaillera at gmail.com> wrote:
> Hi,
>
> I've been working for a while on having the entire python interpreter with
> all his modules statically linked in a binary that can be called with
> arbitrary argument which will be passed to the python interpreter.
>
> I've been able to statically compile Python in a single binary with no
> dependencies for Python2.7 and using using this code to call the interpreter
> with arbitrary code:
>
> #define Py_NO_ENABLE_SHARED
> #include <Python.h>
>
> int main(int argc, char** argv)
> {
> Py_NoSiteFlag = 1;
> Py_InitializeEx(0);
> PyRun_SimpleString(argv[1]);
> Py_Finalize();
> }
>
> This program compiles and does not rely in any dependency but the "non
> frozen" (.py) modules are not loaded so when I do a simple `import random` I
> get:
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
> ImportError: No module named random
>
> I've created a bug issue (https://bugs.python.org/issue34057) speaking about
> it and Nick Colghlan pointed:
>
> " cx_freeze is an illustrative example to look at in that regard, as it
> preconfigures the interpreter to be able to find the cx_freeze generated zip
> archive that has the program's Python modules in it:
> https://github.com/anthony-tuininga/cx_Freeze/blob/master/source/bases/Common.c
>
> The technique that cx_freeze doesn't use yet is to combine the statically
> linked Python binary and the generated zip archive into a single file
> (similar to what zipapp does), and adjust the sys.path definition inside the
> binary to refer back to the executable itself (since executable files can
> have arbitrary content appended, while zip files can have arbitrary content
> *pre*pended). "
>
> As I understand, cx_freeze creates a zip file with the dependencies for a
> specific python code and creates a zip with them. My question is, could I
> create a zip file with every standard module (plus some extra that I may
> install with pip) and using that zip file from C? If so, how can I do that?
>
> I'm happy to work on this if I get indications.
>
> BTW: I do not want to convert a python code snippet to exe or anything like
> that but calling python from C in an absolutely standalone version.

This question is probably more appropriate for python-list, but yes,
you certainly can do this. The "Embedded" distributions of Python for
Windows essentially do this already. IIRC, they are only available for
Python 3.x, so you may find you have some hurdles to overcome if you
have to remain on Python 2.7, but in principle it's possible.

One further point you may need to consider - a proportion of the
standard library is in the form of shared C extensions (PYDs in
Windows - I don't know if you're using Windows or Unix from what you
say above). You can't (without significant extra work) load a binary
extension direct from a zip file, so you'll need to either do that
extra work (which is platform specific and fragile, I believe) or be
prepared to ship supporting DLLs alongside your application (this is
the approach the embedded distribution takes).

Hope this helps,
Paul