[Python-Dev] Python initialization and embedded Python

[Nick Coghlan]

On 18 November 2017 at 10:01, Victor Stinner <victor.stinner at gmail.com> wrote:
> I'm writing this email to ask if this change is an issue or not to
> embedded Python and the Python C API. Is it still possible to call
> "all" functions of the C API before calling Py_Initialize()?

It isn't technically permitted to call any of them, unless their
documentation specifically says that calling them before
`Py_Initialize` is permitted (and that permission is only given for a
select few configuration APIs in
https://docs.python.org/3/c-api/init.html).

While it's still PEP 432's intention to eventually expose a public
multi-phase start-up API, it's *also* the case that we're not actually
ready to do that yet - we're not sure we have the data model right,
and we don't want to commit to a supported API until that's resolved.

So for Python 3.7, I'd suggest pursuing one of the following options:

1. Add a variant of Py_DecodeLocale that accepts a memory allocation
function directly and reports back both the allocated pointer and its
size (allowing the calling program to manage that memory); or
2. Offer a new `Py_SetProgramNameFromString` API that accepts a `char
*` directly. That way, CPython can take care of lazily decoding it
after the decoding machinery has been fully set up, rather than
expecting the embedding application to always do it;

(While we could also make the promise that PyMem_RawMalloc and
Py_DecodeLocale will be callable before Py_Initialize, I don't think
we're far enough into the startup refactoring process to be making
those kinds of promises).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia