[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[Eric Snow]

Eric Snow <ericsnowcurrently at gmail.com> added the comment:

> IMHO for the long term, the best would be to have a structure for
> pre-Py_Initialize configuration, maybe _PyCoreConfig, and pass it
> to functions that can be called before Py_Initialize().

+1

As an alternative to that, we could also deprecate using any of those functions before initializing the runtime.   Instead of calling them, you would set the relevant info on the runtime "config" struct that you pass to the replacement for PyInitialize().  At that point we would not need some of those functions any longer and we could remove them (eventually, once backward-compatibility is resolved).  Given that the community of CPython embedders is relatively small, we're still in a position to iron this out

Regardless, I see where you're coming from.  I'm okay with reverting the Object/obmalloc.c parts, but, like I said, I'd rather avoid it if possible.

Solution #2 (that I listed above), AKA PR #4481, is focused and effective.  Unfortunately, it feels like a bit like a hack to me, though it is a well-contained hack.  So I'm not convinced it's the best solution.  However, I like it as much as I like reverting the allocators.

Solution #3, AKA PR #4495, is nice and clean, but potentially adds a little overhead to all PyMem_RawMalloc() and PyMem_RawFree() calls.  All the other PyMem_* functions are unaffected, so perhaps the overall impact is not significant enough to worry.


>> 2. statically initialize the "raw" allocator with defaults, enough
>>   to make PyMem_RawMalloc() and PyMem_RawFree() work pre-init (this
>>   is what my PR does)
>
> As I explained, the code to initialize PyMem_Raw allocator is complex
> and I would really prefer to only initialize it "partially" to prevent
> bad surprises.

The surprises would only be pre-initialization, right?  After the runtime is initialized, the allocators are in the proper fully-initialized state.  So it mostly boils down to what parts of the C-API embedders can use before initialization and how those functions interact with the raw memory allocator.  Those constraints narrow down the scope of potential problems to a manageable size (I won't say small, but it feels that way).

Ultimately, I favor solution #3 if we can see that it does not impact performance.  If we can't come to an agreement in a timely fashion then I'll go along with #1 (revert), so that we don't leave the embedding story broken.  If we go that route, do you think we could resolve the initialization issues within the 3.7 timeframe?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32096>
_______________________________________