
On 13 September 2017 at 00:35, Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Tue, Sep 12, 2017 at 1:40 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 11 September 2017 at 18:02, Koos Zevenhoven <k7hoven@gmail.com> wrote:
On Mon, Sep 11, 2017 at 8:32 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
The line between it and the "CPython Runtime" is fuzzy for both practical and historical reasons, but the regular Python CLI will always have a "first created, last destroyed" main interpreter, simply because we don't really gain anything significant from eliminating it as a concept.
I fear that emphasizing the main interpreter will lead to all kinds of libraries/programs that somehow unnecessarily rely on some or all tasks being performed in the main interpreter. Then you'll have a hard time running two of them in parallel in the same process, because you don't have two main interpreters.
You don't need to fear this scenario, since it's a description of the status quo (and it's the primary source of overstated claims about subinterpreters being "fundamentally broken").
Well, if that's true, it's hardly a counter-argument to what I said. Anyway, there is no status quo about what is proposed in the PEP.
Yes, there is, since subinterpreters are an existing feature of the CPython implementation. What's new in the PEP is the idea of giving that feature a Python level API so that it's available to regular Python programs, rather than only being available to embedding applications that choose to use it (e.g. mod_wsgi).
And as long as the existing APIs are preserved, why not make the new one less susceptible to overstated fundamental brokenness?
Having a privileged main interpreter isn't fundamentally broken, since you aren't going to run __main__ in more than one interpreter, just as you don't run __main__ in more than one thread (and multiprocessing deliberately avoids running the "if __name__ == '__main__'" sections of it in more than one process).
So no, not everything will be subinterpreter-friendly, just as not everything in Python is thread-safe, and not everything is portable across platforms.
I don't see how the situation benefits from calling something the "main interpreter". Subinterpreters can be a way to take something non-thread-safe and make it thread-safe, because in an interpreter-per-thread scheme, most of the state, like module globals, are thread-local. (Well, this doesn't help for async concurrency, but anyway.)
"The interpreter that runs __main__" is never going to go away as a concept for the regular CPython CLI. Right now, its also a restriction even for applications like mod_wsgi, since the GIL state APIs always register C created threads with the main interpreter.
That's OK - it just means we'll aim to make as many things as possible implicitly subinterpreter-friendly, and for everything else, we'll aim to minimise the adjustments needed to *make* things subinterpreter friendly.
And that's exactly what I'm after here!
No, you're after deliberately making the proposed API non-representative of how the reference implementation actually works because of a personal aesthetic preference rather than asking yourself what the practical benefit of hiding the existence of the main interpreter would be. The fact is that the main interpreter *is* special (just as the main thread is special), and your wishing that things were otherwise won't magically make it so.
I'm mostly just worried about the `get_main()` function. Maybe it should be called `asdfjaosjnoijb()`, so people wouldn't use it. Can't the first running interpreter just introduce itself to its children? And if that's too much to ask, maybe there could be a `get_parent()` function, which would give you the interpreter that spawned the current subinterpreter.
If the embedding application never calls "_Py_ConfigureMainInterpreter", then get_main() could conceivably return None. However, we don't expose that as a public API yet, so for the time being, Py_Initialize() will always call it, and hence there will always be a main interpreter (even in things like mod_wsgi). Whether we invest significant effort in making configuring the main interpreter genuinely optional is still an open question - since most applications are free to just not use the main interpreter for code execution if they don't want to, we haven't found a real world use case that would benefit meaningfully from its non-existence (just as the vast majority of applications don't care about the various ways in which the main thread that runs Py_Initialize() and Py_Finalize() is given special treatment, and for those that do, they're free to avoid using it). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia