Hi Mark, Le 05/03/2021 à 18:06, Mark Shannon a écrit :
Hi Antoine,
On 05/03/2021 4:07 pm, Antoine Pitrou wrote:
On Fri, 5 Mar 2021 15:03:59 +0000 Mark Shannon <mark@hotpy.org> wrote:
There are two issues here. Portability and changes to behaviour.
Regarding portability, I have to admit that PEP is rather vague. That's my fault; I should have done more implementation first :( FWIW, I have an implementation that should be portable. https://github.com/python/cpython/compare/master...markshannon:pep-overflow-...
I looked through this diff and I'm not sure how this works robustly.
This seems to assume: - each thread's stack size is known and is a compile-time constant We just need to know that the stack size is at least the compile-time constant, we don't need to know exactly what it is.
If the stack size is less than the compile-time constant, then a crash is a possibility. However, since PEP 651 would reduce stack consumption, a crash would have been less likely than it currently is.
For all supported platforms, there is a default stack size and it is considerably larger than the chosen value of 512k.
Right, but Python more or less works on a large range of non-formally supported platforms. With your proposal and a hardcoded 512 kB minimum stack size, Python may crash almost immediately on such platforms. I don't know if we care about that (we could tell third-party maintainers to change the #define to an appropriate value), but it's still something to will make porting a bit more cumbersome.
- Python owns the thread and can compute the stack limit reliably from the current stack frame There are two points of entry into Python code. From the high level API and thread_run().
Hmm, actually, there are many entry points into Python code. C code can create its own thread and call arbitrary CPython APIs from it (after having initialized the tstate using the PyGILState API). That call inside a C-created thread can happen after an arbitrary amount of stack was already consumed by the calling C code.
But, this is no worse than the status quo, and probably better due to reduced C stack use.
I'm not sure what you mean with "no worse than the statu quo". Supporting a smaller recursion limit than advertised is one thing, crashing almost immediately when executing Python code (because of trying to access memory outside beyond the reserved stack area) is another. Currently, you would only crash when recursing a lot.
- it's ok to write memory beyond the current stack top (I can imagine verification tools such as Valgrind complaining about that, though this can be worked around using suppressions) It's not much of a stack if you can't grow it :)
The stack size is typically limited (see `ulimit -s`). Regards Antoine.