[Python-Dev] USE_STACKCHECK and running out of stack

[Gregory P. Smith]

On Sat, Jul 28, 2018 at 7:51 AM Ronald Oussoren via Python-Dev <
python-dev at python.org> wrote:

> Hi,
>
> I’m looking at PyOS_CheckStack because this feature might be useful on
> macOS (and when I created bpo-33955 for this someone ran with it and
> created a patch).
>
> Does anyone remember why the interpreter raises MemoryError and not
> RecursionError when PyOS_CheckStack detects that we’re about to run out of
> stack space?
>

Running out of C stack makes more sense to me as a MemoryError, you don't
know that recursion was the primary cause and the fundamental problem is
that you are (about to) run out of a finite memory resource.

All you know is that at least one C frame on the C call stack may have
required a lot of local stack space.  Just as likely as there being tons of
regular sized frames.

RecursionError is only raised when CPython's arbitrary Python recursion
depth limit is reached.  It is not raised for memory reasons.

In small thread stack space environments (64k, etc) I tended to find
problematic code to be C code with local arrays that consumed several
kilobytes each rather than actual recursion.  Python 2's own import system
was notorious for that so if your server is calling service handlers in
threads with a small stack and some of those are doing new imports at
handler time rather than having done all imports before serving began:
boom. stack overflow.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180730/210811c8/attachment-0001.html>