[Python-Dev] Re: Another test_compiler mystery

[Armin Rigo]

Hello,

On Thu, Aug 12, 2004 at 03:41:37PM -0400, Tim Peters wrote:
> Sounds like a good idea not to run out to stack <wink>.

I don't remember if it was mentioned here, but maybe we'd better check
directly whether the C stack is too large rather than (or possibly in addition
to) using a limit on the number of Python iterations.

This is not really ANSI C, but I can't think of a setting in which the
following trick would fail to be a good indication of the amount of stack
space consumed: save a pointer to a local variable "early", e.g. in
Py_Initialize(); then in any other function call, the distance between this
pointer and a pointer to some local equals the amount of stack space consumed
by Python minus a few bytes.

If this sounds too much of a hack, the (usual) recursion limit could be kept
to limit nested Python frames; but all C-level recursions (including
eval_frame) could additionally use the above trick.  Its advantage is that it
is an extremely fast check.

If the total stack size is difficult to know in advance, we can still use
PyOS_CheckStack(), but more efficiently and less randomly than now, by
maintaining a "high tide" pointer that records how much stack we are sure we
have, and calling PyOS_CheckStack() only to attempt to push the "high tide"
mark further.

While I'm in a hacking mood, there might be a way to prevent PyErr_Clear() to
clear away "asynchronuous" exceptions like RuntimeError, MemoryError, and
KeyboardInterrupt: for these exceptions, let's just store them away instead of
clearing them, and re-raise them at the next occasion.  The "next occasion"
would have to be defined more precisely, but there is probably a way to ensure
that it will at least be before the next non-trivial opcode operation starts.  
It would overwrite an exception set later, like those spurious KeyError we get
for dict lookups.  It might be a better-than-nothing quick fix to all these
PyErr_Clear() all around the code base.


A bientôt,

Armin.