Hi there! here has been this patch to the threadstate, which allows to override the tstate's frame access. I just saw the part of the patch that modifies pyexpat: f = PyFrame_New( tstate, /*back*/ c, /*code*/ ! PyEval_GetGlobals(), /*globals*/ NULL /*locals*/ where the PyEval_GetGLobals is used instead of tstate->frame->f_globals Well, this unfortunately is not sufficient for this module, since pyexpat still *has* direct access to tstate->frame, in a much worse way: pyexpat does read and write the frame variable! In line 326, function call_with_frame, pyexpat creates a new frame, assigns it to tstate->frame and later on assigns f_f_back to it. Reason why I'm thinking about this: In order to simplify Stackless, I thought to remove the frame variable, and let it be accessed always via my current tasklet, which holds the frame. Looking for the number of necessary patches, I stumbled over PyExpat, and thought I should better keep my hands off. Too bad. Does it make sense to think about an API for modifying the frame? Or are we at a dead end here? cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Christian Tismer wrote:
Hi there!
here has been this patch to the threadstate, which allows to override the tstate's frame access.
I just saw the part of the patch that modifies pyexpat:
f = PyFrame_New( tstate, /*back*/ c, /*code*/ ! PyEval_GetGlobals(), /*globals*/ NULL /*locals*/
where the PyEval_GetGLobals is used instead of tstate->frame->f_globals
Well, this unfortunately is not sufficient for this module, since pyexpat still *has* direct access to tstate->frame, in a much worse way: pyexpat does read and write the frame variable!
Ah!! Can it be that PyEval_GetFrame() is just indended to signal to an extension like Psyco that it needs to quickly invent a frame now? So it is *not* thought of to be a complete interface for accessing tstate->frame no longer explicitly, is is only meant for read access? So I can't move it elsewhere and probably need to work around that forever, unless we also write PyEval_SetFrame() sigh - ciao - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
Hello Christian, On Fri, Mar 14, 2003 at 07:05:05PM +0100, Christian Tismer wrote:
where the PyEval_GetGLobals is used instead of tstate->frame->f_globals Ah!! Can it be that PyEval_GetFrame() is just indended to signal to an extension like Psyco that it needs to quickly invent a frame now?
Yes, indeed. This was a very limited hack so that the frame would get the correct locals even in the presence of Psyco. Now I realize that it may have been pointless anyway, if this dummy frame is never really used but for tracebacks. Maybe an API to manipulate tstate->frame could be useful and really lightweight. Alternatively, we could consider what pyexpat does as a general pattern and have an API for it, e.g.: PyFrame_Push(PyFrameObject* f) -> pushes 'f' on the frame stack, assert()ing that f->f_back is tstate->frame or pushes a new placeholder frame if 'f' is NULL. This also calls the profile and trace hooks. PyFrame_Pop() -> pops the frame, calling profile and trace hooks, and recording a traceback if PyErr_Occurred(). and maybe a PyFrame_FromC() function that creates a placeholder with controllable parameters as in pyexpat.c:getcode(). A bientôt, Armin.
Armin Rigo <arigo@tunes.org> writes:
Maybe an API to manipulate tstate->frame could be useful and really lightweight. Alternatively, we could consider what pyexpat does as a general pattern and have an API for it, e.g.:
Yes please! Cheers, M. -- Our lecture theatre has just crashed. It will currently only silently display an unexplained line-drawing of a large dog accompanied by spookily flickering lights. -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year)
Christian Tismer writes:
Hi there!
Good afternoon!
here has been this patch to the threadstate, which allows to override the tstate's frame access. ... Reason why I'm thinking about this: In order to simplify Stackless, I thought to remove the frame variable, and let it be accessed always via my current tasklet, which holds the frame.
Looking for the number of necessary patches, I stumbled over PyExpat, and thought I should better keep my hands off. Too bad.
Does it make sense to think about an API for modifying the frame? Or are we at a dead end here?
What's being modified isn't the frame but the tstate, but it may be reasonable to provide some API to manipulate the "current" frame. I think pyexpat is unique in doing this, but it actually makes a lot of sense; there are other modules for which a similar behavior is likely to be appropriate (one example I can think of is Fredrik's sgmlop module). What pyexpat is trying to achieve is fairly simple, and I don't think there's a better way currently. When Python code calls the Parse() or ParseFile() method of a parser object (returned from pyexpat.ParserCreate()), the parser can generate many different callbacks into Python code. pyexpat generates an artificial code object and frame that can be used to generate more useful tracebacks when exceptions are raise within callbacks; the code object indicates which callback Expat triggered, separately from the function assigned to handle that callback. This makes it much easier to debug handler functions. If there were API functions to get/set the frame, pyexpat wouldn't need to poke into the tstate at all. Would that alleviate the difficulties this creates for Stackless / Psycho? -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation
Fred L. Drake, Jr. wrote:
Christian Tismer writes: ...
Does it make sense to think about an API for modifying the frame? Or are we at a dead end here?
What's being modified isn't the frame but the tstate, but it may be reasonable to provide some API to manipulate the "current" frame.
That was what I intended to say.
I think pyexpat is unique in doing this, but it actually makes a lot of sense; there are other modules for which a similar behavior is likely to be appropriate (one example I can think of is Fredrik's sgmlop module).
I just looked into MHammond's files and found AXDebug.cpp reading tstate's frame, too, line 192: PyFrameObject *frame = state ? state->frame : NULL; WOuld be another candidate to use PyEval_GetFrame()
What pyexpat is trying to achieve is fairly simple, and I don't think there's a better way currently. When Python code calls the Parse() or ParseFile() method of a parser object (returned from pyexpat.ParserCreate()), the parser can generate many different callbacks into Python code. pyexpat generates an artificial code object and frame that can be used to generate more useful tracebacks when exceptions are raise within callbacks; the code object indicates which callback Expat triggered, separately from the function assigned to handle that callback. This makes it much easier to debug handler functions.
Yes, this makes very much sense, to just wrap a frame around something to get useful tracebacks.
If there were API functions to get/set the frame, pyexpat wouldn't need to poke into the tstate at all. Would that alleviate the difficulties this creates for Stackless / Psycho?
I think so. For internal functions, inside the main Python implementation, it is no problem to maintain a number of patches, or to even use interface functions whenre they make sense like enabling Psyco. For external modules, it would be nicer if certain implementation details could be hidden, to give more freedom to implement somethign differently, without breaking unknown modules. PyExpat is in a grey zone, since it is already in the Python distribution, and I had no problem to patch it. The reason why I'm asking is that in Stackless 2.0 and above, tstate is always carrying a so-called tasklet object which holds the current frame. There can be many of these, while there is only one current one per tstate, and they can be switched at certain times. At the moment, I always have to take care to preserve a valid tstate->frame variable, whenever I'm switching tasklets. It would make the code much smaller and cleaner, if I could simply redefine the current frame to be just the frame held in the current tasklet. Something like PyEval_SetFrame() would make sense, but there is a problem with the protocol: After changing the frame, the currently running interpreter would not know to execute it, but continue to run the running one. For normal Python, PyEval_SetFrame() would only make sense, if the calling code would also make sure that the new frame is run, or popped off after a special action was done. PyExpat, as I understand, uses this frame just as a wrapper for the case of an exception, when the frame would be used only when a failing function returns NULL. Stackless extends this and also knows to return a special value instead of NULL, in order to tell the calling interpreter to stop its action and let the new frame run. Even in PyExpat's use, it would hard to explain the use of such a function, I fear. But I'd really like to have it. cheers - chris -- Christian Tismer :^) <mailto:tismer@tismer.com> Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/
participants (4)
-
Armin Rigo -
Christian Tismer -
Fred L. Drake, Jr. -
Michael Hudson