C-API functions for reading/writing tstate->exc_* ?
Hi, the Cython and PyPy projects are currently working on getting Cython implemented extensions to build and run in PyPy, interfacing at the C-API level for now. One problem we encountered was that there is currently no "abstract" way to access tstate->exc_type and friends, i.e. the last exception that was caught, aka. sys.exc_info(). Apparently, PyPy stores them at a frame level, whereas CPython makes them available in thread local storage as bare struct fields. Even if PyPy put them there on request, any changes to these fields would pass unnoticed. And Cython needs to set them in order to properly implement the semantics of a try-except statement (and in some other places where exception state is scoped). When compiling for PyPy, Cython therefore needs a way to tell PyPy about any changes. For the tstate->curexc_* fields, there are the two functions PyErr_Fetch() and PyErr_Restore(). Could we have two similar "official" functions for the exc_* fields? Maybe PyErr_FetchLast() and PyErr_RestoreLast()? Note that Cython would not have a reason to actually use them in CPython, and it should be uncommon for non-Cython extension modules to care about the exc_* fields at all. So these functions won't be of much use if actually implemented in CPython (although I wouldn't mind doing that). The question is just if we could have two officially named functions that PyPy (and maybe other Pythons) could implement in order to access the last raised exception in a way that does not depend on implementation details. Stefan
On 19 February 2012 13:04, Stefan Behnel <stefan_ml@behnel.de> wrote:
When compiling for PyPy, Cython therefore needs a way to tell PyPy about any changes. For the tstate->curexc_* fields, there are the two functions PyErr_Fetch() and PyErr_Restore(). Could we have two similar "official" functions for the exc_* fields? Maybe PyErr_FetchLast() and PyErr_RestoreLast()?
It sounds reasonable to simply write a patch implementing and documenting these functions, put it in the tracker, and ask for it to be applied - I can't see an obvious reason not to do so. (There may be reasons not to put them in the "stable API", but that's not so relevant here).
Note that Cython would not have a reason to actually use them in CPython, and it should be uncommon for non-Cython extension modules to care about the exc_* fields at all. So these functions won't be of much use if actually implemented in CPython (although I wouldn't mind doing that). The question is just if we could have two officially named functions that PyPy (and maybe other Pythons) could implement in order to access the last raised exception in a way that does not depend on implementation details.
You're probably worrying too much here. Get them added to Python 3.3, and then you're fine (if PyPy need to implement them for earlier versions, that's no problem for CPython, presumably PyPy don't have quite so stringent backward compatibility requirements yet, and the fact that they exist in 3.3 gives you the standardisation you need). Certainly "to have a standard API for getting at this information, even though it's probably not necessary for CPython extensions" isn't the best justification, but it's not the worst I've seen either :-) Of course, you could always go through the Python API, getting the sys module, extracting the relevant functions and calling them using the abstract API. That's what I'd recommend if this were purely a CPython question. But I assume that (for some reason) that's not appropriate for PyPy. Of course, my opinion doesn't carry a lot of weight here, so don't read too much into this :-) Paul.
On Mon, Feb 20, 2012 at 12:18 AM, Paul Moore <p.f.moore@gmail.com> wrote:
Of course, you could always go through the Python API, getting the sys module, extracting the relevant functions and calling them using the abstract API. That's what I'd recommend if this were purely a CPython question. But I assume that (for some reason) that's not appropriate for PyPy.
I had the same thought, but it actually only works cleanly for the "fetch" half of the equation. You'd have to muck around with actually raising an exception to handle *setting* those fields, which is *really* relying on implementation details). That said, it may be worth further exploring the idea of invoking appropriate snippets of Python code to get the desired effect. My other question would be whether there's an existing *private* C API with the desired behaviour that could be made public, or if this would be a genuinely new addition to the API. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Nick Coghlan, 19.02.2012 15:31:
On Mon, Feb 20, 2012 at 12:18 AM, Paul Moore wrote:
Of course, you could always go through the Python API, getting the sys module, extracting the relevant functions and calling them using the abstract API. That's what I'd recommend if this were purely a CPython question. But I assume that (for some reason) that's not appropriate for PyPy.
I had the same thought, but it actually only works cleanly for the "fetch" half of the equation. You'd have to muck around with actually raising an exception to handle *setting* those fields, which is *really* relying on implementation details). That said, it may be worth further exploring the idea of invoking appropriate snippets of Python code to get the desired effect.
Actually, we currently inline the straight C code for this in CPython for performance reasons, so, no, going through Python code isn't going to be a good idea.
My other question would be whether there's an existing *private* C API with the desired behaviour that could be made public, or if this would be a genuinely new addition to the API.
I'm not aware of any. The code in CPython (especially in ceval.c) always uses direct field access. Stefan
When compiling for PyPy, Cython therefore needs a way to tell PyPy about any changes. For the tstate->curexc_* fields, there are the two functions PyErr_Fetch() and PyErr_Restore(). Could we have two similar "official" functions for the exc_* fields? Maybe PyErr_FetchLast() and PyErr_RestoreLast()?
I wouldn't call the functions *Last, as this may cause confusion with sys.last_*. I'm also unsure why the current API uses this Fetch/Restore pair of functions where Fetch clears the variables. A Get/Set pair of functions would be more natural, IMO (where Get returns "new" references). This would give PyErr_GetExcInfo/PyErr_SetExcInfo. Regards, Martin
"Martin v. Löwis", 19.02.2012 23:24:
When compiling for PyPy, Cython therefore needs a way to tell PyPy about any changes. For the tstate->curexc_* fields, there are the two functions PyErr_Fetch() and PyErr_Restore(). Could we have two similar "official" functions for the exc_* fields? Maybe PyErr_FetchLast() and PyErr_RestoreLast()?
I wouldn't call the functions *Last, as this may cause confusion with sys.last_*. I'm also unsure why the current API uses this Fetch/Restore pair of functions where Fetch clears the variables. A Get/Set pair of functions would be more natural, IMO (where Get returns "new" references). This would give PyErr_GetExcInfo/PyErr_SetExcInfo.
Ok, I added a tracker ticket and I'm working on a patch. http://bugs.python.org/issue14098 Stefan
Stefan Behnel, 23.02.2012 09:01:
"Martin v. Löwis", 19.02.2012 23:24:
When compiling for PyPy, Cython therefore needs a way to tell PyPy about any changes. For the tstate->curexc_* fields, there are the two functions PyErr_Fetch() and PyErr_Restore(). Could we have two similar "official" functions for the exc_* fields? Maybe PyErr_FetchLast() and PyErr_RestoreLast()?
I wouldn't call the functions *Last, as this may cause confusion with sys.last_*. I'm also unsure why the current API uses this Fetch/Restore pair of functions where Fetch clears the variables. A Get/Set pair of functions would be more natural, IMO (where Get returns "new" references). This would give PyErr_GetExcInfo/PyErr_SetExcInfo.
Ok, I added a tracker ticket and I'm working on a patch.
The patch is attached to the ticket, including documentation and test. I'd be happy if someone could review it and apply it. Thanks! Stefan
participants (4)
-
"Martin v. Löwis" -
Nick Coghlan -
Paul Moore -
Stefan Behnel