I see I haven't sent the response here; apologies for that!
On Fri, Aug 28, 2020 at 12:53 AM Nick Coghlan ncoghlan@gmail.com wrote:
On Wed., 26 Aug. 2020, 12:07 am Petr Viktorin, encukou@gmail.com wrote:
On 2020-08-05 14:09, Nick Coghlan wrote:
[note] Other functions, like
PyType_FromSpec(), can also create heap types, butPyType_FromModuleAndSpec()associates the module with the class, granting access to the module state to methods.The class should generally be stored in *both* the module state (for safe access from C) and the module's
__dict__(for access from Python code).In these cases, it may make sense to offer an accessible-from-Python function that checks the module's internal state for consistency with the Python level state. Otherwise projects that try to inject transparent proxies around classes (e.g. some application instrumentation monitoring toolkits) will get weird behaviour, as C code will still see the original class, while Python code will see the wrapped one.
I'm not familiar with these proxies. How do they work? Can we test CPython somehow to make sure they don't break?
wrapt on PyPI is the main example I'm aware (Graham Dumpleton wrote it to enable runtime injection of performance tracing hooks into Python web apps when he was at New Relic)
I welcome people working on tooling and/or documentation on this, but it's not a priority for me. As I see it: if it's safe to arbitrarily modfy/delete the object, use __dict__ and don't bother with C-level state If it's not safe and monkeypatching is allowed, then monkeypatching is likely to break assumptions (i.e. give unexpected results at the Python level; it'll be safe at the C level). The edge cases around this seem, to me, quite hard to define and test. I don't really see how a consistency-checking function would help; perhaps write this up as a separate proposal?
[snip]
### Per-Class scope
It is also not possible to attach state to *types*. While
PyHeapTypeObjectis a variable-size object (PyVarObject), but its variable-size storage is currently consumed by slots. There will also be issues if several classes in an inheritance hierarchy need state.Perhaps mention that the simplest currently available option here is to do the same thing Python code does, and define a private attribute on the defining class, potentially prepending the class name to reduce the risk of name collisions?
I don't want to mention that because it'f unsafe: it'll break when that attribute is modified, which you can do trivially from Python.
Right, but there are different degrees of safety needed in app specific libraries where you also control all the consuming code vs published libraries where you need to be more resilient against potential misuse.
In the former case "don't do that, you'll crash the app" can be a reasonable API usage restriction (especially on smaller projects); it's only in the published library case, or on projects that are so big or long-lived that they may as well be public, that the approach is likely to be fragile.
If there isn't a safe way to do it, I consider this out of scope for this PEP.