Re: [Python-Dev] 2.4.4: backport classobject.c HAVE_WEAKREFS?
The change was for clarity -- most things that have the weakref slots filled-in will also make the flag explicit -- that makes it easier on the brain when verifying code that checks the weakref flag.
I don't understand why you added this flag here;
Perhaps my other post wasn't clear. The change wasn't necessary, so if it bugs you, feel free to take it out. Essentially, it was a "note to self" so that I didn't have to keep looking up what was implied by Py_TPFLAGS_DEFAULT.
the slightly obscure bit that requires some getting used to is that all these flags don't really mean "I have such and such feature" but just "I could have such and such feature, if the corresponding tp_xxx field were set".
I would like to see that cleaned-up for Py3k. Ideally, the NULL or non_NULL status of a slot should serve as its flag. Raymond
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Oct 10, 2006, at 5:47 PM, Raymond Hettinger wrote:
the slightly obscure bit that requires some getting used to is that all these flags don't really mean "I have such and such feature" but just "I could have such and such feature, if the corresponding tp_xxx field were set".
I would like to see that cleaned-up for Py3k. Ideally, the NULL or non_NULL status of a slot should serve as its flag.
+1 TOOWTDI. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBRSwjRHEjvBPtnXfVAQJ0sQQAllzbSdONhCBWc/Rt0PW6J5iANLcm99N4 MkkSEDZBo72SsviijRvTha1+1pvpzB6s4Rf7EOw/OKnQ+a3u37w3BB966ag8WIN1 RItKubCVS6kTpg53BBnIX7P0CGSFFY36pEQm4nNe3G6RH4F0FwmIdv0WyJhSnnDR KRT9PHI9QY8= =bH9r -----END PGP SIGNATURE-----
Raymond Hettinger schrieb:
the slightly obscure bit that requires some getting used to is that all these flags don't really mean "I have such and such feature" but just "I could have such and such feature, if the corresponding tp_xxx field were set".
I would like to see that cleaned-up for Py3k. Ideally, the NULL or non_NULL status of a slot should serve as its flag.
The flag indicates that the field is even present. If you have an extension module from an earlier Python release (in binary form), it won't *have* the field, so you can't test whether it's null. Accessing it will get to some other place in the data segment, and interpreting it as a function pointer will cause a crash. That's why the flags where initially introduced; presence of the flag indicates that the field was there at compile time. Of course, if everybody would always recompile all extension modules for a new Python feature release, those flags weren't necessary. Regards, Martin
Martin v. Löwis wrote:
Of course, if everybody would always recompile all extension modules for a new Python feature release, those flags weren't necessary.
a dynamic registration approach would be even better, with a single entry point used to register all methods and hooks your C extension has implemented, and code on the other side that builds a properly initialized type descriptor from that set, using fallback functions and error stubs where needed. e.g. the impossible-to-write-from-scratch NoddyType struct initialization in http://docs.python.org/ext/node24.html would collapse to static PyTypeObject NoddyType; ... NoddyType = PyType_Setup("noddy.Noddy", sizeof(Noddy)); PyType_Register(NoddyType, PY_TP_DEALLOC, Noddy_dealloc); PyType_Register(NoddyType, PY_TP_DOC, "Noddy objects"); PyType_Register(NoddyType, PY_TP_TRAVERSE, Noddy_traverse); PyType_Register(NoddyType, PY_TP_CLEAR, Noddy_clear); PyType_Register(NoddyType, PY_TP_METHODS, Noddy_methods); PyType_Register(NoddyType, PY_TP_MEMBERS, Noddy_members); PyType_Register(NoddyType, PY_TP_INIT, Noddy_init); PyType_Register(NoddyType, PY_TP_NEW, Noddy_new); if (PyType_Ready(&NoddyType) < 0) return; (a preprocessor that generated this based on suitable "macro decorators" could be implemented in just over 8 lines of Python...) with this in place, we could simply remove all those silly NULL checks from the interpreter. </F>
I wrote:
PyType_Register(NoddyType, PY_TP_METHODS, Noddy_methods);
methods and members could of course be registered to, so the implementation can chose how to store them (e.g. short lists for smaller method lists, dictionaries for others). </F>
On 10/11/06, Fredrik Lundh <fredrik@pythonware.com> wrote:
Martin v. Löwis wrote:
Of course, if everybody would always recompile all extension modules for a new Python feature release, those flags weren't necessary.
a dynamic registration approach would be even better, with a single entry point used to register all methods and hooks your C extension has implemented, and code on the other side that builds a properly initialized type descriptor from that set, using fallback functions and error stubs where needed.
e.g. the impossible-to-write-from-scratch NoddyType struct initialization in
http://docs.python.org/ext/node24.html
would collapse to
static PyTypeObject NoddyType;
...
NoddyType = PyType_Setup("noddy.Noddy", sizeof(Noddy)); PyType_Register(NoddyType, PY_TP_DEALLOC, Noddy_dealloc); PyType_Register(NoddyType, PY_TP_DOC, "Noddy objects"); PyType_Register(NoddyType, PY_TP_TRAVERSE, Noddy_traverse); PyType_Register(NoddyType, PY_TP_CLEAR, Noddy_clear); PyType_Register(NoddyType, PY_TP_METHODS, Noddy_methods); PyType_Register(NoddyType, PY_TP_MEMBERS, Noddy_members); PyType_Register(NoddyType, PY_TP_INIT, Noddy_init); PyType_Register(NoddyType, PY_TP_NEW, Noddy_new); if (PyType_Ready(&NoddyType) < 0) return;
(a preprocessor that generated this based on suitable "macro decorators" could be implemented in just over 8 lines of Python...)
with this in place, we could simply remove all those silly NULL checks from the interpreter.
This is also has the benefit of making it really easy to grep for the function used for the tp_init field since there is no guarantee someone will keep the traditional field comments in their file (I usually grep for PyTypeObject until I find the type I want). If we went with C99 this wouldn't be an issue, but since I don't think that is necessarily in the cards I am very happy to go with this solution. It ends up feeling more like how Ruby does C extensions, and I have to admit I think they may have made it simpler than we have. And of course with the change Raymond put in for checking the PyMethodDef slots can also easily allow people to name methods based on what the slots would be named had it been defined in Python (which we might want to do anyway with the C constants to make it more readable and less obtuse to new extension writers; e.g. change PY_TP_NEW to PY__NEW__). And lastly, this approach makes sure that the basic requirement of what a type must have defined can be enforced in the PyType_Setup() method /F's proposing. +1 from me. -Brett
Hi Fredrik, On Wed, Oct 11, 2006 at 12:35:23PM +0200, Fredrik Lundh wrote:
NoddyType = PyType_Setup("noddy.Noddy", sizeof(Noddy));
It doesn't address the problem Martin explained (you can put neither NULLs nor stubs in tp_xxx fields that are beyond the C extension module's sizeof(Nobby)). But I imagine it could with a bit more tweaking. A bientot, Armin
Armin Rigo wrote:
NoddyType = PyType_Setup("noddy.Noddy", sizeof(Noddy));
It doesn't address the problem Martin explained (you can put neither NULLs nor stubs in tp_xxx fields that are beyond the C extension module's sizeof(Nobby)). But I imagine it could with a bit more tweaking.
umm. last time I checked, the tp fields lived in the type object, not in the instance. </F>
Fredrik Lundh wrote:
Martin v. Löwis wrote:
Of course, if everybody would always recompile all extension modules for a new Python feature release, those flags weren't necessary.
a dynamic registration approach would be even better, with a single entry point used to register all methods and hooks your C extension has implemented, and code on the other side that builds a properly initialized type descriptor from that set, using fallback functions and error stubs where needed.
e.g. the impossible-to-write-from-scratch NoddyType struct initialization in
http://docs.python.org/ext/node24.html
would collapse to
static PyTypeObject NoddyType;
Wouldn't that have to be a pointer to allow the Python runtime complete control of the structure size without recompiling the extension?: static PyTypeObject *NoddyType; NoddyType = PyType_Alloc("noddy.Noddy"); if (!NoddyType) return; PyType_Register(NoddyType, PY_TP_DEALLOC, Noddy_dealloc); PyType_Register(NoddyType, PY_TP_DOC, "Noddy objects"); PyType_Register(NoddyType, PY_TP_TRAVERSE, Noddy_traverse); PyType_Register(NoddyType, PY_TP_CLEAR, Noddy_clear); PyType_Register(NoddyType, PY_TP_METHODS, Noddy_methods); PyType_Register(NoddyType, PY_TP_MEMBERS, Noddy_members); PyType_Register(NoddyType, PY_TP_INIT, Noddy_init); PyType_Register(NoddyType, PY_TP_NEW, Noddy_new); if (PyType_Ready(NoddyType) < 0) return; Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org
Nick Coghlan wrote:
would collapse to
static PyTypeObject NoddyType;
Wouldn't that have to be a pointer to allow the Python runtime complete control of the structure size without recompiling the extension?:
static PyTypeObject *NoddyType;
yeah, that's a silly typo. or maybe I was thinking of something really clever that I can no longer remember.
NoddyType = PyType_Alloc("noddy.Noddy"); if (!NoddyType) return;
the fewer places you have to check for an error, the less chance you have to forget to do it. my proposal implied that the NULL check should be done in Ready. I've posted slightly cleaned up version of my rough proposal here: http://effbot.org/zone/idea-register-type.htm </F>
Fredrik Lundh wrote:
a dynamic registration approach would be even better, with a single entry point used to register all methods and hooks your C extension has implemented, and code on the other side that builds a properly initialized type descriptor from that set, using fallback functions and error stubs where needed.
I knocked out a prototype of this last week, emailed Mr. Lundh about it, then forgot about it. Would anyone be interested in taking a peek at it? I only changed one file to use this new-style initialization, sha256module.c. The resulting init_sha256() looks like this: PyMODINIT_FUNC init_sha256(void) { PyObject *m; SHA224type = PyType_New("_sha256.sha224", sizeof(SHAobject), NULL); if (SHA224type == NULL) return; PyType_SetPointer(SHA224type, pte_dealloc, &SHA_dealloc); PyType_SetPointer(SHA224type, pte_methods, &SHA_methods); PyType_SetPointer(SHA224type, pte_members, &SHA_members); PyType_SetPointer(SHA224type, pte_getset, &SHA_getseters); if (PyType_Ready(SHA224type) < 0) return; SHA256type = PyType_New("_sha256.sha256", sizeof(SHAobject), NULL); if (SHA256type == NULL) return; PyType_SetPointer(SHA256type, pte_dealloc, &SHA_dealloc); PyType_SetPointer(SHA256type, pte_methods, &SHA_methods); PyType_SetPointer(SHA256type, pte_members, &SHA_members); PyType_SetPointer(SHA256type, pte_getset, &SHA_getseters); if (PyType_Ready(SHA256type) < 0) return; m = Py_InitModule("_sha256", SHA_functions); if (m == NULL) return; } In a way this wasn't really a good showpiece for my code. The "methods", "members", and "getseters" structs still need to be passed in. However, I did change all four "as_" structures so you can set those directly. For instance, the "concat" as_sequence method for a PyString object would be set using PyType_SetPointer(PyString_Type, pte_sequence_concat, string_concat); (I actually converted the PyString object to my new code, but had chicken-and-egg initialization problems as a result and backed out of it. The code is still in the branch, just commented out.) Patch available for interested parties, /larry/
Larry Hastings wrote:
I knocked out a prototype of this last week, emailed Mr. Lundh about it, then forgot about it.
It's on my TODO list, so I haven't forgotten about it, but I've been (as usual) busy with other stuff. I'll get there, sooner or later. Posting this to the patch tracker and posting a note to the Py3K mailing list could be a good idea. </F>
participants (8)
-
"Martin v. Löwis" -
Armin Rigo -
Barry Warsaw -
Brett Cannon -
Fredrik Lundh -
Larry Hastings -
Nick Coghlan -
Raymond Hettinger