[Python-3000] A better way to initialize PyTypeObject

[Talin]

Guido van Rossum wrote:
> Have you thought much about the issue of different signature? The
> regular method table only has functions taking one or more objects and
> returning an object. (There are a few flags to indicate variations on
> the call args.) The special slots have all sorts of other signatures,
> some returning int, some returning void, some taking ints or C
> strings, etc. (And changing this would be a huge inefficiency in some
> cases.)
> 
> Not using some variant of C (or C99) struct initialization means more
> chance for undetected errors since you will necessarily have to cast
> everything to a standard type and then the compiler can't type-check
> that the function signature matches the slot's needs.

Yes, there will be some loss of type safety. I'm not sure what to say 
about that. (Although, there is one benefit that you will at least be 
able to declare the "self" argument as its concrete type, rather than as 
PyObject *.)

> Nevertheless, I kind of like this, and would like someone to produce a
> patch for this approach. It is my opinion that without working code it
> is too easy to overlook design flaws. Since you are keen to have this
> in 2.6, the solution would have to be backwards compatible with 2.5,
> so the patch wouldn't have to change every type initializer in the
> system -- it would only have to provide the complete machinery and
> convert a few example types to prove it works.
> 
> Any volunteers? Without code this proposal is dead in the water.

A patch won't be difficult to produce; It all depends on the 
availability of round 'tuits. Probably not this week at the very least.

I'm curious as to what folks would recommend as the best way to do the 
mapping of slot names to slot addresses as a static table in the VM. I'm 
assuming that the result of the mapping would be a "slot descriptor" 
containing the offset of the pointer, and a code indicating whether it 
goes into the main type object or one of the extension structures.

(Oh, and off topic, I have a procedural question about the list: Is it 
considered good etiquette to only reply to the list, and not the 
individuals whom you are replying to? If you "reply all", then folks 
might get two copies of the message, but if you reply to only the list, 
then non-subscribers posting via gmane might not get replies.)

> --Guido
> 
> On 11/28/06, Talin <talin at acm.org> wrote:
>> Guido van Rossum wrote:
>> > On 11/28/06, Talin <talin at acm.org> wrote:
>> >> Guido van Rossum wrote:
>> >> > Some comments:
>> >> >
>> >> > - Fredrik's solution makes one call per registered method. (I don't
>> >> > know if the patch he refers to follows that model.) That seems a 
>> fair
>> >> > amount of code for an average type -- I'm wondering if it's too 
>> early
>> >> > to worry about code bloat (I don't think the speed is going to
>> >> > matter).
>> >>
>> >> One other thought: The special constants could themselves be nothing
>> >> more than the offset into the PyTypeObject struct, i.e.:
>> >>
>> >>     #define SPECMETHOD_NEW ((const 
>> char*)offsetof(PyTypeObject,tp_new))
>> >
>> > I think this would cause too many issues with backwards compatibility.
>> >
>> > I like the idea much better to use special names (e.g. starting with a
>> > ".").
>> >
>> >> In the PyType_Ready code, you would see if the method name had a value
>> >> of less than sizeof(PyTypeObject); If so, then it's a special method
>> >> name, and you fill in the struct at the specified offset.
>> >>
>> >> So the interpretation of the table could be very simple and fast. 
>> It has
>> >> a slight disadvantage from the approach of using actual string 
>> names for
>> >> special methods, in that it doesn't allow the VM to silently
>> >> promote/demote methods to 'special' status.
>> >
>> > I think the interpretation will be fast enough (or else what you said
>> > about premature optimization earlier wouldn't be correct. :-)
>>
>> OK, based on these comments and the other feedback from this thread,
>> here's a more concrete proposal:
>>
>> == Method Table ==
>>
>> Method definitions are stored in a static table, identical in format to
>> the existing PyMethodDef table.
>>
>> For non-method initializers, the most commonly-used ones will be passed
>> in as parameters to the type creation function. Those that are less
>> commonly used can be written in as a secondary step after the type has
>> been created, or in some cases represented in the tp_members table.
>>
>> == Method Names ==
>>
>> As suggested by Guido, we use a naming convention to determine how a
>> method in the method table is handled. I propose that methods be divided
>> into three categories, which are "Normal", "Special", and "Internal"
>> methods, and which are interpreted slightly differently at type
>> initialization time.
>>
>> * Internal methods are those that have no equivalent Python name, such
>> as tp_free/tp_alloc. Internal methods names start with a dot ("."), so
>> tp_alloc would be represented by the string ".tp_alloc".
>>
>> Internal methods are always stored into a slot in the PyTypeObject. If
>> there is no corresponding slot for a given name, that is a runtime error.
>>
>> * Special methods have the double-underscore (__special__) naming
>> convention. A special method may or may not have a slot definition in
>> PyTypeObject. If there is such a slot, the method pointer will be stored
>> into it; If there is no such slot, then the method pointer is stored
>> into the class dict just like a normal method.
>>
>> Because the decision whether to put the method into a slot is made by
>> the VM, the set of available slots can be modified in future Python
>> releases without breaking existing code.
>>
>> * Normal methods are any methods that are neither special or internal.
>> They are not placed in a slot, but are simply stored in the class dict.
>>
>> Brett Cannon brought up the point about __getitem__ being ambiguous,
>> since there are two slots, one for lists and one for mappings. This is
>> handled as follows:
>>
>> The "mapping" version of __getitem__ is a special method, named
>> "__getitem__". The "list" version, however, is considered an internal
>> method (since it's more specialized), and has the name ".tp_getitem".
>>
>> Greg Ewing's point about "next" is handled as follows: A function named
>> "next" will never be treated as a special method name, since it does not
>> follow the naming convention of either internal or special names.
>> However, if you want to fill in the "tp_next" slot of the PyTypeObject,
>> you can use the string ".tp_next" rather than "next".
>>
>> == Type Creation ==
>>
>> For backwards compatibility, the existing PyType_Ready function will
>> continue to work on statically-declared PyTypeObject structures. A new
>> function, 'PyType_Create' will be added that creates a new type from the
>> input parameters and the method initialization tables as described
>> previously. The actual type record may be allocated dynamically, as
>> suggested by Greg Ewing.
>>
>> Structures such as tp_as_sequence which extend the PyTypeObject will be
>> created as needed, if there are any methods that require those extension
>> structures.
>>
>> == Backwards compatibility ==
>>
>> The existing PyType_Ready and C-style static initialization mechanism
>> will continue to work - the new method for type creation will coexist
>> alongside the old.
>>
>> It is an open question as to whether PyType_Ready should attempt to
>> interpret the special method names and fill in the PyTypeObject slots.
>> If it does, then PyType_Create can all PyType_Ready as a subroutine
>> during the type creation process.
>>
>> Otherwise, the only modifications to the interpreter will be the
>> creation of the new PyType_Create function and any required subroutines.
>> Existing code should be unaffected.
>>
>> -- Talin
>> _______________________________________________
>> Python-3000 mailing list
>> Python-3000 at python.org
>> http://mail.python.org/mailman/listinfo/python-3000
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/python-3000/guido%40python.org
>>
> 
>