[Cython] [Python-Dev] C-level duck typing

[mark florisson]

On 28 May 2012 09:54, mark florisson <markflorisson88 at gmail.com> wrote:
> On 27 May 2012 23:12, Nathaniel Smith <njs at pobox.com> wrote:
>> On Sun, May 27, 2012 at 10:24 PM, Dag Sverre Seljebotn
>> <d.s.seljebotn at astro.uio.no> wrote:
>>> On 05/18/2012 10:30 AM, Dag Sverre Seljebotn wrote:
>>>>
>>>> On 05/18/2012 12:57 AM, Nick Coghlan wrote:
>>>>>
>>>>> I think the main things we'd be looking for would be:
>>>>> - a clear explanation of why a new metaclass is considered too complex a
>>>>> solution
>>>>> - what the implications are for classes that have nothing to do with the
>>>>> SciPy/NumPy ecosystem
>>>>> - how subclassing would behave (both at the class and metaclass level)
>>>>>
>>>>> Yes, defining a new metaclass for fast signature exchange has its
>>>>> challenges - but it means that *our* concerns about maintaining
>>>>> consistent behaviour in the default object model and avoiding adverse
>>>>> effects on code that doesn't need the new behaviour are addressed
>>>>> automatically.
>>>>>
>>>>> Also, I'd consider a functioning reference implementation using a custom
>>>>> metaclass a requirement before we considered modifying type anyway, so I
>>>>> think that's the best thing to pursue next rather than a PEP. It also
>>>>> has the virtue of letting you choose which Python versions to target and
>>>>> iterating at a faster rate than CPython.
>>>>
>>>>
>>>> This seems right on target. I could make a utility code C header for
>>>> such a metaclass, and then the different libraries can all include it
>>>> and handshake on which implementation becomes the real one through
>>>> sys.modules during module initialization. That way an eventual PEP will
>>>> only be a natural incremental step to make things more polished, whether
>>>> that happens by making such a metaclass part of the standard library or
>>>> by extending PyTypeObject.
>>>
>>>
>>> So I finally got around to implementing this:
>>>
>>> https://github.com/dagss/pyextensibletype
>>>
>>> Documentation now in a draft in the NumFOCUS SEP repo, which I believe is a
>>> better place to store cross-project standards like this. (The NumPy
>>> docstring standard will be SEP 100).
>>>
>>> https://github.com/numfocus/sep/blob/master/sep200.rst
>>>
>>> Summary:
>>>
>>>  - No common runtime dependency
>>>
>>>  - 1 ns overhead per lookup (that's for the custom slot *alone*, no
>>> fast-callable signature matching or similar)
>>>
>>>  - Slight annoyance: Types that want to use the metaclass must be a
>>> PyHeapExtensibleType, to make the binary layout work with how CPython makes
>>> subclasses from Python scripts
>>>
>>> My conclusion: I think the metaclass approach should work really well.
>>
>> Few quick comments on skimming the code:
>>
>> The complicated nested #ifdef for __builtin_expect could be simplified to
>>  #if defined(__GNUC__) && (__GNUC__ > 2 || __GNUC_MINOR__ > 95)
>>
>> PyCustomSlots_Check should be called PyCustomSlots_CheckExact, surely?
>> And given that, how can this code work if someone does subclass this
>> metaclass?
>
> I think we should provide a wrapper for PyType_Ready, which just
> copies the pointer to the table and the count directly into the
> subclass. If a user then wishes to add stuff, the user can allocate a
> new memory region dynamically, memcpy the base class' stuff in there,
> and append some entries.

Maybe we should also allow each custom type to set a deallocator,
since they are then heap types which can go out of scope. The
metaclass can then call this deallocator to deallocate the table.

>> Stealing a flag bit (but now to indicate this metaclass) would allow
>> us to make a real PyCustomSlots_Check function that was still fast. It
>> would also mean that different implementations didn't have to
>> rendezvous on a single PyExtensibleType_Type, so long as they all used
>> the same flag bit. That would let us skip monkeying around with
>> sys.modules.
>>
>> Speaking of which, surely we should not be using sys.modules for this?
>> Stashing it in sys itself or something would make more sense, if we're
>> going to do it at all.
>
> I think a module makes sense, if mangled appropriately. A module
> really means shared state (even if the only state are the functions
> and classes).
>
>> - N
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel