[Cython] [Python-Dev] C-level duck typing

Nathaniel Smith njs at pobox.com
Mon May 28 13:01:25 CEST 2012

On Mon, May 28, 2012 at 11:55 AM, mark florisson
<markflorisson88 at gmail.com> wrote:
> On 28 May 2012 11:41, Nathaniel Smith <njs at pobox.com> wrote:
>> On Mon, May 28, 2012 at 10:13 AM, mark florisson
>> <markflorisson88 at gmail.com> wrote:
>>> On 28 May 2012 09:54, mark florisson <markflorisson88 at gmail.com> wrote:
>>>> On 27 May 2012 23:12, Nathaniel Smith <njs at pobox.com> wrote:
>>>>> On Sun, May 27, 2012 at 10:24 PM, Dag Sverre Seljebotn
>>>>> <d.s.seljebotn at astro.uio.no> wrote:
>>>>>> On 05/18/2012 10:30 AM, Dag Sverre Seljebotn wrote:
>>>>>>> On 05/18/2012 12:57 AM, Nick Coghlan wrote:
>>>>>>>> I think the main things we'd be looking for would be:
>>>>>>>> - a clear explanation of why a new metaclass is considered too complex a
>>>>>>>> solution
>>>>>>>> - what the implications are for classes that have nothing to do with the
>>>>>>>> SciPy/NumPy ecosystem
>>>>>>>> - how subclassing would behave (both at the class and metaclass level)
>>>>>>>> Yes, defining a new metaclass for fast signature exchange has its
>>>>>>>> challenges - but it means that *our* concerns about maintaining
>>>>>>>> consistent behaviour in the default object model and avoiding adverse
>>>>>>>> effects on code that doesn't need the new behaviour are addressed
>>>>>>>> automatically.
>>>>>>>> Also, I'd consider a functioning reference implementation using a custom
>>>>>>>> metaclass a requirement before we considered modifying type anyway, so I
>>>>>>>> think that's the best thing to pursue next rather than a PEP. It also
>>>>>>>> has the virtue of letting you choose which Python versions to target and
>>>>>>>> iterating at a faster rate than CPython.
>>>>>>> This seems right on target. I could make a utility code C header for
>>>>>>> such a metaclass, and then the different libraries can all include it
>>>>>>> and handshake on which implementation becomes the real one through
>>>>>>> sys.modules during module initialization. That way an eventual PEP will
>>>>>>> only be a natural incremental step to make things more polished, whether
>>>>>>> that happens by making such a metaclass part of the standard library or
>>>>>>> by extending PyTypeObject.
>>>>>> So I finally got around to implementing this:
>>>>>> https://github.com/dagss/pyextensibletype
>>>>>> Documentation now in a draft in the NumFOCUS SEP repo, which I believe is a
>>>>>> better place to store cross-project standards like this. (The NumPy
>>>>>> docstring standard will be SEP 100).
>>>>>> https://github.com/numfocus/sep/blob/master/sep200.rst
>>>>>> Summary:
>>>>>>  - No common runtime dependency
>>>>>>  - 1 ns overhead per lookup (that's for the custom slot *alone*, no
>>>>>> fast-callable signature matching or similar)
>>>>>>  - Slight annoyance: Types that want to use the metaclass must be a
>>>>>> PyHeapExtensibleType, to make the binary layout work with how CPython makes
>>>>>> subclasses from Python scripts
>>>>>> My conclusion: I think the metaclass approach should work really well.
>>>>> Few quick comments on skimming the code:
>>>>> The complicated nested #ifdef for __builtin_expect could be simplified to
>>>>>  #if defined(__GNUC__) && (__GNUC__ > 2 || __GNUC_MINOR__ > 95)
>>>>> PyCustomSlots_Check should be called PyCustomSlots_CheckExact, surely?
>>>>> And given that, how can this code work if someone does subclass this
>>>>> metaclass?
>>>> I think we should provide a wrapper for PyType_Ready, which just
>>>> copies the pointer to the table and the count directly into the
>>>> subclass. If a user then wishes to add stuff, the user can allocate a
>>>> new memory region dynamically, memcpy the base class' stuff in there,
>>>> and append some entries.
>>> Maybe we should also allow each custom type to set a deallocator,
>>> since they are then heap types which can go out of scope. The
>>> metaclass can then call this deallocator to deallocate the table.
>> Custom types are plain old Python objects, they can use tp_dealloc.
> If I set etp_custom_slots to something allocated on the heap, then the
> (shared) metaclass would have to deallocate it. The tp_dealloc of the
> type itself would be called for its instances (which can be used to
> deallocate dynamically allocated memory in the objects if you use a
> custom slot "pointer offset").

Oh, I see. Right, the natural way to handle this would be have each
user define their own metaclass with the behavior they want. Another
argument for supporting multiple metaclasses simultaneously I guess...

- N

More information about the cython-devel mailing list