[Cython] [Python-Dev] C-level duck typing

[Nathaniel Smith]

On Sun, May 27, 2012 at 10:24 PM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 05/18/2012 10:30 AM, Dag Sverre Seljebotn wrote:
>>
>> On 05/18/2012 12:57 AM, Nick Coghlan wrote:
>>>
>>> I think the main things we'd be looking for would be:
>>> - a clear explanation of why a new metaclass is considered too complex a
>>> solution
>>> - what the implications are for classes that have nothing to do with the
>>> SciPy/NumPy ecosystem
>>> - how subclassing would behave (both at the class and metaclass level)
>>>
>>> Yes, defining a new metaclass for fast signature exchange has its
>>> challenges - but it means that *our* concerns about maintaining
>>> consistent behaviour in the default object model and avoiding adverse
>>> effects on code that doesn't need the new behaviour are addressed
>>> automatically.
>>>
>>> Also, I'd consider a functioning reference implementation using a custom
>>> metaclass a requirement before we considered modifying type anyway, so I
>>> think that's the best thing to pursue next rather than a PEP. It also
>>> has the virtue of letting you choose which Python versions to target and
>>> iterating at a faster rate than CPython.
>>
>>
>> This seems right on target. I could make a utility code C header for
>> such a metaclass, and then the different libraries can all include it
>> and handshake on which implementation becomes the real one through
>> sys.modules during module initialization. That way an eventual PEP will
>> only be a natural incremental step to make things more polished, whether
>> that happens by making such a metaclass part of the standard library or
>> by extending PyTypeObject.
>
>
> So I finally got around to implementing this:
>
> https://github.com/dagss/pyextensibletype
>
> Documentation now in a draft in the NumFOCUS SEP repo, which I believe is a
> better place to store cross-project standards like this. (The NumPy
> docstring standard will be SEP 100).
>
> https://github.com/numfocus/sep/blob/master/sep200.rst
>
> Summary:
>
>  - No common runtime dependency
>
>  - 1 ns overhead per lookup (that's for the custom slot *alone*, no
> fast-callable signature matching or similar)
>
>  - Slight annoyance: Types that want to use the metaclass must be a
> PyHeapExtensibleType, to make the binary layout work with how CPython makes
> subclasses from Python scripts
>
> My conclusion: I think the metaclass approach should work really well.

Few quick comments on skimming the code:

The complicated nested #ifdef for __builtin_expect could be simplified to
  #if defined(__GNUC__) && (__GNUC__ > 2 || __GNUC_MINOR__ > 95)

PyCustomSlots_Check should be called PyCustomSlots_CheckExact, surely?
And given that, how can this code work if someone does subclass this
metaclass?

Stealing a flag bit (but now to indicate this metaclass) would allow
us to make a real PyCustomSlots_Check function that was still fast. It
would also mean that different implementations didn't have to
rendezvous on a single PyExtensibleType_Type, so long as they all used
the same flag bit. That would let us skip monkeying around with
sys.modules.

Speaking of which, surely we should not be using sys.modules for this?
Stashing it in sys itself or something would make more sense, if we're
going to do it at all.

- N