[Cython] [Python-Dev] C-level duck typing

[mark florisson]

On 28 May 2012 12:01, Nathaniel Smith <njs at pobox.com> wrote:
> On Mon, May 28, 2012 at 11:55 AM, mark florisson
> <markflorisson88 at gmail.com> wrote:
>> On 28 May 2012 11:41, Nathaniel Smith <njs at pobox.com> wrote:
>>> On Mon, May 28, 2012 at 10:13 AM, mark florisson
>>> <markflorisson88 at gmail.com> wrote:
>>>> On 28 May 2012 09:54, mark florisson <markflorisson88 at gmail.com> wrote:
>>>>> On 27 May 2012 23:12, Nathaniel Smith <njs at pobox.com> wrote:
>>>>>> On Sun, May 27, 2012 at 10:24 PM, Dag Sverre Seljebotn
>>>>>> <d.s.seljebotn at astro.uio.no> wrote:
>>>>>>> On 05/18/2012 10:30 AM, Dag Sverre Seljebotn wrote:
>>>>>>>>
>>>>>>>> On 05/18/2012 12:57 AM, Nick Coghlan wrote:
>>>>>>>>>
>>>>>>>>> I think the main things we'd be looking for would be:
>>>>>>>>> - a clear explanation of why a new metaclass is considered too complex a
>>>>>>>>> solution
>>>>>>>>> - what the implications are for classes that have nothing to do with the
>>>>>>>>> SciPy/NumPy ecosystem
>>>>>>>>> - how subclassing would behave (both at the class and metaclass level)
>>>>>>>>>
>>>>>>>>> Yes, defining a new metaclass for fast signature exchange has its
>>>>>>>>> challenges - but it means that *our* concerns about maintaining
>>>>>>>>> consistent behaviour in the default object model and avoiding adverse
>>>>>>>>> effects on code that doesn't need the new behaviour are addressed
>>>>>>>>> automatically.
>>>>>>>>>
>>>>>>>>> Also, I'd consider a functioning reference implementation using a custom
>>>>>>>>> metaclass a requirement before we considered modifying type anyway, so I
>>>>>>>>> think that's the best thing to pursue next rather than a PEP. It also
>>>>>>>>> has the virtue of letting you choose which Python versions to target and
>>>>>>>>> iterating at a faster rate than CPython.
>>>>>>>>
>>>>>>>>
>>>>>>>> This seems right on target. I could make a utility code C header for
>>>>>>>> such a metaclass, and then the different libraries can all include it
>>>>>>>> and handshake on which implementation becomes the real one through
>>>>>>>> sys.modules during module initialization. That way an eventual PEP will
>>>>>>>> only be a natural incremental step to make things more polished, whether
>>>>>>>> that happens by making such a metaclass part of the standard library or
>>>>>>>> by extending PyTypeObject.
>>>>>>>
>>>>>>>
>>>>>>> So I finally got around to implementing this:
>>>>>>>
>>>>>>> https://github.com/dagss/pyextensibletype
>>>>>>>
>>>>>>> Documentation now in a draft in the NumFOCUS SEP repo, which I believe is a
>>>>>>> better place to store cross-project standards like this. (The NumPy
>>>>>>> docstring standard will be SEP 100).
>>>>>>>
>>>>>>> https://github.com/numfocus/sep/blob/master/sep200.rst
>>>>>>>
>>>>>>> Summary:
>>>>>>>
>>>>>>>  - No common runtime dependency
>>>>>>>
>>>>>>>  - 1 ns overhead per lookup (that's for the custom slot *alone*, no
>>>>>>> fast-callable signature matching or similar)
>>>>>>>
>>>>>>>  - Slight annoyance: Types that want to use the metaclass must be a
>>>>>>> PyHeapExtensibleType, to make the binary layout work with how CPython makes
>>>>>>> subclasses from Python scripts
>>>>>>>
>>>>>>> My conclusion: I think the metaclass approach should work really well.
>>>>>>
>>>>>> Few quick comments on skimming the code:
>>>>>>
>>>>>> The complicated nested #ifdef for __builtin_expect could be simplified to
>>>>>>  #if defined(__GNUC__) && (__GNUC__ > 2 || __GNUC_MINOR__ > 95)
>>>>>>
>>>>>> PyCustomSlots_Check should be called PyCustomSlots_CheckExact, surely?
>>>>>> And given that, how can this code work if someone does subclass this
>>>>>> metaclass?
>>>>>
>>>>> I think we should provide a wrapper for PyType_Ready, which just
>>>>> copies the pointer to the table and the count directly into the
>>>>> subclass. If a user then wishes to add stuff, the user can allocate a
>>>>> new memory region dynamically, memcpy the base class' stuff in there,
>>>>> and append some entries.
>>>>
>>>> Maybe we should also allow each custom type to set a deallocator,
>>>> since they are then heap types which can go out of scope. The
>>>> metaclass can then call this deallocator to deallocate the table.
>>>
>>> Custom types are plain old Python objects, they can use tp_dealloc.
>>>
>> If I set etp_custom_slots to something allocated on the heap, then the
>> (shared) metaclass would have to deallocate it. The tp_dealloc of the
>> type itself would be called for its instances (which can be used to
>> deallocate dynamically allocated memory in the objects if you use a
>> custom slot "pointer offset").
>
> Oh, I see. Right, the natural way to handle this would be have each
> user define their own metaclass with the behavior they want. Another
> argument for supporting multiple metaclasses simultaneously I guess...
>
> - N
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel

That bludgeons your constant time type check. It's easier to just
reserve an extra slot for a deallocator pointer :) It would probably
be set to NULL in the common case anyway, since you allocate your
slots statically.