[Python-Dev] Why does __getitem__ slot of builtin call sequence methods first?

[Travis Oliphant]

Guido van Rossum wrote:

>On 10/1/05, Travis Oliphant <oliphant at ee.byu.edu> wrote:
>  
>
>>The new ndarray object of scipy core (successor to Numeric Python) is a
>>C extension type that has a getitem defined in both the as_mapping and
>>the as_sequence structure.
>>
>>The as_sequence mapping is just so PySequence_GetItem will work correctly.
>>
>>As exposed to Python the ndarray object has a .__getitem__  wrapper method.
>>
>>Why does this wrapper call the sequence getitem instead of the mapping
>>getitem method?
>>
>>Is there anyway to get at a mapping-style __getitem__ method from Python?
>>    
>>
>
>Hmm... I'm sure the answer is in typeobject.c, but that is one of the
>more obfuscated parts of Python's guts. I wrote it four years ago and
>since then I've apparently lost enough brain cells (or migrated them
>from language implementation to to language design service :) that I
>don't understand it inside out any more like I did while I was in the
>midst of it.
>
>However, I wonder if the logic isn't such that if you define both
>sq_item and mp_subscript, __getitem__ calls sq_item; I wonder if by
>removing sq_item it might call mp_subscript? Worth a try, anyway.
>
>  
>

Thanks for the tip.  I think I figured out the problem, and it was my 
misunderstanding of how types inherit in C that was the source of my 
problem.  

Basically, Python is doing what you would expect, the mp_item is used 
for __getitem__ if both mp_item and sq_item are present.  However, the 
addition of these descriptors  (and therefore the resolution of any 
comptetion for __getitem__ calls) is done  *before*  the inheritance of 
any slots takes place. 

The new ndarray object inherits from a "big" array object that doesn't 
define the sequence and buffer protocols (which have the size limiting 
int dependencing in their interfaces).   The ndarray object has standard 
tp_as_sequence and tp_as_buffer slots filled.  

Figuring the array object would inherit its tp_as_mapping protocol from 
"big" array (which it does just fine), I did not explicitly set that 
slot in its Type object.    Thus, when PyType_Ready was called on the 
ndarray object, the tp_as_mapping was NULL and so __getitem__ mapped to 
the sequence-defined version.  Later the tp_as_mapping slots were 
inherited but too late for __getitem__ to be what I expected.

The easy fix was to initialize the tp_as_mapping slot before calling 
PyType_Ready.    Hopefully, somebody else searching in the future for an 
answer to their problem will find this discussion useful.  

Thanks for your help,

-Travis