[Python-Dev] PySequence_Fast_GET_ITEM in string join

[Andrew Dalke]

The Sourceforge tracker is kaputt so I'm sending it here, in part
because the effbot says it's interesting.

I can derive from list and override __getitem__

 >>> class Spam(list):
...   def __getitem__(self, i):
...     print i
...     return 2
...
 >>> Spam()[1]
1
2
 >>> Spam()[9]
9
2
 >>>

Now consider the following

 >>> class Spam(list):
...   def __getitem__(self, i):
...     print "Asking for", i
...     if i == 0: return "zero"
...     if i == 1: return "one"
...     raise IndexError, i
...
 >>> Spam()[1]
Asking for 1
'one'
 >>>

Spiffy!  For my next trick....

 >>> "".join(Spam())
''
 >>>

The relevant code in stringobject uses PySequence_Fast_GET_ITEM(seq, i),
which likely doesn't know about my derived __getitem__.

         p = PyString_AS_STRING(res);
         for (i = 0; i < seqlen; ++i) {
                 size_t n;
                 item = PySequence_Fast_GET_ITEM(seq, i);
                 n = PyString_GET_SIZE(item);
                 memcpy(p, PyString_AS_STRING(item), n);
                 p += n;
                 if (i < seqlen - 1) {
                         memcpy(p, sep, seplen);
                         p += seplen;
                 }
         }


The Unicode join has the same problem

 >>> class Spam(list):
...   def __getitem__(self, i):
...     print "Asking for", i
...     if i == 0: return "zero"
...     if i == 1: return u"one"
...     raise IndexError, i
...
 >>> "".join(Spam())
''

While if my class is not derived from list, everything is copacetic.

 >>> class Spam:
...   def __getitem__(self, i):
...     print "Asking for", i
...     if i == 0: return "zero"
...     if i == 1: return u"one"
...     raise IndexError, i
...
 >>> "".join(Spam())
Asking for 0
Asking for 1
Asking for 2
u'zeroone'
 >>>

Ditto for deriving from object.

 >>> class Spam(object):
...   def __getitem__(self, i):
...     print "Asking for", i
...     if i == 0: return "zero"
...     if i == 1: return "one"
...     raise IndexError, i
...
 >>> "".join(Spam())
Asking for 0
Asking for 1
Asking for 2
'zeroone'
 >>>

					Andrew
					dalke at dalkescientific.com