Bug? If not, how to work around it?

[Alex Martelli]

Michele Simionato wrote:
   ...
> The point is that if you define a fake special method via __getattr__, it
> works if it is called as F.__len___(f), but it DOES not work if it is
> called as len(f). I suspect Gonzalo's problem is the same with iter.

You're right.  Special methods are looked up in the 'slots' (tp_iter and
the like), for the purpose of Python operations and built-ins (iter() and
the like), and __getattr__, being dynamic, doesn't affect those slots.

E.g.:

def fakeiter(x):
    return iter('ciao')

class witer(type):
    def __getattr__(cls, name):
        if name == '__iter__':
            cls.__iter__ = fakeiter
            return cls.__iter__
        raise AttributeError, name

__metaclass__ = witer

class X: pass

try:
    for x in iter(X()):
        print x,
    print
except TypeError:
    print 'TypeError, as expected'
else:
    print "No TypeError (?!)"

try:
    for x in X().__iter__(): print x,
    print
except AttributeError:
    print 'AttributeError, as expected'
else:
    print "No AttributeError (?!)"

for x in X.__iter__(X()): print x,
print

try:
    for x in iter(X()):
        print x,
    print
except TypeError:
    print 'TypeError, alas'
else:
    print "No TypeError (good!)"

Once X.__iter__ IS finally accessed, it gets installed in the
class object, thereby affecting the slots and making iter happy.

But until that happens, the class's tp_iter is null, iter()
does not cause a __getattr__ call, and X().__iter__, being an
access on the instance, doesn't go up to the metaclass either.

 
> Built-in like iter, len, etc. look directly for special methods __iter__,
> __len__ etc., *without* checking if they are defined by __getattr__. I
> am not saying that this is necessarely a bug, I am saying that this is
> not documented and that at least three persons have been beaten by this

I agree.  The way the Nutshell puts it is "in the new-style object
model, implicit use of special methods always relies on the class-level
binding of the special method, if any" -- and this is correct and
complete, even though probably too terse for anything except perhaps
a Language Reference (which IS allowed to be very terse as long as
it's correct and complete).  __getattr__ is not a BINDING of the
special method, though it may be considered a DEFINITION of it, which
is why the current phrase in the Language Reference is not 100% correct
and complete -- only 99.44%, and I agree that the remaining 0.56%
_is_ a delicate defect in the documentation.

It would probably be more helpful to most readers if a section on
new-style classes, one on old-style classes, and/or one on special
methods, expanded on this issue just a wee little bit.  Alas, the
Language Reference being meant for language-lawyers only, it's hard
to argue that it should deign to clarify things, as long as they
_are_ expressed in ways that are correct and complete.

> issue in the last few months. Not me personally, hence the reason why I
> didn't submit a bug report, but this time (if Gonzalo is not going to do
> that), I will submit the report, unless Alex is going to prove that this
> is already documented ;)

It's not _correctly and completely_ documented, I have to agree -- the
docs use the potentialy-slightly-ambiguous verb 'define' rather than the
more specific one 'bind'.


Alex