[Python-ideas] Fix that broken callable builtin

Fri Apr 17 23:39:10 CEST 2015

I think you've found an unintended and undocumented backdoor. I admit I
don't understand how this works in CPython. Overloaded operators like
__add__ or __call__ should be methods in the class, and we don't look for
them in the instance. But somehow defining them with @property works (I
guess because @property is in the class).

What's different for __call__ is that callable() exists. And this is
probably why I exorcised it Python 3.0 -- but apparently it's back. :-(

In the end callable() doesn't always produce a correct answer; but maybe we
can make it work in this case by first testing the class and then the
instance? Something like (untested):

def callable(x):
    return hasattr(x.__class__, '__call__') and hasattr(x, '__call__')

On Fri, Apr 17, 2015 at 2:19 PM, Ionel Cristian Mărieș <contact at ionelmc.ro>
wrote:

> __add__ as a property/descriptor seems to work fine, eg:
>
> >>> class C:
>> ...  @property
>> ...  def __add__(self):
>> ...   return lambda other: [self, other]
>> ...
>> >>> C() + C()
>> [<__main__.C object at 0x0000000003652AC8>, <__main__.C object at
>> 0x0000000003652CC0>]
>>
>
> Am I missing something?
>
>
> Thanks,
> -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
>
> On Sat, Apr 18, 2015 at 12:15 AM, Guido van Rossum <guido at python.org>
> wrote:
>
>> You won't have any more luck defining __add__ as a property -- just don't
>> do that.
>>
>> On how to implement a proxy, I'll let other explain. But this is not it.
>>
>> On Fri, Apr 17, 2015 at 2:04 PM, Ionel Cristian Mărieș <
>> contact at ionelmc.ro> wrote:
>>
>>> Well yes, from that example it look right, because the call operator
>>> uses the __call__ attribute from the type of the object. However, when the
>>> call operator gets the __call__ method it will actually use it as a
>>> descriptor. From that perspective it's inconsistent.
>>>
>>> Also there's the issue about not being able to implement a true proxy
>>> (as outlined before).
>>>
>>> What actually prevents this being fixed?
>>>
>>>
>>> Thanks,
>>> -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
>>>
>>> On Fri, Apr 17, 2015 at 11:55 PM, Guido van Rossum <guido at python.org>
>>> wrote:
>>>
>>>> I think you're fighting windmills. Like most special operations (e.g.
>>>> __add__), a __call__ attribute on the object does not work, i.e. it does
>>>> not make the object callable. E.g.
>>>>
>>>> $ python3
>>>> Python 3.5.0a2 (v3.5.0a2:0337bd7ebcb6, Mar  8 2015, 01:12:06)
>>>> [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>> >>> class C: pass
>>>> ...
>>>> >>> c = C()
>>>> >>> c.__call__ = lambda *a: a
>>>> >>> c()
>>>> Traceback (most recent call last):
>>>>   File "<stdin>", line 1, in <module>
>>>> TypeError: 'C' object is not callable
>>>> >>> callable(c)
>>>> False
>>>> >>> hasattr(c, '__call__')
>>>> True
>>>> >>>
>>>>
>>>> On Fri, Apr 17, 2015 at 1:45 PM, Ionel Cristian Mărieș <
>>>> contact at ionelmc.ro> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I had an issue today with the `callable` builtin because it doesn't
>>>>> correctly check that the object has the __call__ attribute.
>>>>>
>>>>> Effectively what `callable(a)` does is `hasattr(type(a), '__call__')`
>>>>> but that's not very straightforward. A more straightforward implementation
>>>>> would do something like `hasattr(a, '__call__')`.
>>>>>
>>>>> For example:
>>>>>
>>>>> Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600
>>>>>> 64 bit (AMD64)] on win32
>>>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>>> >>> callable
>>>>>> <built-in function callable>
>>>>>> >>> class A:
>>>>>> ...  @property
>>>>>> ...  def __call__(self):
>>>>>> ...   raise AttributeError('go away')
>>>>>> ...
>>>>>> >>> a = A()
>>>>>> >>> a
>>>>>> <__main__.A object at 0x000000000365B5C0>
>>>>>> >>> a.__call__
>>>>>> Traceback (most recent call last):
>>>>>>   File "<stdin>", line 1, in <module>
>>>>>>   File "<stdin>", line 4, in __call__
>>>>>> AttributeError: go away
>>>>>> >>> callable(a)
>>>>>> True
>>>>>> >>> # it should be False :(
>>>>>>
>>>>>
>>>>> So it boils down to this:
>>>>>
>>>>>> >>> hasattr(a, "__call__")
>>>>>> False
>>>>>> >>> hasattr(type(a), "__call__")
>>>>>> True
>>>>>
>>>>>
>>>>> My issue is that I didn't call `callable(type(a))` but just
>>>>> `callable(a)`. Clearly mismatching what happens when you do hasattr(a,
>>>>> "__call__").
>>>>>
>>>>> To put in contrast, this is legal and clearly indicates the
>>>>> descriptors are being used as expected:
>>>>>
>>>>> >>> class B:
>>>>>> ...  @property
>>>>>> ...  def __call__(self):
>>>>>> ...   return lambda: 1
>>>>>> ...
>>>>>> >>> b = B()
>>>>>> >>> b()
>>>>>> 1
>>>>>>
>>>>>
>>>>> There
>>>>> 's some more discussing in issue 23990
>>>>> <http://bugs.python.org/issue23990>
>>>>>  where I get slightly angry, sorry.
>>>>>
>>>>>
>>>>> So were is this change actually useful? Proxies! Since new-style
>>>>> objects in Python you cannot really proxy the callable aspect of objects,
>>>>> because `callable` just checks that a field is set in a C struct.
>>>>>  This is fairly inconvenient because you have to know upfront if your
>>>>> target is going to be callable or not.
>>>>>
>>>>>
>>>>>
>>>>> Thanks,
>>>>> -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
>>>>>
>>>>> _______________________________________________
>>>>> Python-ideas mailing list
>>>>> Python-ideas at python.org
>>>>> https://mail.python.org/mailman/listinfo/python-ideas
>>>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --Guido van Rossum (python.org/~guido)
>>>>
>>>
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150417/86ae4de7/attachment.html>