Python 3, new-style classes and __class__
I was looking through the errors which occur when running the test suite of Django's py3k branch under Python 3, and one particular set of errors caught my eye which is unrelated to the bytes/str dance. These errors occur in some Django utility code, which supplies a SimpleLazyObject (new-style) class [1]. This implements a proxy, which is initialised using a callable. The callable returns the object to be wrapped, and it's called when needed to set up the wrapped instance. The SimpleLazyObject needs to pretend to be the class of the wrapped object, e.g. for equality tests. This pretending is done by declaring __class__ as a property in SimpleLazyObject which fetches and returns the __class__ attribute of the wrapped object. This approach doesn't work in Python 3, however: the property named __class__ doesn't show up in the class dict of SimpleLazyObject, and moreover, there are restrictions on what you can set __class__ to - e.g. Python complains if you try and set a __class__ attribute on the instance to anything other than a new-style class. What's the simplest way in Python 3 of implementing the equivalent approach to pretending to be a different class? Any pointers appreciated. Thanks and regards, Vinay Sajip [1] http://goo.gl/1Jlbj
On 19/11/2011 22:06, Vinay Sajip wrote:
I was looking through the errors which occur when running the test suite of Django's py3k branch under Python 3, and one particular set of errors caught my eye which is unrelated to the bytes/str dance. These errors occur in some Django utility code, which supplies a SimpleLazyObject (new-style) class [1]. This implements a proxy, which is initialised using a callable. The callable returns the object to be wrapped, and it's called when needed to set up the wrapped instance.
The SimpleLazyObject needs to pretend to be the class of the wrapped object, e.g. for equality tests. This pretending is done by declaring __class__ as a property in SimpleLazyObject which fetches and returns the __class__ attribute of the wrapped object. This approach doesn't work in Python 3, however: the property named __class__ doesn't show up in the class dict of SimpleLazyObject, and moreover, there are restrictions on what you can set __class__ to - e.g. Python complains if you try and set a __class__ attribute on the instance to anything other than a new-style class.
What's the simplest way in Python 3 of implementing the equivalent approach to pretending to be a different class? Any pointers appreciated. That works fine in Python 3 (mock.Mock does it):
class Foo(object): ... @property ... def __class__(self): ... return int ... a = Foo() isinstance(a, int) True a.__class__ <class 'int'>
There must be something else going on here. All the best, Michael Foord
Thanks and regards,
Vinay Sajip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
That works fine in Python 3 (mock.Mock does it):
class Foo(object): ... @property ... def __class__(self): ... return int ... a = Foo() isinstance(a, int) True a.__class__ <class 'int'>
There must be something else going on here.
Michael, thanks for the quick response. Okay, I'll dig in a bit further: the definition in SimpleLazyObject is __class__ = property(new_method_proxy(operator.attrgetter("__class__"))) so perhaps the problem is something related to the specifics of the definition. Here's what I found in initial exploration: -------------------------------------------------------------------------- Python 2.7.2+ (default, Oct 4 2011, 20:06:09) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from django.utils.functional import SimpleLazyObject fake_bool = SimpleLazyObject(lambda: True) fake_bool.__class__ <type 'bool'> fake_bool.__dict__ {'_setupfunc': <function <lambda> at 0xca9ed8>, '_wrapped': True} SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0xca9de8>, '__deepcopy__': <function __deepcopy__ at 0xca9c08>, '__str__': <function inner at 0xca9b18>, '_setup': <function _setup at 0xca9aa0>, '__class__': <property object at 0xca5730>, '__hash__': <function inner at 0xca9d70>, '__unicode__': <function inner at 0xca9b90>, '__bool__': <function inner at 0xca9de8>, '__eq__': <function inner at 0xca9cf8>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0xca9a28> })
Python 3.2.2 (default, Sep 5 2011, 21:17:14) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from django.utils.functional import SimpleLazyObject fake_bool = SimpleLazyObject(lambda : True) fake_bool.__class__ <class 'django.utils.functional.SimpleLazyObject'> fake_bool.__dict__ { '_setupfunc': <function <lambda> at 0x1c36ea8>, '_wrapped': <object object at 0x1d88b70> } SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0x1f56490>, '__deepcopy__': <function __deepcopy__ at 0x1f562f8>, '__str__': <function inner at 0x1f561e8>, '_setup': <function _setup at 0x1f56160>, '__hash__': <function inner at 0x1f56408>, '__unicode__': <function inner at 0x1f56270>, '__bool__': <function inner at 0x1f56490>, '__eq__': <function inner at 0x1f56380>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0x1f560d8> })
In Python 3, there's no __class__ property as there is in Python 2, the fake_bool's type isn't bool, and the callable to set up the wrapped object never gets called (which is why _wrapped is not set to True, but to an anonymous object - this is set in SimpleLazyObject.__init__). Puzzling! Regards, Vinay Sajip
On 19 November 2011 23:11, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
That works fine in Python 3 (mock.Mock does it):
class Foo(object): ... @property ... def __class__(self): ... return int ... a = Foo() isinstance(a, int) True a.__class__ <class 'int'>
There must be something else going on here.
Michael, thanks for the quick response. Okay, I'll dig in a bit further: the definition in SimpleLazyObject is
__class__ = property(new_method_proxy(operator.attrgetter("__class__")))
so perhaps the problem is something related to the specifics of the definition. Here's what I found in initial exploration:
-------------------------------------------------------------------------- Python 2.7.2+ (default, Oct 4 2011, 20:06:09) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from django.utils.functional import SimpleLazyObject fake_bool = SimpleLazyObject(lambda: True) fake_bool.__class__ <type 'bool'> fake_bool.__dict__ {'_setupfunc': <function <lambda> at 0xca9ed8>, '_wrapped': True} SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0xca9de8>, '__deepcopy__': <function __deepcopy__ at 0xca9c08>, '__str__': <function inner at 0xca9b18>, '_setup': <function _setup at 0xca9aa0>, '__class__': <property object at 0xca5730>, '__hash__': <function inner at 0xca9d70>, '__unicode__': <function inner at 0xca9b90>, '__bool__': <function inner at 0xca9de8>, '__eq__': <function inner at 0xca9cf8>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0xca9a28> })
Python 3.2.2 (default, Sep 5 2011, 21:17:14) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from django.utils.functional import SimpleLazyObject fake_bool = SimpleLazyObject(lambda : True) fake_bool.__class__ <class 'django.utils.functional.SimpleLazyObject'> fake_bool.__dict__ { '_setupfunc': <function <lambda> at 0x1c36ea8>, '_wrapped': <object object at 0x1d88b70> } SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0x1f56490>, '__deepcopy__': <function __deepcopy__ at 0x1f562f8>, '__str__': <function inner at 0x1f561e8>, '_setup': <function _setup at 0x1f56160>, '__hash__': <function inner at 0x1f56408>, '__unicode__': <function inner at 0x1f56270>, '__bool__': <function inner at 0x1f56490>, '__eq__': <function inner at 0x1f56380>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0x1f560d8> })
In Python 3, there's no __class__ property as there is in Python 2, the fake_bool's type isn't bool, and the callable to set up the wrapped object never gets called (which is why _wrapped is not set to True, but to an anonymous object - this is set in SimpleLazyObject.__init__).
The Python compiler can do strange things with assignment to __class__ in the presence of super. This issue has now been fixed, but it may be what is biting you: http://bugs.python.org/issue12370 If this *is* the problem, then see the workaround suggested in the issue. (alias super to _super in the module scope and use the old style super calling convention.) Michael
Puzzling!
Regards,
Vinay Sajip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
The Python compiler can do strange things with assignment to __class__ in the presence of super. This issue has now been fixed, but it may be what is biting you:
http://bugs.python.org/issue12370
If this *is* the problem, then see the workaround suggested in the issue.
Yes, that workaround worked. Good catch - thanks! Regards, Vinay Sajip
Um, what?! __class__ *already* has a special meaning. Those examples violate that meaning. No wonder they get garbage results. The correct way to override isinstance is explained here: http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubcla... . --Guido On Sat, Nov 19, 2011 at 6:13 PM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 19 November 2011 23:11, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
That works fine in Python 3 (mock.Mock does it):
>>> class Foo(object): ... @property ... def __class__(self): ... return int ... >>> a = Foo() >>> isinstance(a, int) True >>> a.__class__ <class 'int'>
There must be something else going on here.
Michael, thanks for the quick response. Okay, I'll dig in a bit further: the definition in SimpleLazyObject is
__class__ = property(new_method_proxy(operator.attrgetter("__class__")))
so perhaps the problem is something related to the specifics of the definition. Here's what I found in initial exploration:
-------------------------------------------------------------------------- Python 2.7.2+ (default, Oct 4 2011, 20:06:09) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from django.utils.functional import SimpleLazyObject fake_bool = SimpleLazyObject(lambda: True) fake_bool.__class__ <type 'bool'> fake_bool.__dict__ {'_setupfunc': <function <lambda> at 0xca9ed8>, '_wrapped': True} SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0xca9de8>, '__deepcopy__': <function __deepcopy__ at 0xca9c08>, '__str__': <function inner at 0xca9b18>, '_setup': <function _setup at 0xca9aa0>, '__class__': <property object at 0xca5730>, '__hash__': <function inner at 0xca9d70>, '__unicode__': <function inner at 0xca9b90>, '__bool__': <function inner at 0xca9de8>, '__eq__': <function inner at 0xca9cf8>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0xca9a28> })
Python 3.2.2 (default, Sep 5 2011, 21:17:14) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from django.utils.functional import SimpleLazyObject fake_bool = SimpleLazyObject(lambda : True) fake_bool.__class__ <class 'django.utils.functional.SimpleLazyObject'> fake_bool.__dict__ { '_setupfunc': <function <lambda> at 0x1c36ea8>, '_wrapped': <object object at 0x1d88b70> } SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0x1f56490>, '__deepcopy__': <function __deepcopy__ at 0x1f562f8>, '__str__': <function inner at 0x1f561e8>, '_setup': <function _setup at 0x1f56160>, '__hash__': <function inner at 0x1f56408>, '__unicode__': <function inner at 0x1f56270>, '__bool__': <function inner at 0x1f56490>, '__eq__': <function inner at 0x1f56380>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0x1f560d8> })
In Python 3, there's no __class__ property as there is in Python 2, the fake_bool's type isn't bool, and the callable to set up the wrapped object never gets called (which is why _wrapped is not set to True, but to an anonymous object - this is set in SimpleLazyObject.__init__).
The Python compiler can do strange things with assignment to __class__ in the presence of super. This issue has now been fixed, but it may be what is biting you:
http://bugs.python.org/issue12370
If this *is* the problem, then see the workaround suggested in the issue. (alias super to _super in the module scope and use the old style super calling convention.)
Michael
Puzzling!
Regards,
Vinay Sajip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
--
May you do good and not evil May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
On 20 Nov 2011, at 16:35, Guido van Rossum wrote:
Um, what?! __class__ *already* has a special meaning. Those examples violate that meaning. No wonder they get garbage results.
The correct way to override isinstance is explained here: http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubcla... .
Proxy classes have been using __class__ as a descriptor for this purpose for years before ABCs were introduced. This worked fine up until Python 3 where the compiler magic broke it when super is used. That is now fixed anyway. If I understand correctly, ABCs are great for allowing classes of objects to pass isinstance checks (etc) - what proxy, lazy and mock objects need is to be able to allow individual instances to pass different isinstance checks. All the best, Michael Foord
--Guido
On Sat, Nov 19, 2011 at 6:13 PM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 19 November 2011 23:11, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
That works fine in Python 3 (mock.Mock does it):
> class Foo(object): ... @property ... def __class__(self): ... return int ... > a = Foo() > isinstance(a, int) True > a.__class__ <class 'int'>
There must be something else going on here.
Michael, thanks for the quick response. Okay, I'll dig in a bit further: the definition in SimpleLazyObject is
__class__ = property(new_method_proxy(operator.attrgetter("__class__")))
so perhaps the problem is something related to the specifics of the definition. Here's what I found in initial exploration:
-------------------------------------------------------------------------- Python 2.7.2+ (default, Oct 4 2011, 20:06:09) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from django.utils.functional import SimpleLazyObject fake_bool = SimpleLazyObject(lambda: True) fake_bool.__class__ <type 'bool'> fake_bool.__dict__ {'_setupfunc': <function <lambda> at 0xca9ed8>, '_wrapped': True} SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0xca9de8>, '__deepcopy__': <function __deepcopy__ at 0xca9c08>, '__str__': <function inner at 0xca9b18>, '_setup': <function _setup at 0xca9aa0>, '__class__': <property object at 0xca5730>, '__hash__': <function inner at 0xca9d70>, '__unicode__': <function inner at 0xca9b90>, '__bool__': <function inner at 0xca9de8>, '__eq__': <function inner at 0xca9cf8>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0xca9a28> })
Python 3.2.2 (default, Sep 5 2011, 21:17:14) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from django.utils.functional import SimpleLazyObject fake_bool = SimpleLazyObject(lambda : True) fake_bool.__class__ <class 'django.utils.functional.SimpleLazyObject'> fake_bool.__dict__ { '_setupfunc': <function <lambda> at 0x1c36ea8>, '_wrapped': <object object at 0x1d88b70> } SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0x1f56490>, '__deepcopy__': <function __deepcopy__ at 0x1f562f8>, '__str__': <function inner at 0x1f561e8>, '_setup': <function _setup at 0x1f56160>, '__hash__': <function inner at 0x1f56408>, '__unicode__': <function inner at 0x1f56270>, '__bool__': <function inner at 0x1f56490>, '__eq__': <function inner at 0x1f56380>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0x1f560d8> })
In Python 3, there's no __class__ property as there is in Python 2, the fake_bool's type isn't bool, and the callable to set up the wrapped object never gets called (which is why _wrapped is not set to True, but to an anonymous object - this is set in SimpleLazyObject.__init__).
The Python compiler can do strange things with assignment to __class__ in the presence of super. This issue has now been fixed, but it may be what is biting you:
http://bugs.python.org/issue12370
If this *is* the problem, then see the workaround suggested in the issue. (alias super to _super in the module scope and use the old style super calling convention.)
Michael
Puzzling!
Regards,
Vinay Sajip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
--
May you do good and not evil May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On Sun, Nov 20, 2011 at 10:44 AM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 20 Nov 2011, at 16:35, Guido van Rossum wrote:
Um, what?! __class__ *already* has a special meaning. Those examples violate that meaning. No wonder they get garbage results.
The correct way to override isinstance is explained here: http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubcla... .
Proxy classes have been using __class__ as a descriptor for this purpose for years before ABCs were introduced. This worked fine up until Python 3 where the compiler magic broke it when super is used. That is now fixed anyway.
Hm, okay. Though it's disheartening that it took three releases of 3.x to figure this out. And there was a PEP even!
If I understand correctly, ABCs are great for allowing classes of objects to pass isinstance checks (etc) - what proxy, lazy and mock objects need is to be able to allow individual instances to pass different isinstance checks.
Ah, oops. Yes, __instancecheck__ is for the class to override isinstance(inst, cls); for the *instance* to override apparently you'll need to mess with __class__. I guess my request at this point would be to replace '@__class__' with some other *legal* __identifier__ that doesn't clash with existing use -- I don't like the arbitrary use of @ here. --Guido
All the best,
Michael Foord
--Guido
On Sat, Nov 19, 2011 at 6:13 PM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 19 November 2011 23:11, Vinay Sajip <vinay_sajip@yahoo.co.uk> wrote:
Michael Foord <fuzzyman <at> voidspace.org.uk> writes:
That works fine in Python 3 (mock.Mock does it):
>>> class Foo(object): ... @property ... def __class__(self): ... return int ... >>> a = Foo() >>> isinstance(a, int) True >>> a.__class__ <class 'int'>
There must be something else going on here.
Michael, thanks for the quick response. Okay, I'll dig in a bit further: the definition in SimpleLazyObject is
__class__ = property(new_method_proxy(operator.attrgetter("__class__")))
so perhaps the problem is something related to the specifics of the definition. Here's what I found in initial exploration:
-------------------------------------------------------------------------- Python 2.7.2+ (default, Oct 4 2011, 20:06:09) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
> from django.utils.functional import SimpleLazyObject > fake_bool = SimpleLazyObject(lambda: True) > fake_bool.__class__ <type 'bool'> > fake_bool.__dict__ {'_setupfunc': <function <lambda> at 0xca9ed8>, '_wrapped': True} > SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0xca9de8>, '__deepcopy__': <function __deepcopy__ at 0xca9c08>, '__str__': <function inner at 0xca9b18>, '_setup': <function _setup at 0xca9aa0>, '__class__': <property object at 0xca5730>, '__hash__': <function inner at 0xca9d70>, '__unicode__': <function inner at 0xca9b90>, '__bool__': <function inner at 0xca9de8>, '__eq__': <function inner at 0xca9cf8>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0xca9a28> })
Python 3.2.2 (default, Sep 5 2011, 21:17:14) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
> from django.utils.functional import SimpleLazyObject > fake_bool = SimpleLazyObject(lambda : True) > fake_bool.__class__ <class 'django.utils.functional.SimpleLazyObject'> > fake_bool.__dict__ { '_setupfunc': <function <lambda> at 0x1c36ea8>, '_wrapped': <object object at 0x1d88b70> } > SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__': <function inner at 0x1f56490>, '__deepcopy__': <function __deepcopy__ at 0x1f562f8>, '__str__': <function inner at 0x1f561e8>, '_setup': <function _setup at 0x1f56160>, '__hash__': <function inner at 0x1f56408>, '__unicode__': <function inner at 0x1f56270>, '__bool__': <function inner at 0x1f56490>, '__eq__': <function inner at 0x1f56380>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__': <function __init__ at 0x1f560d8> })
In Python 3, there's no __class__ property as there is in Python 2, the fake_bool's type isn't bool, and the callable to set up the wrapped object never gets called (which is why _wrapped is not set to True, but to an anonymous object - this is set in SimpleLazyObject.__init__).
The Python compiler can do strange things with assignment to __class__ in the presence of super. This issue has now been fixed, but it may be what is biting you:
http://bugs.python.org/issue12370
If this *is* the problem, then see the workaround suggested in the issue. (alias super to _super in the module scope and use the old style super calling convention.)
Michael
Puzzling!
Regards,
Vinay Sajip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
--
May you do good and not evil May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
-- http://www.voidspace.org.uk/
May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
-- --Guido van Rossum (python.org/~guido)
On 20/11/2011 21:41, Guido van Rossum wrote:
On Sun, Nov 20, 2011 at 10:44 AM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 20 Nov 2011, at 16:35, Guido van Rossum wrote:
Um, what?! __class__ *already* has a special meaning. Those examples violate that meaning. No wonder they get garbage results.
The correct way to override isinstance is explained here: http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubcla... .
Proxy classes have been using __class__ as a descriptor for this purpose for years before ABCs were introduced. This worked fine up until Python 3 where the compiler magic broke it when super is used. That is now fixed anyway. Hm, okay. Though it's disheartening that it took three releases of 3.x to figure this out. And there was a PEP even!
If I understand correctly, ABCs are great for allowing classes of objects to pass isinstance checks (etc) - what proxy, lazy and mock objects need is to be able to allow individual instances to pass different isinstance checks. Ah, oops. Yes, __instancecheck__ is for the class to override isinstance(inst, cls); for the *instance* to override apparently you'll need to mess with __class__.
I guess my request at this point would be to replace '@__class__' with some other *legal* __identifier__ that doesn't clash with existing use -- I don't like the arbitrary use of @ here.
The problem with using a valid identifier name is that it leaves open the possibility of the same "broken" behaviour (removing from the class namespace) for whatever name we pick. That means we should document the name used - and it's then more likely that users will start to rely on this odd (but documented) internal implementation detail. This in turn puts a burden on other implementations to use the same mechanism, even if this is less than ideal for them. This is why a deliberately invalid identifier was picked. All the best, Michael Foord
--Guido
All the best,
Michael Foord
--Guido
On Sat, Nov 19, 2011 at 6:13 PM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 19 November 2011 23:11, Vinay Sajip<vinay_sajip@yahoo.co.uk> wrote:
Michael Foord<fuzzyman<at> voidspace.org.uk> writes:
That works fine in Python 3 (mock.Mock does it):
>>> class Foo(object): ... @property ... def __class__(self): ... return int ... >>> a = Foo() >>> isinstance(a, int) True >>> a.__class__ <class 'int'>
There must be something else going on here.
Michael, thanks for the quick response. Okay, I'll dig in a bit further: the definition in SimpleLazyObject is
__class__ = property(new_method_proxy(operator.attrgetter("__class__")))
so perhaps the problem is something related to the specifics of the definition. Here's what I found in initial exploration:
-------------------------------------------------------------------------- Python 2.7.2+ (default, Oct 4 2011, 20:06:09) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
>> from django.utils.functional import SimpleLazyObject >> fake_bool = SimpleLazyObject(lambda: True) >> fake_bool.__class__ <type 'bool'> >> fake_bool.__dict__ {'_setupfunc':<function<lambda> at 0xca9ed8>, '_wrapped': True} >> SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__':<function inner at 0xca9de8>, '__deepcopy__':<function __deepcopy__ at 0xca9c08>, '__str__':<function inner at 0xca9b18>, '_setup':<function _setup at 0xca9aa0>, '__class__':<property object at 0xca5730>, '__hash__':<function inner at 0xca9d70>, '__unicode__':<function inner at 0xca9b90>, '__bool__':<function inner at 0xca9de8>, '__eq__':<function inner at 0xca9cf8>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__':<function __init__ at 0xca9a28> })
Python 3.2.2 (default, Sep 5 2011, 21:17:14) [GCC 4.6.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.
>> from django.utils.functional import SimpleLazyObject >> fake_bool = SimpleLazyObject(lambda : True) >> fake_bool.__class__ <class 'django.utils.functional.SimpleLazyObject'> >> fake_bool.__dict__ { '_setupfunc':<function<lambda> at 0x1c36ea8>, '_wrapped':<object object at 0x1d88b70> } >> SimpleLazyObject.__dict__ dict_proxy({ '__module__': 'django.utils.functional', '__nonzero__':<function inner at 0x1f56490>, '__deepcopy__':<function __deepcopy__ at 0x1f562f8>, '__str__':<function inner at 0x1f561e8>, '_setup':<function _setup at 0x1f56160>, '__hash__':<function inner at 0x1f56408>, '__unicode__':<function inner at 0x1f56270>, '__bool__':<function inner at 0x1f56490>, '__eq__':<function inner at 0x1f56380>, '__doc__': '\n A lazy object initialised from any function.\n\n Designed for compound objects of unknown type. For builtins or objects of\n known type, use django.utils.functional.lazy.\n ', '__init__':<function __init__ at 0x1f560d8> })
In Python 3, there's no __class__ property as there is in Python 2, the fake_bool's type isn't bool, and the callable to set up the wrapped object never gets called (which is why _wrapped is not set to True, but to an anonymous object - this is set in SimpleLazyObject.__init__).
The Python compiler can do strange things with assignment to __class__ in the presence of super. This issue has now been fixed, but it may be what is biting you:
http://bugs.python.org/issue12370
If this *is* the problem, then see the workaround suggested in the issue. (alias super to _super in the module scope and use the old style super calling convention.)
Michael
Puzzling!
Regards,
Vinay Sajip
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
--
May you do good and not evil May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
-- --Guido van Rossum (python.org/~guido)
-- http://www.voidspace.org.uk/
May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html
On Mon, Nov 21, 2011 at 9:22 AM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 20/11/2011 21:41, Guido van Rossum wrote:
On Sun, Nov 20, 2011 at 10:44 AM, Michael Foord <fuzzyman@voidspace.org.uk> wrote:
On 20 Nov 2011, at 16:35, Guido van Rossum wrote:
Um, what?! __class__ *already* has a special meaning. Those examples violate that meaning. No wonder they get garbage results.
The correct way to override isinstance is explained here:
http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubcla... .
Proxy classes have been using __class__ as a descriptor for this purpose for years before ABCs were introduced. This worked fine up until Python 3 where the compiler magic broke it when super is used. That is now fixed anyway.
Hm, okay. Though it's disheartening that it took three releases of 3.x to figure this out. And there was a PEP even!
If I understand correctly, ABCs are great for allowing classes of objects to pass isinstance checks (etc) - what proxy, lazy and mock objects need is to be able to allow individual instances to pass different isinstance checks.
Ah, oops. Yes, __instancecheck__ is for the class to override isinstance(inst, cls); for the *instance* to override apparently you'll need to mess with __class__.
I guess my request at this point would be to replace '@__class__' with some other *legal* __identifier__ that doesn't clash with existing use -- I don't like the arbitrary use of @ here.
The problem with using a valid identifier name is that it leaves open the possibility of the same "broken" behaviour (removing from the class namespace) for whatever name we pick.
That means we should document the name used - and it's then more likely that users will start to rely on this odd (but documented) internal implementation detail. This in turn puts a burden on other implementations to use the same mechanism, even if this is less than ideal for them.
This is why a deliberately invalid identifier was picked.
Hm. There are many, many places in Python where a __special__ identifier is used in such a way that a user who stomps on it can cause themselves pain. This is why the language reference is quite serious about reserving *all* __special__ names and states that only documented uses of them are allowed (and at least implying that undocumented uses are not necessarily flagged as errors). While I see that PEP 3119 made a mistake in giving __class__ two different, incompatible special uses, I don't agree that this case is so special that we should use an "invalid" identifier. I don't see that the name use should actually be documented -- users should not make *any* use of undocumented __names__. Let's please continue the tradition of allowing experts to mess around creatively with internals. -- --Guido van Rossum (python.org/~guido)
participants (3)
-
Guido van Rossum -
Michael Foord -
Vinay Sajip