Re: [Python-Dev] Instance variable access and descriptors
At 12:23 AM 6/10/2007 +0300, Eyal Lotem wrote:
A. It will break code that uses instance.__dict__['var'] directly, when 'var' exists as a property with a __set__ in the class. I believe this is not significant. B. It will simplify getattr's semantics. Python should _always_ give precedence to instance attributes over class ones, rather than have very weird special-cases (such as a property with a __set__).
Actually, these are features that are both used and desirable; I've been using them both since Python 2.2 (i.e., for many years now). I'm -1 on removing these features from any version of Python, even 3.0.
C. It will greatly speed up instance variable access, especially when the class has a large mro.
...at the cost of slowing down access to properties and __slots__, by adding an *extra* dictionary lookup there. Note, by the way, that if you want to change attribute lookup semantics, you can always override __getattribute__ and make it work whatever way you like, without forcing everybody else to change *their* code.
I agree with Phillip with regard to the semantics. They are semantically desirable. However, there is a patch to add a mro cache to speed up these sorts of cases on the Python tracker, originally submitted by Armin Rigo. He saw ~20% speedups, others see less. It is currently just sitting there with no apparent activity. So if the overhead of mro lookups is that bothersome, it may be well worth your time to review the patch. URL: http://sourceforge.net/tracker/index.php?func=detail&aid=1700288&group_id=5470&atid=305470 -Kevin On 6/9/07, Phillip J. Eby <pje@telecommunity.com> wrote:
At 12:23 AM 6/10/2007 +0300, Eyal Lotem wrote:
A. It will break code that uses instance.__dict__['var'] directly, when 'var' exists as a property with a __set__ in the class. I believe this is not significant. B. It will simplify getattr's semantics. Python should _always_ give precedence to instance attributes over class ones, rather than have very weird special-cases (such as a property with a __set__).
Actually, these are features that are both used and desirable; I've been using them both since Python 2.2 (i.e., for many years now). I'm -1 on removing these features from any version of Python, even 3.0.
C. It will greatly speed up instance variable access, especially when the class has a large mro.
...at the cost of slowing down access to properties and __slots__, by adding an *extra* dictionary lookup there.
Note, by the way, that if you want to change attribute lookup semantics, you can always override __getattribute__ and make it work whatever way you like, without forcing everybody else to change *their* code.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/jacobs%40bioinformed.com
I must be missing something, as I really see no reason to keep the existing semantics other than backwards compatibility (which can be achieved by introducing a __fastattr__ or such). Can you explain under which situations or find any example situation where the existing semantics are desirable? As for the mro cache - thanks for pointing it out - I think it can serve as a platform for another idea that in conjunction with psyco, can possibly speed up CPython very significantly (will create a thread about this soon). Please note that speeding up the mro-lookup solves only half of the problem (if it was solved - which it seems not to have been), the more important half of the problem remains, allow me to emphasize: ALL instance attribute accesses look up in both instance and class dicts, when it could look just in the instance dict. This is made worse by the fact that the class dict lookup is more expensive (with or without the mro cache). Some code that accesses a lot of instance attributes in an inner loop can easily be sped up by a factor of 2 or more (depending on the size of the mro). Eyal On 6/10/07, Kevin Jacobs <jacobs@bioinformed.com> <bioinformed@gmail.com> wrote:
I agree with Phillip with regard to the semantics. They are semantically desirable. However, there is a patch to add a mro cache to speed up these sorts of cases on the Python tracker, originally submitted by Armin Rigo. He saw ~20% speedups, others see less. It is currently just sitting there with no apparent activity. So if the overhead of mro lookups is that bothersome, it may be well worth your time to review the patch.
URL: http://sourceforge.net/tracker/index.php?func=detail&aid=1700288&group_id=5470&atid=305470
-Kevin
On 6/9/07, Phillip J. Eby <pje@telecommunity.com> wrote:
At 12:23 AM 6/10/2007 +0300, Eyal Lotem wrote:
A. It will break code that uses instance.__dict__['var'] directly, when 'var' exists as a property with a __set__ in the class. I believe this is not significant. B. It will simplify getattr's semantics. Python should _always_ give precedence to instance attributes over class ones, rather than have very weird special-cases (such as a property with a __set__).
Actually, these are features that are both used and desirable; I've been using them both since Python 2.2 (i.e., for many years now). I'm -1 on removing these features from any version of Python, even
3.0.
C. It will greatly speed up instance variable access, especially when the class has a large mro.
...at the cost of slowing down access to properties and __slots__, by adding an *extra* dictionary lookup there.
Note, by the way, that if you want to change attribute lookup semantics, you can always override __getattribute__ and make it work whatever way you like, without forcing everybody else to change *their*
code.
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
http://mail.python.org/mailman/options/python-dev/jacobs%40bioinformed.com
Hi Eyal, On Sun, Jun 10, 2007 at 04:13:38AM +0300, Eyal Lotem wrote:
I must be missing something, as I really see no reason to keep the existing semantics other than backwards compatibility (which can be achieved by introducing a __fastattr__ or such).
Can you explain under which situations or find any example situation where the existing semantics are desirable?
The existing semantics are essential when dealing with metaclasses. Many of the descriptors of the 'type' class would stop working without it. For example, the fact that 'x.__class__' normally gives the type of 'x' for any object x relies on this. Reading the '__dict__' attribute of types is also based on this. Before proposing changes, be sure you understand exactly how the following works: >>> object.__class__ <type 'type'> >>> object.__dict__['__class__'] <attribute '__class__' of 'object' objects> >>> class A(object): ... pass >>> A.__dict__ <dictproxy object at 0xb7c98e6c> >>> A.__dict__['__dict__'] <attribute '__dict__' of 'A' objects> A bientot, Armin.
Phillip J. Eby wrote:
...at the cost of slowing down access to properties and __slots__, by adding an *extra* dictionary lookup there.
Rather than spend time tinkering with the lookup order, it might be more productive to look into implementing a cache for attribute lookups. That would help with method lookups as well, which are probably more frequent than instance var accesses. -- Greg
On Tue, Jun 12, 2007 at 08:10:26PM +1200, Greg Ewing wrote:
Phillip J. Eby wrote:
...at the cost of slowing down access to properties and __slots__, by adding an *extra* dictionary lookup there.
Rather than spend time tinkering with the lookup order, it might be more productive to look into implementing a cache for attribute lookups. That would help with method lookups as well, which are probably more frequent than instance var accesses.
Was wondering the same; specifically, hijacking pep280 celldict appraoch for this. Downside, this would break code that tries to do PyDict_* calls on a class tp_dict; haven't dug extensively, but I'm sure there are a few out there. Main thing I like about that approach is that it avoids the staleness verification crap, single lookup- it's there or it isn't. It would also be resuable for 280. If folks don't much like the hit from tracing back to a cell holding an actual value, could always implement it such that upon change, the change propagates out to instances registered (iow, change a.__dict__, it notifies b.__dict__ of the change, etc, till it hits a point where the change doesn't need to go further). ~harring
Hi, On Tue, Jun 12, 2007 at 08:10:26PM +1200, Greg Ewing wrote:
Rather than spend time tinkering with the lookup order, it might be more productive to look into implementing a cache for attribute lookups.
See patch #1700288. Armin
participants (6)
-
Armin Rigo -
Brian Harring -
Eyal Lotem -
Greg Ewing -
Kevin Jacobs <jacobs＠bioinformed.com> -
Phillip J. Eby