[Python-Dev] Idea: Dictionary references

Franklin? Lee leewangzhong+python at gmail.com
Thu Dec 17 14:19:43 EST 2015


On Thu, Dec 17, 2015 at 12:30 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
> On Dec 17, 2015, at 07:38, Franklin? Lee <leewangzhong+python at gmail.com> wrote:
>>
>> The nested dictionaries are only for nested scopes (and inner
>> functions don't create nested scopes). Nested scopes will already
>> require multiple lookups in parents.
>
> I think I understand what you're getting at here, but it's a really confusing use of terminology. In Python, and in programming in general, nested scopes refer to exactly inner functions (and classes) being lexically nested and doing lookup through outer scopes. The fact that this is optimized at compile time to FAST vs. CELL vs. GLOBAL/NAME, cells are optimized at function-creation time, and only global and name have to be resolved at the last second doesn't mean that there's no scoping, or some other form of scoping besides lexical. The actual semantics are LEGB, even if L vs. E vs. GB and E vs. further-out E can be optimized.

Oh, I've never actually read the Python scoping rules spelled out. I
wasn't sure if there were other cases.

The other two cases I thought of as "nesting" were: object to its
class, and class to its superclasses.

> Also, reading your earlier post, it sounds like you're trying to treat attribute lookup as a special case of global lookup, only with a chain of superclasses beyond the class instead of just a single builtins. But they're totally different. Class lookup doesn't just look in a series of dicts, it calls __getattribute__ which usually calls __getattr__ which may or may not look in the __dict__s (which may not even exist) to find a descriptor and then calls its __get__ method to get the value. You'd have to somehow handle the case where the search only went through object.__getattribute__ and __getattr__ and found a result by looking in a dict, to make a RefCell to that dict which is marked in some way that says "I'm not a value, I'm a descriptor you have to call each time", and then apply some guards that will detect whether that class or any intervening class dict touched that key, whether the MRO changed, whether that class or any intervening class added or changed implementations for __getatttibute__ or __getattr__, and probably more things I haven't thought of. What do those guards look like? (Also, you need a different set of rules to cache, and guard for, special method lookup--you could just ignore that, but I think those are the lookups that would benefit most from optimization.)

Doesn't __getattr__ only get called if all the mro __dict__ lookups failed?

I forgot about __getattribute__. That might be the point at which refs
are optimized.

As for descriptors versus RefCells, I'm guessing that can be resolved,
as soon as I figure out how descriptors actually work... If
descriptors don't modify the __dict__, then RefCells shouldn't get
involved. If they do, then there's some unwrapping going on there, and
RefCells should fit right in (though whether they'll improve anything
is a different question). RefCells are just a shortcut for dict
lookups.

For guards, I think Victor Stinner's idea could supplement this.
Alternatively, in my other email, I said there could be a rule of,
"Create intermediate RefCells for anything BEFORE a successful
lookup." So if we look in A, B, C, D, and find it in C, then we create
and save RefCells in A, B, C, but not D (where D = object). This MIGHT
result in a lot of intermediate RefCells, but I'd guess most things
aren't looked up just once, and it's saying, "It's possible for B to
gain member B.x and catch me on my way to C.x."

> So, trying to generalize global vs. builtin to a general notion of "nested scope" that isn't necessary for builtins and doesn't work for anything else seems like overcomplicating things for no benefit.

Probably. The globals() and __builtin__ case is simpler than the class case.


More information about the Python-Dev mailing list