[Python-ideas] Optional extra globals dict for function objects

[Neil Toronto]

Jim Jewett wrote:
> On 11/17/07, Neil Toronto <ntoronto at cs.byu.edu> wrote:
>> I set out trying to redo the 3.0 autosuper metaclass
>> in 2.5 without bytecode hacking and ran into a problem:
>>      a function's func_globals isn't polymorphic.
>> That is, the interpreter uses PyDict_* calls to access it,
>> and in one case (LOAD_GLOBAL), actually inlines
>> PyDict_GetItem manually.
> 
> (1)  Is this just one of the "this must be a real dict, not just any
> mapping" limits, or is there something else I'm missing?

That's all it is, yes.

> (2)  Isn't the func_globals already (a read-only reference to) the
> module's __dict__?  So is this really about changing the promise of
> the module type, instead of just about func_globals?

My original question was about extending (with an optional dictionary) 
the behavior of a function with regard to its func_globals. Because of 
speed concerns, I didn't suggest weakening the type constraint to allow 
just anything that meets the dict API.

> Note that weakening the module.__dict__ promise to only meeting the
> dict API would make it easier to implement the various
> speed-up-globals suggestions.

By "implement" do you mean proof-of-concept, final, or both? At least 
for proof-of-concept, I totally agree. And thanks for the use case 
(which sort of applies to my original flawed idea), my lack of which 
Brett has raked me over the coals for. :) (But it didn't hurt much!)

> And to be honest, I think that assuming
> a UserDict.DictMixin wouldn't be that bad.  How often is a module's
> dict used for anything time-critical except get (and maybe set,
> delete, iterate)?

I doubt that delete and iterate are common enough that they'd have to be 
regarded as time-critical. Maybe set - maybe. It hardly happens 
(especially compared to get), and when it does, it's almost never in a 
time-critical inner loop.

DictMixin is currently pure Python. That's a speed concern that wouldn't 
be *too* hard to address, I suppose.

>> I propose adding a read-only attribute func_extra_globals
>> to the function object, default NULL. In the interpreter loop,
>> global lookups try func_extra_globals first if it's not NULL.
> 
> Would this really be a global dict though, or just a closure inserted
> between the func and the normal globals?

Basically a customizable closure, yeah.

> Is the real problem that you can't change which variables are in a
> closure (rather than fully global) after the function is compiled?

Really, that's it. That's why I made the silly bytecode hack to insert 
function parameters, which actually works better than augmenting a 
function's globals with a polymorphic dict.

Assuming func_globals is a DictMixin is intriguing, though.

Neil