[Python-ideas] Allow function __globals__ to be arbitrary mapping not just dict

[Terry Reedy]

On 3/18/2012 8:27 AM, Steven D'Aprano wrote:
> Currently, if you try to construct a function from parts, the mapping
> that becomes func.__globals__ must be an actual dict:
>
>
> py> class Mapping:
> ... def __getitem__(self, key):
> ... if key == 'y':
> ... return 42
> ... raise KeyError(key)
> ...
> py> from types import FunctionType
> py> f = lambda x: x + y
> py> g = FunctionType(f.__code__, Mapping(), 'g')
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> TypeError: function() argument 2 must be dict, not Mapping

The API of internal classes is intentionally not documented in the types 
module doc. I take this to mean that the api, if not the existence, of 
such classes, is implementation specific. Hence any such call is a 
'cpython' rather than generic python call.

Someone pointed out on python-list that FunctionType checks and rejects 
non-dict second args because the CPython ceval loop code for CPython 
LOAD_GLOBAL and STORE_GLOBAL opcodes directly access dicts rather than 
using the generic mapping interface.

> I propose to allow function.__globals__ to accept any mapping type.

The important question is whether the current ceval code in merely a 
holdover from ancient days when dict was the only mapping type and the 
user function type was not accessible to users, or whether the direct 
dict access is an important speed optimization that affects essentially 
all code run on cpython.  If not done already, experiments are needed to 
assess the degree of slowdown.

Something to keep in mind: LOAD_GLOBAL is *also* used to access builtins 
that are not in globals() itself via globals['__builtins__']. So not 
just any mapping will work as a replacement. See 'Alternative proposal' 
below.

> That, plus the new collections.ChainMap class in Python 3.3, would allow
> some interesting experiments with namespaces and scoping rules.
>
> E.g. if I want to write a function with a custom namespace, I have to do
> something like this:
>
> ns = ChainMap( ... ) # set up a namespace
> def func(a, ns=ns):
> x = a + ns['b']
> y = ns['some_func'](ns['c'])
> z = ns['another_func'](x, y)
> ns['d'] = (x, y, z)
> return ns['one_last_thing'](d)
>
> which is not a very natural way of writing code. But if we could use
> non-dict mappings as __globals__, I could write that function like this:
>
> ns = ChainMap( ... ) # set up a namespace
> def func(a):
> global d
> x = a + b
> y = some_func(c)
> z = another_func(x, y)
> d = (x, y, z)
> return one_last_thing(d)
>
>
> # This could be a decorator.
> func = FunctionType(func.__code__, ns, func.__name__)
>
> (By the way, ChainMap is only one possible example namespace.)

Alternative proposal: write a function to replace LOAD_GLOBAL and 
STORE_GLOBAL for non-builtin names with the opcodes to access a mapping 
passed in as an arg to the rewrite function or the function itself. The 
latter would be perhaps easier since the name of the replacement mapping 
would already be in the code object.

def f(a, _globals = {}):pass
f = reglobalize(f) # assume '_globals' is the replacement
# now call f with whatever 'global' dict you want on a per-call basis.

-- 
Terry Jan Reedy