[Tutor] set locals

[spir]

On 12/18/2013 11:51 AM, eryksun wrote:
> On Tue, Dec 17, 2013 at 10:52 AM, spir <denis.spir at gmail.com> wrote:
>> is it at all possible to set new vars (or any symbol) into an existing scope
>> (typically locals())?
>>
>>      scope[name] = value
>> raises by me an error like:
>>      TypeError: 'mappingproxy' object does not support item assignment
>>
>> I guess 'mappingproxy' is the implementation name of a scope (here, local),
>> and I thought scopes were just dicts; so what is the issue? Do you see an
>> alternative?
>
> For a CPython function, built-in locals() creates/updates the frame's
> f_locals mapping against the fast locals by calling the C function
> PyFrame_FastToLocals. The reverse update can be done (but why?) with
> the help of an extension module or ctypes to call
> PyFrame_LocalsToFast. For example:
>
>      import sys
>      from ctypes import *
>
>      pythonapi.PyFrame_LocalsToFast.argtypes = [py_object, c_int]
>
>      def f(x, clear=0):
>          if not clear:
>              locals()['x'] += 1
>          else:
>              del locals()['x']
>          frame = sys._getframe()
>          pythonapi.PyFrame_LocalsToFast(frame, clear)
>          return x
>
>      >>> f(1)
>      2
>
> If cleared this way (akin to `del x`), the fast local for x is set to
> NULL (unbound), so trying to return it raises an UnboundLocalError:
>
>      >>> f(1, clear=1)
>      Traceback (most recent call last):
>        File "<stdin>", line 1, in <module>
>        File "<stdin>", line 9, in f
>      UnboundLocalError: local variable 'x' referenced before assignment
>
> There's no facilitated way to add new fast locals. The memory used for
> the frame's stack, fast locals, and closure cells is allocated when
> the frame is instantiated, based on attributes of the compiled code
> object.
>
> On the other hand, a class body is unoptimized (intentionally), so it
> uses the frame's f_locals mapping (that's a dict, unless you're using
> the PEP 3115 __prepare__ hook). The populated mapping gets passed to
> the metaclass __new__ (e.g. type.__new__), which copies it to a new
> dict for the class.

All right, I don't understand the details (not knowing anything about CPython's 
implentation internals), but the general scheme is clear enough, thank you!

> You can use locals() to your heart's content in a class body:
>
>      class Cls:
>          locals()['x'] = 'spam'
>
>      >>> Cls.x
>      'spam'

All right. So, when a user defines a grammar (parser) inside a class (as I like 
to do myself) they could call my utility naming func directly from _inside_ the 
class def.

class some_parser:	# no capital, it's a parser, not an actual class
     digit = Range("09")
     # lots of other pattern defs
     Pattern.name(locals())

I'll try it to see if setting inside a class's locals works fine... works! So, 
apparently, I don't need to special-case classes anymore, do I?

I still ask for your advice because, since I don't get all details, despite my 
quick trial I'm not 100% sure there aren't cases where it would not work. I just 
need to register copies of patterns in the given scope/namespace in case they 
already have a non-None .name attribute.

> A class's __dict__ attribute is a descriptor defined by the metaclass.
> In CPython, for example, this descriptor needs to know the offset into
> the PyTypeObject where the dict is stored. The code it calls is simple
> enough to include here:
>
>      static PyObject *
>      type_dict(PyTypeObject *type, void *context)
>      {
>          if (type->tp_dict == NULL) {
>              Py_INCREF(Py_None);
>              return Py_None;
>          }
>          return PyDictProxy_New(type->tp_dict);
>      }
>
> If you aren't familiar with descriptors, read the following:
>
> http://docs.python.org/3/howto/descriptor.html
>
> And try the following:
>
>      class Cls(object, metaclass=type):
>          pass
>
>      type_dict_descr = vars(type)['__dict__']
>      type_dict_descr.__get__(Cls, type)
>
> Calling the __get__ method eventually makes its way to the above C
> function type_dict, which creates the mappingproxy by calling
> PyDictProxy_New.
>
> The proxy wraps the type's dict to keep you from hacking it directly.
> The functions to do so aren't even defined (i.e. mp_ass_subscript and
> sq_ass_item). So the abstract C API function PyObject_SetItem just
> raises a TypeError. If you need to set attributes dynamically, use
> built-in setattr().

I have been familiar with Python metaclasses some years ago (used them to 
simulate toy languages in python itself as interpretor! ;-) including one 
prototype-based à la Lua, Self, Io or JS) but this did not give me any precise 
clue about the other side of the business, in particular descriptors. I remember 
however having stepped on them once or twice, but at the time I knew too few 
about language implementations in C to be able to get anything.

I'll have another look at them following your links, thank you very much, "eryksun"!

denis