[Python-ideas] "Loose" descriptors

[James Edwards]

On Sat, Mar 28, 2015 at 10:34 PM, David Wilson <dw+python-ideas at hmmz.org> wrote:
> FWIW you could probably experiment with this using PyPy's object spaces
> feature without modifying the interpreter:
>
>     http://pypy.readthedocs.org/en/latest/objspace-proxies.html
>

That's David, I'll look into these.  "Proxy" type objects were exactly
the motivation for my suggestion.  I'm working with very deep nested
dictionaries -- not the most efficient data structure, but they're a
very direct representation of what I'm trying to do.

I wanted to create an ItemProxy object, for example:

    class ItemProxy:
        def __init__(self, obj, key):
            self.obj = obj
            self.key = key

        def get(self):      return self.obj[self.key]
        def set(self, val): self.obj[self.key] = val


    d = {
        '0': {'a':'foo', 'b':'bar'},
        '1': {'c':'car', 'd':'baz'},
    }

    p = ItemProxy(d['0'], 'a')

    print(p.get())
    p.set('abc')
    print(p.get())

But instead of writing print(p.get()), and p.set('abc'), I wanted to
just write print(p), and p='abc'.

w.r.t. using eval(), since posting I've dug up this Mar 2011 email to
this list from Raymond Hettinger: "descriptors outside of classes"

    https://mail.python.org/pipermail/python-ideas/2011-March/009657.html

Here he describes a method for simulating the object.__getattribute__
indirection that descriptors provide explicitly by using eval and
passing a locals dictionary.


On Sun, Mar 29, 2015 at 7:23 PM, Nathaniel Smith <njs at pobox.com> wrote:
> It got implemented. In 3.5, you can do:
> # mymodule.py:
> class MyModuleType(types.ModuleType):
>     ...
> sys.modules[__name__].__class__ = MyModuleType
> and now accessing mymodule.someattr will print a message.

I saw this approach also, (I believe, I can't test this in 3.5
currently and fails on 3.4) using slightly different process but with
the same end result (and possible on 2.7, at least):
http://www.dr-josiah.com/2013/12/properties-on-python-modules.html

And while this certainly works, it is restricted to modules, and I'd
imagine breaks traditional static checking.  In order to not break
static checking, you could use the approach as described in the link,
but the process described there is certainly a workaround and not a
great solution.


On Sat, Mar 28, 2015 at 10:17 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> (1) Can modules use a dict subclass for __dict__?
> (2) Can you replace the __dict__ on the fly like this?
> (3) If not, how do you bootstrap the module to have a magic dict instead
>     of an ordinary dict?

If this were possible, it seems like it would provide the flexibility
necessary to implement "global" descriptors, or at least module-level
descriptors, but as far as I can tell, it is not.

Trying to directly replace __dict__ results in AttributeError:
readonly attribute

Even if you try to replace the module reference in sys.modules with a
class that implements a custom __dict__, it doesn't appear to be not
used.  For example:

    # mymod.py
    import sys

    class CustomDict(dict):
        def __getattribute__(self, *args, **kwargs):
            print(["getattribute", args, kwargs])
            return dict.__getattribute__(self, *args, **kwargs)

        def __setattr__(self, *args, **kwargs):
            print(["setattr", args, kwargs])
            return dict.__setattr__(*args, **kwargs)

        def __missing__(self, key):
            print("Missing:", key)
            return None

    class Module: pass

    bar = "bar"     # Some module attribute, that will get carried into module

    module = Module()
    module.__dict__ = CustomDict(globals())

    sys.modules[module.__name__] = module

When inspected, the dict type is "correct":

    print(type(mymod.__dict__))  # <class 'mymod.CustomDict'>

But when module attributes are accessed/set, the expected print
functions aren't shown:

    print(mymod.bar)    # bar (and nothing else)
    mymod.bar = "baz"   # (nothing)
    print(mymod.bar)    # baz (and nothing else)

    # Attempt to access undefined module attribute, hoping to trigger
__missing__
    print(mymod.zzz)    # AttributeError: 'Module' object has no attribute 'zzz'

If it were possible to do this on a per-module basis, I think it would
provide much added flexibility, while restricting the lookup penalty
to modules that chose to implement it.