[Python-ideas] generic Liftable abc-mixin breaks at MRO

[Stephan Sahm]

I now created both, and in fact the code for Liftable-signal seems much
cleaner
Nevertheless, the code is rather long for this thread, but it might be
useful for someone

import abc
import inspect
from contextlib import contextmanager

def use_as_needed(func, kwargs):
    meta = inspect.getargspec(func)
    if meta.keywords is not None:
            return func(**kwargs)
    else:
        # not generic super-constructor - pick only the relevant subentries:
        return func(**{k:kwargs[k] for k in kwargs if k in meta.args})

class NotLiftable(RuntimeError):
    pass

@contextmanager
def super_liftable(cls, self):
    """ this is kind of a hack to replace super.super, however I haven't
found any other nice way to do it """
    if cls is object:
        raise NotLiftable()
    liftables = [l for l in cls.__bases__ if type(l).__name__ == "Liftable"]
    if not liftables:
        raise NotLiftable()

    orig_class = self.__class__
    self.__class__ = liftables[0]
    yield self
    self.__class__ = orig_class


def LiftableFrom(base_cls_name):

    class Liftable(type):
        def __init__(cls, name, bases, dct):
            # for base_cls nothing should be done, as this is the one to
refer to by Lifting
            if not cls.__name__ == base_cls_name:
                if "__init__" in dct:
                    raise TypeError("Descendents of Liftable are not
allowed to have own __init__ method. Instead overwrite __initialize__")

                def lifted__init__(self, **kwargs):
                    with super_liftable(cls, self) as s:
                        use_as_needed(s.__init__, kwargs)
                    if hasattr(self, "__initialize__"):
                        use_as_needed(self.__initialize__, kwargs)

                cls.__init__ = lifted__init__
                #setattr(cls, "__init__", lifted__init__)

            super(Liftable, cls).__init__(name, bases, dct)

    Liftable.base_cls_name = base_cls_name
    #Liftable.__name__ = "LiftableFrom" + base_cls_name   # to show that
this is possible
    return Liftable


def lift(self, new_class, **kwargs): #TODO adapt to work with both
definitions above
    # Stop Conditions:
    if self.__class__ is new_class:
        return # nothing to do
    elif new_class is object: # Base Case
        # break recursion at once:
        raise NotLiftable()

    ls = [l for l in new_class.__bases__ if type(l).__name__ == "Liftable"]
    if not ls:
        raise NotLiftable()

    # recursive case:
    if not self.__class__ is ls[0]: # it would also be possible to use tree
like left-first-search here
        lift(self, ls[0], **kwargs)
    # own case:
    self.__class__ = new_class
    use_as_needed(self.__initialize__, kwargs)


The least beautiful thing needed is too give the name of the base-class (in
this case "A") as an additional parameter for the lift-meta-class. I
haven't found a way to access A.__name__ directly or even better
automatically get the class where this meta-class was originally inserted.
A second point is that I had to use an own version of super(), however this
works like a charm as far as I can see.


The Metaclass ensures that any child must not have a __init__ but instead
can only use __initialize__ like a replacement. Here an example which works:


class A(object):
    __metaclass__ = LiftableFrom("A")
    def __init__(self, a):
        self.a = a

class B(A):
    def __initialize__(self, b):
        print "initialize b"
        self.b = b

class C(B):
    def __initialize__(self, c):
        print "initialize c"
        self.c = c

a = A(a=1)
a.a
# 1

lift(a, C, b=2, c=3)
print type(a)
print a.a, a.b, a.c
#initialize b
#initialize c
#<class '__main__.C'>
#1 2 3


​cheers,
Stephan​


On 10 December 2015 at 22:05, Stephan Sahm <Stephan.Sahm at gmx.de> wrote:

> Dear Andrew,
>
> thank you very much for this impressively constructive response. It is for
> sure more constructive than I can react on now.
>
> For the concrete usecase, the Liftable-signal in might be the most
> interesting option, as then already the base class can inherit the
> Liftable-signal and can itself already use lift.
> However, I cannot see how to make the __init__ method conform in this
> setting, but by inidividual implementations (I in fact thought that
> enforcing it by the mixin makes things safer, and it of course should
> reduce boilerplate code)
>
> The Liftable(T) in fact seems also great, as I cannot see how to avoid
> this lifting from a false class in the Liftable-signal chain. I only want
> to Lift from one class at the moment, so this is in fact kind of what I was
> after. The rough outline would look like
>
> class B(Lift(A)):
>     pass
> class C(Lift(B)):
>     pass
>
>
> which seems rather beautiful to read - thank you very much for pointing
> this out.
>
> If you have an idea how to automatically create the right __init__ method
> when using the Liftable-signal-chain, I would highly welcome it.
>
> I myself need to recap some metaclass basics again before seriously
> tackling this.
> Best,
> Stephan
>
>
>
> On 10 December 2015 at 19:38, Andrew Barnert <abarnert at yahoo.com> wrote:
>
>> On Dec 10, 2015, at 00:58, Stephan Sahm <Stephan.Sahm at gmx.de> wrote:
>>
>> Dear all,
>>
>> I think I found a crucial usecase where the standard MRO does not work
>> out. I would appreciate your help to still solve this usecase, or might MRO
>> even be adapted?
>>
>>
>> First, do you have an actual use case for this? And are you really
>> looking to suggest changes for Python 3.6, or looking for help with using
>> Python 2.7 as-is?
>>
>> Anyway, I think the first problem here is that you're trying to put the
>> same class, Liftable, on the MRO twice. That doesn't make sense--the whole
>> point of superclass linearization is that each class only appears once in
>> the list.
>>
>> If you weren't using a metaclass, you wouldn't see this error--but then
>> you'd just get the more subtle problem that C can't be lifted from A to B
>> because Liftable isn't getting called there.
>>
>> If you made Liftable a class factory, so two calls to Liftable() returned
>> different class objects, then you might be able to make this work. (I
>> suppose you could hide that from the user by giving Liftable a custom
>> metaclass that constructs new class objects for each copy of Liftable in
>> the bases list before calling through to type, but that seems like magic
>> you really don't want to hide if you want anyone to be able to debut this
>> code.)
>>
>> In fact, you could even make it Liftable(T), which makes your type
>> Liftable from T, rather than from whatever class happens to come after you
>> on the MRO chain. (Think about how this would work with other mixins, or
>> pure-interface ABCs, or full multiple inheritance--you may end up declaring
>> that C can be lifted from Sequence rather than B, which is nonsense, and
>> which will be hard to debug if you don't understand the C3 algorithm.)
>>
>> Or, if you actually _want_ to be liftable from whatever happens to come
>> next, then isn't liftability a property of the entire tree of classes, not
>> of individual classes in that tree, so you should only be specifying
>> Liftable once (either at A, or at B) in the hierarchy in the first place?
>> From what I can tell, the only benefit you get from installing it twice is
>> tricking the ABCMeta machinery into enforcing that all classes implement
>> _initialize_ instead of just enforcing that one does; the easy solution
>> there is to just write your own metaclass that does that check directly.
>>
>> Or maybe, instead of enforcing it, use it as a signal: build a "lift
>> chain" for each Liftable type out of all classes on the MRO that directly
>> implement _initialize_ (or just dynamically look for it as you walk the MRO
>> in lift). So lift only works between those classes. I think that gets you
>> all the same benefits as Liftable(T), without needing a class factory, and
>> without having to specify it more than once on a hierarchy.
>>
>> The idea is to build a generic Lift-type which I call this way because
>> the derived classes should be able to easily lift from subclasses. So for
>> example if I have an instance *a* from *class A* and a *class B(A)* I
>> want to make *a* an instance of *B* in a straightforward way.
>>
>> My implementation (Python 2.7):
>>
>> import abc
>> import inspect
>>
>> def use_as_needed(func, kwargs):
>>     meta = inspect.getargspec(func)
>>     if meta.keywords is not None:
>>         return meta(**kwargs)
>>     else:
>>         # not generic super-constructor - pick only the relevant
>> subentries:
>>         return func(**{k:kwargs[k] for k in kwargs if k in meta.args})
>>
>> class Liftable(object):
>>     __metaclass__ = abc.ABCMeta
>>
>>     def __init__(self, **kwargs):
>>         use_as_needed(super(Liftable,self).__init__, kwargs)
>>         use_as_needed(self.__initialize__, kwargs)
>>
>>     @abc.abstractmethod
>>     def __initialize__(self, **kwargs):
>>         return NotImplemented()
>>
>> class NoMatchingAncestor(RuntimeError):
>>     pass
>>
>> class NotLiftable(RuntimeError):
>>     pass
>>
>> def lift(self, new_class, **kwargs):
>>     # Stop Conditions:
>>     if self.__class__ is new_class:
>>         return # nothing to do
>>     elif new_class is object: # Base Case
>>         # break recursion at once:
>>         raise NoMatchingAncestor()
>>     elif new_class.__base__ is not Liftable: #to ensure this is save
>>         raise NotLiftable("Class {} is not Liftable (must be first
>> parent)".format(new_class.__name__))
>>
>>     # recursive case:
>>     if not self.__class__ is new_class.__bases__[1]:
>>         lift(self, new_class.__bases__[1], **kwargs)
>>     # own case:
>>     self.__class__ = new_class
>>     use_as_needed(self.__initialize__, kwargs)
>>
>>
>> and the example usecase:
>>
>> class A(object):
>>     def __init__(self, a):
>>         self.a = a
>>
>> class B(Liftable, A):
>>     def __initialize__(self, b):
>>         self.b = b
>>
>> a = A(1)
>> print a.a, a.__class__
>> # 1 <class '__main__.A'>
>>
>> lift(a, B, b=2)
>> print a.a, a.b, a.__class__
>> # 1 2 <class '__main__.B'>
>>
>>
>> this works so far, however if I now put a further level of Liftable
>> (which in principal already works with the generic definition
>>
>> class C(Liftable, B):
>>     def __initialize__(self, c):
>>         self.c = c
>>
>>
>> I get the error
>>
>> TypeError: Error when calling the metaclass bases Cannot create a
>> consistent method resolution order (MRO) for bases Liftable, B
>>
>>
>> ​I read about MRO, and it seems to be the case that this setting somehow
>> raises this generic Error, however I really think having such a Lifting is
>> save and extremely useful​ - how can I make it work in python?
>>
>> (one further comment: switching the order of inheritance, i.e. class B(A,
>> Liftable) will call A.__init__ before Liftable.__init__ which makes the
>> whole idea senseless)
>>
>> Any constructive help is appreciated!
>> best,
>> Stephan
>>
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20151211/beec7c28/attachment-0001.html>