Mailman 3 December 2015 - Python-ideas

generic Liftable abc-mixin breaks at MRO
by Stephan Sahm Dec. 11, 2015

Dec. 11, 2015

Dear all, I think I found a crucial usecase where the standard MRO does not work out. I would appreciate your help to still solve this usecase, or might MRO even be adapted? The idea is to build a generic Lift-type which I call this way because the derived classes should be able to easily lift from subclasses. So for example if I have an instance *a* from *class A* and a *class B(A)* I want to make *a* an instance of *B* in a straightforward way. My implementation (Python 2.7): import abc … [View More]import inspect def use_as_needed(func, kwargs): meta = inspect.getargspec(func) if meta.keywords is not None: return meta(**kwargs) else: # not generic super-constructor - pick only the relevant subentries: return func(**{k:kwargs[k] for k in kwargs if k in meta.args}) class Liftable(object): __metaclass__ = abc.ABCMeta def __init__(self, **kwargs): use_as_needed(super(Liftable,self).__init__, kwargs) use_as_needed(self.__initialize__, kwargs) @abc.abstractmethod def __initialize__(self, **kwargs): return NotImplemented() class NoMatchingAncestor(RuntimeError): pass class NotLiftable(RuntimeError): pass def lift(self, new_class, **kwargs): # Stop Conditions: if self.__class__ is new_class: return # nothing to do elif new_class is object: # Base Case # break recursion at once: raise NoMatchingAncestor() elif new_class.__base__ is not Liftable: #to ensure this is save raise NotLiftable("Class {} is not Liftable (must be first parent)".format(new_class.__name__)) # recursive case: if not self.__class__ is new_class.__bases__[1]: lift(self, new_class.__bases__[1], **kwargs) # own case: self.__class__ = new_class use_as_needed(self.__initialize__, kwargs) and the example usecase: class A(object): def __init__(self, a): self.a = a class B(Liftable, A): def __initialize__(self, b): self.b = b a = A(1) print a.a, a.__class__ # 1 <class '__main__.A'> lift(a, B, b=2) print a.a, a.b, a.__class__ # 1 2 <class '__main__.B'> this works so far, however if I now put a further level of Liftable (which in principal already works with the generic definition class C(Liftable, B): def __initialize__(self, c): self.c = c I get the error TypeError: Error when calling the metaclass bases Cannot create a consistent method resolution order (MRO) for bases Liftable, B I read about MRO, and it seems to be the case that this setting somehow raises this generic Error, however I really think having such a Lifting is save and extremely useful - how can I make it work in python? (one further comment: switching the order of inheritance, i.e. class B(A, Liftable) will call A.__init__ before Liftable.__init__ which makes the whole idea senseless) Any constructive help is appreciated! best, Stephan [View Less]

4 9

MRO local precedence ordering revisited
by Stephan Sahm Dec. 11, 2015

Dec. 11, 2015

Dear all, I just pushed a very specific question which as I now recognized is far more general: It concerns python MRO (method resolution order). As reference I take https://www.python.org/download/releases/2.3/mro/ There the section "Bad Method Resolution Orders" starts with an example which behaviour in the current python 2.7 is rather unintuitive to me. I rename the example slightly to illustrate its usecase. class Mixin(object): > pass class A(Mixin): > pass > class B(… [View More]

8 13

ExitStack: Allow exiting individual context managers
by Ram Rachum Dec. 9, 2015

Dec. 9, 2015

Hi guys, I'm using `contextlib.ExitStack` today, and pushing context managers into it. I find myself wanting to exit specific context managers that I've pushed into it, while still inside the `with` suite of the `ExitStack`. In other words, I want to exit one of the context managers but still keep the `ExitStack`, and all other context managers, acquired. This isn't currently possible, right? What do you think about implementing this? Thanks, Ram.

4 7

Using functools.lru_cache only on some arguments of a function
by Bill Winslow Dec. 7, 2015

Dec. 7, 2015

This is a question I posed to reddit, with no real resolution: https://www.reddit.com/r/learnpython/comments/3v75g4/using_functoolslru_cac… The summary for people here is the following: Here's a pattern I'm using for my code: def deterministic_recursive_calculation(input, partial_state=None): condition = do_some_calculations(input) if condition: return deterministic_recursive_calculation(reduced_input, some_state) Basically, in calculating the results of the subproblem, the … [View More]subproblem can be calculated quicker by including/sharing some partial results from the superproblem. (Calling the subproblem without the partial state still gives the same result, but takes substantially longer.) I want to memoize this function for obvious reasons, but I need the lru_cache to ignore the partial_state argument, for its value does not affect the output, only the computation expense. Is there any reasonable way to do this? Things such as memoizing a wrapper don't actually solve the problem. About the only way I can think of with current technology is either to have a hidden singleton class which maintains state, or a hidden global variable, which amount to the same thing of storing the partial state outside the function. But those strike me as unwieldy and unpythonic. What I'd really like to do is to have some way to tell functools.lru_cache to ignore some arguments of a function it's memoizing for the purposes of caching. One way would be to add an "arg_filter" argument, which for purposes of this example would be used like so: @lru_cache(arg_filter=lambda args, kwargs: args[:1], {}) def deterministic_recursive_calculation(input, partial_state=None): condition = do_some_calculations(input) if condition: return deterministic_recursive_calculation(reduced_input, some_state) This effectively ignores all arguments except the first positional one for the purposes of caching. Such an option could be implemented as in the diff below (provided only for discussion purposes). So what I want to know is: 1) Am I sane? Is there no particularly good way currently to go about caching functions following the given pattern? 2) Assuming the answer to 1) is "Yes I am sane, and there's no good way currently", is my proposal a reasonable one? Mostly on philosophical grounds, not necessarily specifics of how it works. Thank you for your time and consideration. Bill ---------------------------------------------------------------------------------------------------------------------------------- https://hg.python.org/cpython/file/3.5/Lib/functools.py diff functools.py functools.py.orig 363c363 < def _make_key(args, kwds, typed, arg_filter, --- > def _make_key(args, kwds, typed, 377,378d376 < if arg_filter is not None: < args, kwds = arg_filter(args, kwds) 393c391 < def lru_cache(maxsize=128, typed=False, arg_filter=None): --- > def lru_cache(maxsize=128, typed=False): 403,406d400 < *arg_filter* is an optional function which filters user-speicified portions < of the arguments from the caching key (e.g. if an argument affects the < computation but not the final result). < 428,430d421 < if arg_filter is not None and not callable(arg_filter): < raise TypeError('Expected arg_filter to be a callable') < 432,433c423 < wrapper = _lru_cache_wrapper(user_function, maxsize, typed, arg_filter, < _CacheInfo) --- > wrapper = _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo) 438c428 < def _lru_cache_wrapper(user_function, maxsize, typed, arg_filter, _CacheInfo): --- > def _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo): 466c456 < key = make_key(args, kwds, typed, arg_filter) --- > key = make_key(args, kwds, typed) 481c471 < key = make_key(args, kwds, typed, arg_filter) --- > key = make_key(args, kwds, typed) [View Less]

6 9

Multiple arguments for decorators
by Emanuel Barry Dec. 3, 2015

Dec. 3, 2015

An idea that I had a couple of times was to have a syntactic way to define multiple arguments for a @decorator. This would be good if you want to, for example, define a property with a getter and a setter. I had a few ideas, but none of them seem to be "the obvious way to do it". I'm including them anyway, but I'd rather see if people are interested in the idea or if it's just something that's not worth it. I'll gladly write a PEP if it gains a lot of interest. Method 1: Add parens around all … [View More]

18 39

Re: [Python-ideas] [Python-ideos] Dedicated overloadable boolean operators
by Henry Schreiner Dec. 1, 2015

Dec. 1, 2015

Sorry, the order was incorrect in that last message. It should be: When x and y is written, if x has __booland__, then x.__booland__(y) is returned. If not, then return x if x==False else if x==True, then if y has y.__rbooland__(x), that is returned. If not, then return y. On Monday, November 30, 2015 at 10:11:03 AM UTC-6, Henry Schreiner wrote: > > It seems to me there are two ideas being discussed. One is adding 2 new > operators, something like && and ||, and providing … [View More]them for some sort of > "element wise" or other boolean context, and giving them a better order of > evaluations than bitwise ops. This would have the advantage of being usable > on the built in python lists/tuples if needed, but otherwise doesn't seem > to solve all the issues above, including expressions like -1 < mat < 1, > which work for python variables but not for things like Numpy arrays. (for > example, selecting values from an array works like this: mat[(-1<mat) & > (mat<1)] but not like this: mat[-1 < mat < 1] or this: mat[-1<mat & > mat<1] ). > > The common "general purpose infix notation" pops up here, of course, but > I'd consider it a variant of the first idea. > > The other idea is that overloads could be added to the existing and and or > operators. This, I believe, would solve all the main problems, and would > not add more to the language for a newcomer to learn. (In fact, most people > could forget about the current bitwise and and or, since that's not what > most people need, at least those using databases and Numpy). The downside > is that the current and and or operators are short-circuiting, and adding > a way to overload a short-circuiting operator is tricky/ugly/expensive > (though the last one might not be that bad anymore). But, as I think Andrew > was mentioning, what about a mixed solution? One that provides a > non-short-circuiting overload, but the original short-circuiting behavior > is default? It would look like this: > > Two new magic methods would be added, for now called __booland__ and > __boolor__, along with __rbooland__ and __rboolor__. > > > > Evaluation would work like this, using and for an example: > When x and y is written, if x has __booland__, then y.__booland__(y) is > called. If it does not, then if y exists and has y.__rbooland__(x), that > is called. If neither, then x and y is performed in a normal, > short circuiting manner, calling bool(x) then calling bool(y) only if x > == True. (The second line here might be the hard part due to the way > short circuiting works) > > > > This would allow object that want custom and/or behavior to implement > these magic functions. They would lose (explicit) short-circuiting, but > they would gain custom behavior. Most of these special objects do not need > or expect the short circuit behavior (Only if an array was all True or > False would it even make sense in numpy , and this would allow > expressions like -1 < mat < 1 to work as expected in Python. This would > be easy to teach to newcomers, too, since only objects that have a definite > True/False should short circuit, others have the special methods. The > old behavior could be forced on overriding objects by wrapping them in > bool(). > > > Would something like that be possible and better than PEP 335? > > PS: > Either type of solution would also be useful for the Plumbum library, by > the way, as it would be useful to add and-ing and or-ing to pipelines, but > the bitwise operators have a different meaning for Plumbum. Due to the lazy > evaluation of Plumbum, even the above solution (or any solution) would > still allow short-circuiting in the pipeline. > > > [View Less]

3 4