Mailman 3 October 2019 - Python-ideas

Ability to customize how Enum interprets values with access to names
by Steve Jorgensen Oct. 28, 2019

Oct. 28, 2019

Here's a working example of what I'm talking about. In this example, `EnumX` is a tweak of `Enum` to that has the capability that I'm talking about, and I have also pasted the code for `EnumX` later in this email. Please note that the important thing in this example is NOT what this `ChoiceEnum` class does but the ability to create custom enums that can do custom processing of assigned value/key such as what `ChoiceEnum` does, using a documented feature of `Enum`, without having to either … [View More]write and debug a complicated hacks (that might not work with future updates to `Enum`) or replace hunks of `Enum` (as my `EnumX` shown here does) in ways that might not work or might not be feature-complete with future `Enum` updates. # Example usage of enhanced Enum with _preprocess_value_ support. def _label_from_key(key): return key.replace('_', ' ').title() class _ChoiceValue(str): """Easy way to have an object with immutable string content that is identified for equality by its content with a label attribute that is not significant for equality. """ def __new__(cls, content, label): obj = str.__new__(cls, content) obj.label = label return obj def __repr__(self): content_repr = str.__repr__(self) return '%s(%r, %r)' % ( self.__class__.__name__, content_repr, self.label) def get_value_label_pair(self): return (f'{self}', self.label) class ChoiceEnum(EnumX): def _preprocess_value_(key, src): value = None label = None if src == (): pass elif isinstance(src, tuple) and src != (): value, label = src elif isinstance(str, str) and src.startswith(','): label = src[1:] else: value = src if value is None: value = key label = label or _label_from_key(key) return _ChoiceValue(value, label) def __getitem__(self, key): return (f'{self._value_}', self._value_.label).__getitem__(key) def __len__(self): return 2 @property def value(self): return self._value_.get_value_label_pair() @property def label(self): return self._value_.label class Food(ChoiceEnum): APPLE = () CHEESE = () HAMBURGER = 'BURGER' SOUFFLE = ',Soufflé' CHICKEN_MCNUGGETS = ('CHX_MCNUG', 'Chicken McNuggets') DEFAULT = 'APPLE' for food in Food: print(repr(food)) # Prints... # <Food.APPLE: _ChoiceValue("'APPLE'", 'Apple')> # <Food.CHEESE: _ChoiceValue("'CHEESE'", 'Cheese')> # <Food.HAMBURGER: _ChoiceValue("'BURGER'", 'Hamburger')> # <Food.SOUFFLE: _ChoiceValue("',Soufflé'", 'Souffle')> # <Food.CHICKEN_MCNUGGETS: _ChoiceValue("'CHX_MCNUG'", 'Chicken McNuggets')> print(f'Food.DEFAULT is Food.APPLE: {Food.DEFAULT is Food.APPLE}') # Prints... # Food.DEFAULT is Food.APPLE: True Here's the implementation of `EnumX` that is being used for the above. There are just a handful of those lines that represent changes to the implementation at https://github.com/python/cpython/blob/3.8/Lib/enum.py and I there are inline comments to draw attention to those. from enum import ( Enum, EnumMeta, auto, _is_sunder, _is_dunder, _is_descriptor, _auto_null) # Copy of EnumDict with tweaks to support _preprocess_value_ class _EnumDictX(dict): def __init__(self): super().__init__() self._member_names = [] self._last_values = [] self._ignore = [] def __setitem__(self, key, value): """Duplicate all of _EnumDict.__setitem__ in order to insert a hook """ if _is_sunder(key): if key not in ( '_order_', '_create_pseudo_member_', '_generate_next_value_', '_missing_', '_ignore_', '_preprocess_value_', # <-- ): raise ValueError('_names_ are reserved for future Enum use') if key == '_generate_next_value_': setattr(self, '_generate_next_value', value) # ==================== if key == '_preprocess_value_': setattr(self, '_preprocess_value', value) # ==================== elif key == '_ignore_': if isinstance(value, str): value = value.replace(',', ' ').split() else: value = list(value) self._ignore = value already = set(value) & set(self._member_names) if already: raise ValueError( '_ignore_ cannot specify already set names: %r' % ( already, )) elif _is_dunder(key): if key == '__order__': key = '_order_' elif key in self._member_names: # descriptor overwriting an enum? raise TypeError('Attempted to reuse key: %r' % key) elif key in self._ignore: pass elif not _is_descriptor(value): if key in self: # enum overwriting a descriptor? raise TypeError('%r already defined as: %r' % (key, self[key])) # ==================== value = self._preprocess_value(key, value) # ==================== if isinstance(value, auto): if value.value == _auto_null: value.value = self._generate_next_value( key, 1, len(self._member_names), self._last_values[:]) value = value.value self._member_names.append(key) self._last_values.append(value) dict.__setitem__(self, key, value) # Subclass of EnumMeta with tweak to support _preprocess_value_ class EnumMetaX(EnumMeta): # Copy of EnumMeta.__prepare__ with tweak to support _preprocess_value_ @classmethod def __prepare__(metacls, cls, bases): # create the namespace dict enum_dict = _EnumDictX() # inherit previous flags and _generate_next_value_ function member_type, first_enum = metacls._get_mixins_(bases) if first_enum is not None: # ==================== enum_dict['_preprocess_value_'] = getattr( first_enum, '_preprocess_value_', None) # ==================== enum_dict['_generate_next_value_'] = getattr(first_enum, '_generate_next_value_', None) return enum_dict # Subclass of Enum using EnumMetaX as metaclass and with default # implementation of _preprocess_value_. class EnumX(Enum, metaclass=EnumMetaX): def _preprocess_value_(key, value): return value [View Less]

2 2

PEP 584: Add + and += operators to the built-in dict class.
by Brandt Bucher Oct. 25, 2019

Oct. 25, 2019

At long last, Steven D'Aprano and I have pushed a second draft of PEP 584 (dictionary addition): https://www.python.org/dev/peps/pep-0584/ The accompanying reference implementation is on GitHub: https://github.com/brandtbucher/cpython/tree/addiction This new draft incorporates much of the feedback that we received during the first round of debate here on python-ideas. Most notably, the difference operators (-/-=) have been dropped from the proposal, and the implementations have been updated … [View More]

29 127

Start argument for itertools.accumulate() [Was: Proposal: A Reduce-Map Comprehension and a "last" builtin]
by Raymond Hettinger Oct. 23, 2019

Oct. 23, 2019

> On Friday, April 6, 2018 at 8:14:30 AM UTC-7, Guido van Rossum wrote: > On Fri, Apr 6, 2018 at 7:47 AM, Peter O'Connor <peter.ed...(a)gmail.com> wrote: >> So some more humble proposals would be: >> >> 1) An initializer to itertools.accumulate >> functools.reduce already has an initializer, I can't see any controversy to adding an initializer to itertools.accumulate > > See if that's accepted in the bug tracker. It did come-up once but was closed … [View More]for a number reasons including lack of use cases. However, Peter's signal processing example does sound interesting, so we could re-open the discussion. For those who want to think through the pluses and minuses, I've put together a Q&A as food for thought (see below). Everybody's design instincts are different -- I'm curious what you all think think about the proposal. Raymond --------------------------------------------- Q. Can it be done? A. Yes, it wouldn't be hard. _sentinel = object() def accumulate(iterable, func=operator.add, start=_sentinel): it = iter(iterable) if start is _sentinel: try: total = next(it) except StopIteration: return else: total = start yield total for element in it: total = func(total, element) yield total Q. Do other languages do it? A. Numpy, no. R, no. APL, no. Mathematica, no. Haskell, yes. * http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.accumulate.… * https://stat.ethz.ch/R-manual/R-devel/library/base/html/cumsum.html * http://microapl.com/apl/apl_concepts_chapter5.html \+ 1 2 3 4 5 1 3 6 10 15 * https://reference.wolfram.com/language/ref/Accumulate.html * https://www.haskell.org/hoogle/?hoogle=mapAccumL Q. How much work for a person to do it currently? A. Almost zero effort to write a simple helper function: myaccum = lambda it, func, start: accumulate(chain([start], it), func) Q. How common is the need? A. Rare. Q. Which would be better, a simple for-loop or a customized itertool? A. The itertool is shorter but more opaque (especially with respect to the argument order for the function call): result = [start] for x in iterable: y = func(result[-1], x) result.append(y) versus: result = list(accumulate(iterable, func, start=start)) Q. How readable is the proposed code? A. Look at the following code and ask yourself what it does: accumulate(range(4, 6), operator.mul, start=6) Now test your understanding: How many values are emitted? What is the first value emitted? Are the two sixes related? What is this code trying to accomplish? Q. Are there potential surprises or oddities? A. Is it readily apparent which of assertions will succeed? a1 = sum(range(10)) a2 = sum(range(10), 0) assert a1 == a2 a3 = functools.reduce(operator.add, range(10)) a4 = functools.reduce(operator.add, range(10), 0) assert a3 == a4 a4 = list(accumulate(range(10), operator.add)) a5 = list(accumulate(range(10), operator.add, start=0)) assert a5 == a6 Q. What did the Python 3.0 Whatsnew document have to say about reduce()? A. "Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable." Q. What would this look like in real code? A. We have almost no real-world examples, but here is one from a StackExchange post: def wsieve(): # wheel-sieve, by Will Ness. ideone.com/mqO25A->0hIE89 wh11 = [ 2,4,2,4,6,2,6,4,2,4,6,6, 2,6,4,2,6,4,6,8,4,2,4,2, 4,8,6,4,6,2,4,6,2,6,6,4, 2,4,6,2,6,4,2,4,2,10,2,10] cs = accumulate(cycle(wh11), start=11) yield( next( cs)) # cf. ideone.com/WFv4f ps = wsieve() # codereview.stackexchange.com/q/92365/9064 p = next(ps) # 11 psq = p*p # 121 D = dict( zip( accumulate(wh11, start=0), count(0))) # start from sieve = {} for c in cs: if c in sieve: wheel = sieve.pop(c) for m in wheel: if not m in sieve: break sieve[m] = wheel # sieve[143] = wheel@187 elif c < psq: yield c else: # (c==psq) # map (p*) (roll wh from p) = roll (wh*p) from (p*p) x = [p*d for d in wh11] i = D[ (p-11) % 210] wheel = accumulate(cycle(x[i:] + x[:i]), start=psq) p = next(ps) ; psq = p*p next(wheel) ; m = next(wheel) sieve[m] = wheel [View Less]

17 49

Re: [Python-Dev] 4 first-time contributions that need core review.
by Dong-hee Na Oct. 23, 2019

Oct. 23, 2019

> https://github.com/python/cpython/pull/16680: bpo-38419: fix "check-c-globals" path This PR was merged by Carol today :) 2019년 10월 23일 (수) 오전 3:53, Brandt Bucher <brandtbucher(a)gmail.com>님이 작성: > > Hi all. There are a few *very simple* PRs from first-time contributors that have been sitting un-core-reviewed for several weeks. > > https://github.com/python/cpython/pull/16680: bpo-38419: fix "check-c-globals" path > https://github.com/python/cpython/pull/16678: bpo-… [View More]

1 1

RFC: PEP xxx: Python Compatibility Version
by Victor Stinner Oct. 22, 2019

Oct. 22, 2019

Hi, I propose the following PEP to add sys.set_python_compat_version(version) to introduce a partial compatibility with Python 3.8 in the next Python 3.9 version. I also propose this PEP in a pull request: https://github.com/python/peps/pull/1209 (Please avoid using the temporary PEP number until this PR is merged: officially, the PEP has no number yet.) If possible, please try to read the whole PEP before replying. I would prefer to avoid knee-jerk reactions :-) The backward compatibility … [View More]

8 9

Extending @ syntax to allow expressions
by Yonatan Zunger Oct. 22, 2019

Oct. 22, 2019

Hey everyone, I came across a case which *might* be a use case for a syntax extension, but I'm not sure. Wanted to get feedback from the group. *The extension: *Extend the decorator syntax from decorator ::= "@" dotted_name ["(" [argument_list [","]] ")"] NEWLINE to decorator ::= "@" expression NEWLINE where the expression must return a callable that accepts a function of the indicated signature as its argument. *The motivating case: *I'm using function decorators to define cron jobs in … [View More]my system, with the rather nice syntax "@cronJob(parameters) def myFunction(standard args)". The decorator registers the function, this causes all sorts of magic to occur, and out the other end of the pipe jobs run in production. It's working very nicely. However, we'd like to improve the syntax to allow different cron configurations for the job in different production environments. Since the default-plus-overrides model is natural for the actual use cases, a nice way to do this would be to replace the cronJob function with a class whose __init__ and __call__ methods respectively load up the parameters and do the actual work of registration, but which also has an override() method that lets you specify per-environment overrides and then returns self. This would allow a syntax like: @CronJob('job-name', params...).override('dev', more-params...) def myFunction(...) However, this is invalid syntax in Python 3.8 because you can't have a general expression in a decorator statement. Instead, the nearest option is myJob = CronJob('job-name', params...).override('dev', more-params...) @myJob def myFunction(...) Which is uglier for no obvious benefit. *Possible pros and cons: * This extends the existing syntax in a way that's very intuitive w.r.t. the current one -- the docs say "A function definition may be wrapped by one or more decorator expressions. Decorator expressions are evaluated when the function is defined, in the scope that contains the function definition. The result must be a callable, which is invoked with the function object as the only argument. The returned value is bound to the function name instead of the function object. Multiple decorators are applied in nested fashion." It's not obvious from the syntax why the decorator expression *needs* to have this limited form. It increases consistency, by eliminating this one unusual use of dotted expressions as special relative to other expressions. (NB this is the only call to ast_for_dotted_expr in ast.c!) The meaning of the operation remains unambiguous, and is just as accessible to tools like linters, type checkers, and syntax highlighters, as the @ operator simply modifies the succeeding expression. On the downside, it's more flexible, and so offers more chances for a user to shoot themselves in the foot. It's somewhat of a corner case, so it's not obvious that the syntax extension is worth it. What do people think? [View Less]

7 10

Copying a multiple values of a dict to another with a lot easier syntax
by gedizgursu＠gmail.com Oct. 22, 2019

Oct. 22, 2019

m and n are lists or dicts or enumerates or classes or anything it can be assigned like following: instead of : m.a=n.a; m.b=n.b; m.c=n.c; ... I suggest: a,b,c of m to n ; I thought of this while writing something in javascript since I love pyhton above everything I can type (even more than native language sentences) I wanted to post this idea to here It is my first idea while coding ... In javascript, I was assinging a boundingBoxRect values to a div it was going like this: adiv=… [View More]

7 7

Copying a multiple values of a dict to another with a lot easier syntax
by Gediz GÜRSU Oct. 21, 2019

Oct. 21, 2019

m and n are lists or dicts or enumerates or classes or anything it can be assigned like following: instead of : m.a=n.a; m.b=n.b; m.c=n.c; ... I suggest a,b,c of m to n ; I thought of this while writing something in javascript since I love pyhton above everything I can type (even more than native language sentences) I wanted to post this idea to here It is my first idea while coding ... In javascript, I was assinging a boundingBoxRect values to a div it was going like this: adiv=document.… [View More]

1 0

Expose Py_TabcheckFlag as other PyAPI_DATA flag.
by GAUDRY Eloi Oct. 21, 2019

Oct. 21, 2019

Hi When looking at the pydebug.h header file, I see that some symbols are not declared as exported. For instance, I would like to be able to change the value of the Py_TabcheckFlag when running specific codelines. Any chance this patch could be added ? Thanks for your feedback, Eloi Index: Include/pydebug.h =================================================================== --- Include/pydebug.h (revision 118072) +++ Include/pydebug.h (working copy) @@ -16,6 +16,7 @@ PyAPI_DATA(int) … [View More]

4 5

Expose _PyGC_generation0 for allowing internal use directly from a CPython extension
by GAUDRY Eloi Oct. 21, 2019

Oct. 21, 2019

Hi I would like to be able to get an handle on PyGC_Head*_PyGC_generation0 from a CPython extension. This is possible when building Python on a Posix host, but not on Windows because of a missing PyAPI_DATA wrapping of the said object in the objimpl.h header. Having the possibility to inspect the PyGC_Head here can be very useful when customizing the way a garbage-collectable object created from CPython extension would be cleaned/inspected, etc. Any chance this patch could be added to get … [View More]

4 4