Mailman 3 April 2018 - Python-ideas

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow Feb. 18, 2022

Feb. 18, 2022

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, … [View More]

17 32

Dataclasses, keyword args, and inheritance
by George Leslie-Waksman March 12, 2021

March 12, 2021

The proposed implementation of dataclasses prevents defining fields with defaults before fields without defaults. This can create limitations on logical grouping of fields and on inheritance. Take, for example, the case: @dataclass class Foo: some_default: dict = field(default_factory=dict) @dataclass class Bar(Foo): other_field: int this results in the error: 5 @dataclass ----> 6 class Bar(Foo): 7 other_field: int 8 ~/.pyenv/versions/3.6.2/envs/… [View More]clover_pipeline/lib/python3.6/site-packages/dataclasses.py in dataclass(_cls, init, repr, eq, order, hash, frozen) 751 752 # We're called as @dataclass, with a class. --> 753 return wrap(_cls) 754 755 ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in wrap(cls) 743 744 def wrap(cls): --> 745 return _process_class(cls, repr, eq, order, hash, init, frozen) 746 747 # See if we're being called as @dataclass or @dataclass(). ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _process_class(cls, repr, eq, order, hash, init, frozen) 675 # in __init__. Use "self" if possible. 676 '__dataclass_self__' if 'self' in fields --> 677 else 'self', 678 )) 679 if repr: ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _init_fn(fields, frozen, has_post_init, self_name) 422 seen_default = True 423 elif seen_default: --> 424 raise TypeError(f'non-default argument {f.name!r} ' 425 'follows default argument') 426 TypeError: non-default argument 'other_field' follows default argument I understand that this is a limitation of positional arguments because the effective __init__ signature is: def __init__(self, some_default: dict = <something>, other_field: int): However, keyword only arguments allow an entirely reasonable solution to this problem: def __init__(self, *, some_default: dict = <something>, other_field: int): And have the added benefit of making the fields in the __init__ call entirely explicit. So, I propose the addition of a keyword_only flag to the @dataclass decorator that renders the __init__ method using keyword only arguments: @dataclass(keyword_only=True) class Bar(Foo): other_field: int --George Leslie-Waksman [View Less]

12 35

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Re: [Python-ideas] Allow star unpacking within an slice expression
by Neil Girdhar March 7, 2020

March 7, 2020

I didn't think of this when we were discussing 448. I ran into this today, so I agree with you that it would be nice to have this. Best, Neil On Monday, December 4, 2017 at 1:02:09 AM UTC-5, Eric Wieser wrote: > > Hi, > > I've been thinking about the * unpacking operator while writing some numpy > code. PEP 448 allows the following: > > values = 1, *some_tuple, 2 > object[(1, *some_tuple, 2)] > > It seems reasonable to me that it should be extended to … [View More]

2 2

Specify number of items to allocate for array.array() constructor
by Sven Rahmann Feb. 22, 2020

Feb. 22, 2020

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int … [View More]

14 20

Please consider adding of functions file system operations to pathlib
by George Fischhof Nov. 16, 2019

Nov. 16, 2019

Good day all, as a continuation of thread "OS related file operations (copy, move, delete, rename...) should be placed into one module" https://mail.python.org/pipermail/python-ideas/2017-January/044217.html please consider making pathlib to a central file system module with putting file operations (copy, move, delete, rmtree etc) into pathlib. BR, George

3 3

Start argument for itertools.accumulate() [Was: Proposal: A Reduce-Map Comprehension and a "last" builtin]
by Raymond Hettinger Oct. 23, 2019

Oct. 23, 2019

> On Friday, April 6, 2018 at 8:14:30 AM UTC-7, Guido van Rossum wrote: > On Fri, Apr 6, 2018 at 7:47 AM, Peter O'Connor <peter.ed...(a)gmail.com> wrote: >> So some more humble proposals would be: >> >> 1) An initializer to itertools.accumulate >> functools.reduce already has an initializer, I can't see any controversy to adding an initializer to itertools.accumulate > > See if that's accepted in the bug tracker. It did come-up once but was closed … [View More]for a number reasons including lack of use cases. However, Peter's signal processing example does sound interesting, so we could re-open the discussion. For those who want to think through the pluses and minuses, I've put together a Q&A as food for thought (see below). Everybody's design instincts are different -- I'm curious what you all think think about the proposal. Raymond --------------------------------------------- Q. Can it be done? A. Yes, it wouldn't be hard. _sentinel = object() def accumulate(iterable, func=operator.add, start=_sentinel): it = iter(iterable) if start is _sentinel: try: total = next(it) except StopIteration: return else: total = start yield total for element in it: total = func(total, element) yield total Q. Do other languages do it? A. Numpy, no. R, no. APL, no. Mathematica, no. Haskell, yes. * http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.accumulate.… * https://stat.ethz.ch/R-manual/R-devel/library/base/html/cumsum.html * http://microapl.com/apl/apl_concepts_chapter5.html \+ 1 2 3 4 5 1 3 6 10 15 * https://reference.wolfram.com/language/ref/Accumulate.html * https://www.haskell.org/hoogle/?hoogle=mapAccumL Q. How much work for a person to do it currently? A. Almost zero effort to write a simple helper function: myaccum = lambda it, func, start: accumulate(chain([start], it), func) Q. How common is the need? A. Rare. Q. Which would be better, a simple for-loop or a customized itertool? A. The itertool is shorter but more opaque (especially with respect to the argument order for the function call): result = [start] for x in iterable: y = func(result[-1], x) result.append(y) versus: result = list(accumulate(iterable, func, start=start)) Q. How readable is the proposed code? A. Look at the following code and ask yourself what it does: accumulate(range(4, 6), operator.mul, start=6) Now test your understanding: How many values are emitted? What is the first value emitted? Are the two sixes related? What is this code trying to accomplish? Q. Are there potential surprises or oddities? A. Is it readily apparent which of assertions will succeed? a1 = sum(range(10)) a2 = sum(range(10), 0) assert a1 == a2 a3 = functools.reduce(operator.add, range(10)) a4 = functools.reduce(operator.add, range(10), 0) assert a3 == a4 a4 = list(accumulate(range(10), operator.add)) a5 = list(accumulate(range(10), operator.add, start=0)) assert a5 == a6 Q. What did the Python 3.0 Whatsnew document have to say about reduce()? A. "Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable." Q. What would this look like in real code? A. We have almost no real-world examples, but here is one from a StackExchange post: def wsieve(): # wheel-sieve, by Will Ness. ideone.com/mqO25A->0hIE89 wh11 = [ 2,4,2,4,6,2,6,4,2,4,6,6, 2,6,4,2,6,4,6,8,4,2,4,2, 4,8,6,4,6,2,4,6,2,6,6,4, 2,4,6,2,6,4,2,4,2,10,2,10] cs = accumulate(cycle(wh11), start=11) yield( next( cs)) # cf. ideone.com/WFv4f ps = wsieve() # codereview.stackexchange.com/q/92365/9064 p = next(ps) # 11 psq = p*p # 121 D = dict( zip( accumulate(wh11, start=0), count(0))) # start from sieve = {} for c in cs: if c in sieve: wheel = sieve.pop(c) for m in wheel: if not m in sieve: break sieve[m] = wheel # sieve[143] = wheel@187 elif c < psq: yield c else: # (c==psq) # map (p*) (roll wh from p) = roll (wh*p) from (p*p) x = [p*d for d in wh11] i = D[ (p-11) % 210] wheel = accumulate(cycle(x[i:] + x[:i]), start=psq) p = next(ps) ; psq = p*p next(wheel) ; m = next(wheel) sieve[m] = wheel [View Less]

17 49

Dart like multi line strings identation
by Marius Räsener Feb. 8, 2019

Feb. 8, 2019

Hey List, this is my very first approach to suggest a Python improvement I'd think worth discussing. At some point, maybe with Dart 2.0 or a little earlier, Dart is now supporting multiline strings with "proper" identation (tried, but I can't find the according docs at the moment. probably due to the rather large changes related to dart 2.0 and outdated docs.) What I have in mind is probably best described with an Example: print(""" I am a multiline String. """) the closing … [View More]

23 47

Pypi private repo's
by Nick Humrich Oct. 29, 2018

Oct. 29, 2018

I am sure this has been discussed before, and this might not even be the best place for this discussion, but I just wanted to make sure this has been thought about. What if pypi.org supported private repos at a cost, similar to npm? This would be able to help support the cost of pypi, and hopefully make it better/more reliable, thus in turn improving the python community. If this discussion should happen somewhere else, let me know. Nick

5 5

Re: [Python-ideas] [Distutils] Pypi private repo's
by Nick Coghlan Oct. 28, 2018

Oct. 28, 2018

On 5 April 2018 at 07:58, Jannis Gebauer <ja.geb(a)me.com> wrote: > What if there was some kind of “blessed” entity that runs these services and puts the majority of the revenue into a fund that funds development on PyPi (maybe trough the PSF)? Having a wholly owned for-profit subsidiary that provides commercial services as a revenue raising mechanism is certainly one way to approach something like this without alienating sponsors or tax authorities (although it may still alienate the … [View More]vendors of now competing services). It would require a big time commitment on the PSF side to get everything set up though, as well as interest from key folks in joining what would essentially be a single-language-focused start up in an already crowded cross-language developer tools marketplace. When the PSF as a whole is still operating with only a handful of full or part time employees, it's far from clear that setting something like that up would be the most effective possible use of their time and energy. At a more basic level, that kind of arrangement technically doesn't require anyone's blessing, it could be as straightforward as downstream tooling vendors signing up as PSF sponsors and saying "please allocate our sponsorship contribution to the Packaging WG's budget so that PyPI keeps operating well and the PyPA tooling keeps improving, increasing the level of demand for our commercial Python repository management services". Historically that wouldn't have helped much, since the PSF itself has struggled with effective project management (for a variety of reasons), but one of the things I think the success of the MOSS grant has shown is the significant strides that the PSF has made in budget management in recent years, such that if funding is made available, it can and will be spent effectively. Cheers, Nick. P.S. PyPA contributors are also free agents in their own right, so folks offering Python-centric developer workflow management tools or features may decide that it's worth their while to invest more directly in smoothing out some of the rough edges that currently still exist. It's a mercenary way of looking at things, but in many cases, it is *absolutely* possible to pay for the time and attention of existing contributors, and if you can persuade them that your proposals are reasonable, they'll often have an easier time than most convincing other community contributors that it's a good way to go :) -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia [View Less]

3 5

Asynchronous exception handling around with/try statement borders
by Erik Bray Sept. 24, 2018

Sept. 24, 2018

Hi folks, I normally wouldn't bring something like this up here, except I think that there is possibility of something to be done--a language documentation clarification if nothing else, though possibly an actual code change as well. I've been having an argument with a colleague over the last couple days over the proper way order of statements when setting up a try/finally to perform cleanup of some action. On some level we're both being stubborn I think, and I'm not looking for resolution … [View More]as to who's right/wrong or I wouldn't bring it to this list in the first place. The original argument was over setting and later restoring os.environ, but we ended up arguing over threading.Lock.acquire/release which I think is a more interesting example of the problem, and he did raise a good point that I do want to bring up. </prologue> My colleague's contention is that given lock = threading.Lock() this is simply *wrong*: lock.acquire() try: do_something() finally: lock.release() whereas this is okay: with lock: do_something() Ignoring other details of how threading.Lock is actually implemented, assuming that Lock.__enter__ calls acquire() and Lock.__exit__ calls release() then as far as I've known ever since Python 2.5 first came out these two examples are semantically *equivalent*, and I can't find any way of reading PEP 343 or the Python language reference that would suggest otherwise. However, there *is* a difference, and has to do with how signals are handled, particularly w.r.t. context managers implemented in C (hence we are talking CPython specifically): If Lock.__enter__ is a pure Python method (even if it maybe calls some C methods), and a SIGINT is handled during execution of that method, then in almost all cases a KeyboardInterrupt exception will be raised from within Lock.__enter__--this means the suite under the with: statement is never evaluated, and Lock.__exit__ is never called. You can be fairly sure the KeyboardInterrupt will be raised from somewhere within a pure Python Lock.__enter__ because there will usually be at least one remaining opcode to be evaluated, such as RETURN_VALUE. Because of how delayed execution of signal handlers is implemented in the pyeval main loop, this means the signal handler for SIGINT will be called *before* RETURN_VALUE, resulting in the KeyboardInterrupt exception being raised. Standard stuff. However, if Lock.__enter__ is a PyCFunction things are quite different. If you look at how the SETUP_WITH opcode is implemented, it first calls the __enter__ method with _PyObjet_CallNoArg. If this returns NULL (i.e. an exception occurred in __enter__) then "goto error" is executed and the exception is raised. However if it returns non-NULL the finally block is set up with PyFrame_BlockSetup and execution proceeds to the next opcode. At this point a potentially waiting SIGINT is handled, resulting in KeyboardInterrupt being raised while inside the with statement's suite, and finally block, and hence Lock.__exit__ are entered. Long story short, because Lock.__enter__ is a C function, assuming that it succeeds normally then with lock: do_something() always guarantees that Lock.__exit__ will be called if a SIGINT was handled inside Lock.__enter__, whereas with lock.acquire() try: ... finally: lock.release() there is at last a small possibility that the SIGINT handler is called after the CALL_FUNCTION op but before the try/finally block is entered (e.g. before executing POP_TOP or SETUP_FINALLY). So the end result is that the lock is held and never released after the KeyboardInterrupt (whether or not it's handled somehow). Whereas, again, if Lock.__enter__ is a pure Python function there's less likely to be any difference (though I don't think the possibility can be ruled out entirely). At the very least I think this quirk of CPython should be mentioned somewhere (since in all other cases the semantic meaning of the "with:" statement is clear). However, I think it might be possible to gain more consistency between these cases if pending signals are checked/handled after any direct call to PyCFunction from within the ceval loop. Sorry for the tl;dr; any thoughts? [View Less]

7 15

Positional-only parameters
by Victor Stinner Sept. 11, 2018

Sept. 11, 2018

Hi, For technical reasons, many functions of the Python standard libraries implemented in C have positional-only parameters. Example: ------- $ ./python Python 3.7.0a0 (default, Feb 25 2017, 04:30:32) >>> help(str.replace) replace(self, old, new, count=-1, /) # <== notice "/" at the end ... >>> "a".replace("x", "y") # ok 'a' >>> "a".replace(old="x", new="y") # ERR! TypeError: replace() takes at least 2 arguments (0 given) ------- When converting the … [View More]

19 27

Proposal: A Reduce-Map Comprehension and a "last" builtin
by Peter O'Connor May 31, 2018

May 31, 2018

Dear all, In Python, I often find myself building lists where each element depends on the last. This generally means making a for-loop, create an initial list, and appending to it in the loop, or creating a generator-function. Both of these feel more verbose than necessary. I was thinking it would be nice to be able to encapsulate this common type of operation into a more compact comprehension. I propose a new "Reduce-Map" comprehension that allows us to write: signal = [math.sin(i*0.01) +… [View More]

22 65

Allow multiple imports from a package while preserving its namespace
by Julian DeMille May 22, 2018

May 22, 2018

I personally would like a feature where instead of doing `from ... import ...` (which imports the specified items into the current namespace), one could use something along the lines of `import <lib>.{ <mod1>, <mod2>, ... }` such that the imported modules/attributes could be accessed as `<lib>.<mod1>`, etc. -- Thanks, Julian DeMille CEO, demilleTech, LLC This email and any files transmitted with it are confidential and intended solely for the use of the … [View More]

10 19

A cute Python implementation of itertools.tee
by Tim Peters May 14, 2018

May 14, 2018

Just for fun - no complaint, no suggestion, just sharing a bit of code that tickled me. The docs for `itertools.tee()` contain a Python work-alike, which is easy to follow. It gives each derived generator its own deque, and when a new value is obtained from the original iterator it pushes that value onto each of those deques. Of course it's possible for them to share a single deque, but the code gets more complicated. Is it possible to make it simpler instead? What it "really" needs is a … [View More]shared singly-linked list of values, pointing from oldest value to newest. Then each derived generator can just follow the links, and yield its next result in time independent of the number of derived generators. But how do you know when a new value needs to be obtained from the original iterator, and how do you know when space for an older value can be recycled (because all of the derived generators have yielded it)? I ended up with almost a full page of code to do that, storing with each value and link a count of the number of derived generators that had yet to yield the value, effectively coding my own reference-count scheme by hand, along with "head" and "tail" pointers to the ends of the linked list that proved remarkably tricky to keep correct in all cases. Then I thought "this is stupid! Python already does reference counting." Voila! Vast swaths of tedious code vanished, giving this remarkably simple implementation: def mytee(xs, n): last = [None, None] def gen(it, mylast): nonlocal last while True: mylast = mylast[1] if not mylast: mylast = last[1] = last = [next(it), None] yield mylast[0] it = iter(xs) return tuple(gen(it, last) for _ in range(n)) There's no need to keep a pointer to the start of the shared list at all - we only need a pointer to the end of the list ("last"), and each derived generator only needs a pointer to its own current position in the list ("mylast"). What I find kind of hilarious is that it's no help at all as a prototype for a C implementation: Python recycles stale `[next(it), None]` pairs all by itself, when their internal refcounts fall to 0. That's the hardest part. BTW, I certainly don't suggest adding this to the itertools docs either. While it's short and elegant, it's too subtle to grasp easily - if you think "it's obvious", you haven't yet thought hard enough about the problem ;-) [View Less]

7 11

string method count()
by Julia Kim May 7, 2018

May 7, 2018

Hi, There’s an error with the string method count(). x = ‘AAA’ y = ‘AA’ print(x.count(y)) The output is 1, instead of 2. I write programs on SoloLearn mobile app. Warm regards, Julia Kim

7 8

A "local" pseudo-function
by Tim Peters May 2, 2018

May 2, 2018

A brain dump, inspired by various use cases that came up during the binding expression discussions. Idea: introduce a "local" pseudo-function to capture the idea of initialized names with limited scope. As an expression, it's "local" "(" arguments ")" - Because it "looks like" a function call, nobody will expect the targets of named arguments to be fancier than plain names. - `a=12` in the "argument" list will (& helpfully so) mean pretty much the same as "a=12" in a "def" … [View More]statement. - In a "local call" on its own, the scope of a named argument begins at the start of the next (if any) argument, and ends at the closing ")". For the duration, any variable of the same name in an enclosing scope is shadowed. - The parentheses allow for extending over multiple lines without needing to teach editors (etc) any new tricks (they already know how to format function calls with arglists spilling over multiple lines). - The _value_ of a local "call" is the value of its last "argument". In part, this is a way to sneak in C's comma operator without adding cryptic new line noise syntax. Time for an example. First a useless one: a = 1 b = 2 c = local(a=3) * local(b=4) Then `c` is 12, but `a` is still 1 and `b` is still 2. Same thing in the end: c = local(a=3, b=4, a*b) And just to be obscure, also the same: c = local(a=3, b=local(a=2, a*a), a*b) There the inner `a=2` temporarily shadows the outer `a=3` just long enough to compute `a*a` (4). This is one that little else really handled nicely: r1, r2 = local(D = b**2 - 4*a*c, sqrtD = math.sqrt(D), twoa = 2*a, ((-b + sqrtD)/twoa, (-b - sqrtD)/twoa)) Everyone's favorite: if local(m = re.match(regexp, line)): print(m.group(0)) Here's where it's truly essential that the compiler know everything about "local", because in _that_ context it's required that the new scope extend through the end of the entire block construct (exactly what that means TBD - certainly through the end of the `if` block, but possibly also through the end of its associated (if any) `elif` and `else` blocks - and similarly for while/else constructs). Of course that example could also be written as: if local(m = re.match(regexp, line), m): print(m.group(0)) or more specifically: if local(m = re.match(regexp, line), m is not None): print(m.group(0)) or even: if local(m = re.match(regexp, line)) is not None: print(m.group(0)) A listcomp example, building the squares of integers from an iterable but only when the square is a multiple of 18: squares18 = [i2 for i in iterable if local(i2=i*i) % 18 == 0] That's a bit mind-bending, but becomes clear if you picture the kinda-equivalent nest: for i in iterable: if local(i2=i*i) % 18 == 0: append i2 to the output list That should also make clear that if `iterable` or `i` had been named `i2` instead, no problem. The `i2` created by `local()` is in a wholly enclosed scope. Drawbacks: since this is just a brain dump, absolutely none ;-) Q: Some of those would be clearer if it were the more Haskell-like local(...) "in" expression A: Yup, but for some of the others needing to add "in m" would be annoyingly redundant noise. Making an "in" clause optional doesn't really fly either, because then local(a='z') in 'xyz' would be ambiguous. Is it meant to return `'xyz'`, or evaluate `'z' in 'xyz'`? And any connector other than "in" would make the loose resemblance to Haskell purely imaginary ;-) Q: Didn't you drone on about how assignment expressions with complex targets seemed essentially useless without also introducing a "comma operator" - and now you're sneaking the latter in but _still_ avoiding complex targets?! A. Yes, and yes :-) The syntactic complexity of the fully general assignment statement is just too crushing to _sanely_ shoehorn into any "expression-like" context. Q: What's the value of this? local(a=7, local(a=a+1, a*2)) A: 16. Obviously. Q: Wow - that _is_ obvious! OK, what about this, where there is no `a` in any enclosing scope: local(a) A: I think it should raise NameError, just like a function call would. There is no _intent_ here to allow merely declaring a local variable without supplying an initial value. Q: What about local(2, 4, 5)? A: It should return 5, and introduce no names. I don't see a point to trying to outlaw stupidity ;-) Then again, it would be consistent with the _intent_ to require that all but the last "argument" be of the `name=expression` form. Q: Isn't changing the meaning of scope depending on context waaaay magical? A: Yup! But in a language with such a strong distinction between statements and expressions, without a bit of deep magic there's no single syntax I can dream up that could work well for both that didn't require _some_ deep magic. The gimmick here is something I expect will be surprising the first time it's seen, less so the second, and then you're never confused about it again. Q: Are you trying to kill PEP 572? A: Nope! But since this largely subsumes the functionality of binding expressions, I did want to put this out there before 572's fate is history. Binding expressions are certainly easier to implement, and I like them just fine :-) Note: the thing I'm most interested in isn't debates, but in whether this would be of real use in real code. [View Less]

21 85

Objectively Quantifying Readability
by Matt Arcidy May 2, 2018

May 2, 2018

The number and type of arguments about readability as a justification, or an opinion, or an opinion about an opinion seems counter-productive to reaching conclusions efficiently. I think they are very important either way, but the justifications used are not rich enough in information to be very useful. A study has been done regarding readability in code which may serve as insight into this issue. Please see page 8, fig 9 for a nice chart of the results, note the negative/positive coloring of … [View More]

13 24

Should __builtins__ have some kind of pass-through print function, for debugging?
by Nathaniel Smith May 1, 2018

May 1, 2018

Hi all, This came up in passing in one of the PEP 572 threads, and I'm curious if folks think it's a good idea or not. When debugging, sometimes you have a somewhat complicated expression that's not working: # Hmm, is func2() returning the right thing? while (func1() + 2 * func2()) < func3(): ... It'd be nice to print out what func2() returns, but to do that we have to refactor this code, which might be rather tricky in a case like this. I think if you want to use print() directly … [View More]here, the simplest way to do that is: while True: tmp = func2() print(tmp) if not (func1() + 2 * func2()) < func3()): break ... Obviously this is annoying and error prone – especially for beginners, who are the ones most likely to need to print out lots of stuff to figure out why their code isn't working. (Chris Angelico mentioned that he finds this to be a common problem when teaching beginners.) There is a better way: if you define a trivial helper like: # "debug print": prints and then returns its argument def dp(obj): print(repr(obj)) return obj then the rewritten code becomes: while (func1() + 2 * dp(func2())) < func3(): ... Of course, this is trivial -- for me or you. But the leap to first realize that this is a useful thing, and then implement it correctly, is really asking a lot of beginners, who by assumption are struggling to do *anything* with Python syntax. And similarly, putting a package on PyPI is useful (cf. the venerable 'q' package), but still adds a significant barrier to entry: you need to be able to install packages, and you need to add an import. In fact, I can imagine that you might want to teach this trick even before you teach what imports are. So, would it make sense to include a utility like this in __builtins__? PEP 553, the breakpoint() builtin, provides some relevant precedent. Looking at it, I see it also emphasized the value of letting IDEs override the debugger, and I can see some similar value here: e.g. fancy REPLs like Spyder or Jupyter could potentially capture the objects passed to dp() and make them available for interactive viewing (imagine if they're like a large dataframe or something). Points to argue over if people like the general idea: - The name: p(), dp(), debug(), debugprint(), ...? - __str__ or __repr__? Presumably __repr__ since it's a debugging tool. - Exact semantics: there should probably be some way to add a bit of metadata that gets printed out, for cases like: while (dp(func1(), "func1") + 2 * dp(func2(), "func2")) < dp(func3(), "func3"): ... Maybe other tweaks would be useful as well. -n -- Nathaniel J. Smith -- https://vorpus.org [View Less]

9 16

Sublocal scoping at its simplest
by Chris Angelico April 30, 2018

April 30, 2018

There's been a lot of talk about sublocal scopes, within and without the context of PEP 572. I'd like to propose what I believe is the simplest form of sublocal scopes, and use it to simplify one specific special case in Python. There are no syntactic changes, and only a very slight semantic change. def f(): e = 2.71828 try: 1/0 except Exception as e: print(e) print(e) f() The current behaviour of the 'except... as' statement is as follows: 1) Bind the caught … [View More]

5 7

Change magic strings to enums
by Jacco van Dorp April 30, 2018

April 30, 2018

A bit ago I was reading some of the python docs ( https://docs.python.org/3.6/library/warnings.html ), the warning module, and I noticed a table of magic strings. I can think of a few other places where magic strings are used - for example, string encoding/decoding locales and strictness, and probably a number of other places. Since Python 3.4, We've been having Enums. Wouldn't it be cleaner to use enums by default instead of those magic strings ? for example, for warnings filter actions, (… [View More]

11 30

Re: [Python-ideas] A "local" pseudo-function
by Ken Hilton April 29, 2018

April 29, 2018

> local { m = re.match(regexp, line) > if m: > print(m.group(0)) > } Or how about making "local" a pseudo-statement of sorts? local (m=re.match(exp, string)) { if m: print(m.group(0)) } The grammar would be as follows: local_stmt = "local" "(" local_assignments [ "," local_assignments ... ] ")" "{" BLOCK "}" local_assignments = NAME "=" EXPR There would be no question about the scope of things in BLOCK - the … [View More]

2 1

Re: [Python-ideas] __loader__.get_source(): semantics of returning None
by Jeroen Demeyer April 28, 2018

April 28, 2018

On 2018-04-28 18:26, Paul Moore wrote: > """ > If a file named filename is not found, the function will look for it > in the module search path, sys.path, after first checking for a PEP > 302 __loader__ in module_globals, in case the module was imported from > a zipfile or other non-filesystem import source. > """ Sorry, I should have been more precise. linecache does the following (which is what is in the doc you quoted above): (1) Look for the filename exactly as given. (… [View More]

2 1

__loader__.get_source(): semantics of returning None
by Jeroen Demeyer April 28, 2018

April 28, 2018

The documentation of __loader__.get_source() says "Returns None if no source is available (e.g. a built-in module)." But what does "no source is available" mean precisely? It could mean either of two things: (A) I am absolutely certain that there is no source anywhere to be found. (B) I don't know where to find the source, but if you look hard enough, you may find it. Currently, linecache interprets it as (A). Is there any chance that we can either change the interpretation for … [View More]

2 1

Re: [Python-ideas] Should __builtins__ have some kind of pass-through print function, for debugging?
by Ken Hilton April 28, 2018

April 28, 2018

Perhaps there could be a special magic method that classes could implement for such a function? I.e. something like this: class Pizza(object): diameter = 1 toppings = [] def __init__(self, diameter=1, toppings=[]): self.diameter = diameter self.toppings = toppings def __str__(self): return '{}-inch pizza, toppings: {}'.format(self.diameter, ', '.join(self.toppings)) def __repr__(self): return '<{}-… [View More]

2 1

Re: [Python-ideas] [Python-Dev] (name := expression) doesn't fit the narrative of PEP 20
by Steven D'Aprano April 28, 2018

April 28, 2018

On Fri, Apr 27, 2018 at 04:24:35PM -0400, Wes Turner wrote: > if ((1) or (x := 3)): > if ((y := func(x)) if x else (x := 3)) Wes, there is *absolutely nothing new* here. This sort of error is already possible in Python. Do you see a lot of code like: if (1 or sequence.append(3) or sequence[-1]): in real life? If you do, then I'm really, really sorry that you are forced to deal with such rubbish code, but honestly, the vast bulk of Python programmers do not write like that, and … [View More]

1 1

Re: [Python-ideas] Change magic strings to enums
by Jeroen Demeyer April 25, 2018

April 25, 2018

On 2018-04-25 09:57, Ivan Levkivskyi wrote: > In the latter case rewriting EnumMeta in C ... or Cython. It's a great language and I'm sure that the Python standard library could benefit a lot from it.

2 1

collections.Counter should implement __mul__, __rmul__
by Peter Norvig April 22, 2018

April 22, 2018

For most types that implement __add__, `x + x` is equal to `2 * x`. That is true for all numbers, list, tuple, str, timedelta, etc. -- but not for collections.Counter. I can add two Counters, but I can't multiply one by a scalar. That seems like an oversight. It would be worthwhile to implement multiplication because, among other reasons, Counters are a nice representation for discrete probability distributions, for which multiplication is an even more fundamental operation than addition. … [View More]

10 37

Checking interned string after stringobjects concat?
by Yinbin Ma April 21, 2018

April 21, 2018

Hi all: I notice that if concatenating two stringobjects, PVM will not check the dictionary of interned string. For example: >>> a = "qwerty" >>> b = "qwe" >>> c = "rty" >>> d = b+c >>> id(a) 4572089736 >>> id(d) 4572111176 >>> e = "".join(["qwe","rty"]) >>> id(e) 4546460280 But if concatenating two string directly, PVM would check the dictionary: >>> a = "qwerty" >>> b = "qwe"+"rty" >>> id(a) … [View More]

5 6

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 19, 2018

April 19, 2018

> > > 0. > > while (items[i := i+1] := read_next_item()) is not None: > print(r'%d/%d' % (i, len(items)), end='\r') > > 1. > > while (read_next_item() -> items[(i+1) -> i]) is not None: > print(r'%d/%d' % (i, len(items)), end='\r') > > 2. > > while (item := read_next_item()) is not None: > items[i := (i+1)] = item > print(r'%d/%d' % (i, len(items)), end='\r') > > 3. > > while (read_next_item() -> item) is … [View More]

4 3

PEP 572: Assignment Expressions (post #4)
by Chris Angelico April 17, 2018

April 17, 2018

Wholesale changes since the previous version. Statement-local name bindings have been dropped (I'm still keeping the idea in the back of my head; this PEP wasn't the first time I'd raised the concept), and we're now focusing primarily on assignment expressions, but also with consequent changes to comprehensions. Sorry for the lengthy delays; getting a reference implementation going took me longer than I expected or intended. ChrisA PEP: 572 Title: Assignment Expressions Author: Chris … [View More]Angelico <rosuav(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Feb-2018 Python-Version: 3.8 Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018 Abstract ======== This is a proposal for creating a way to assign to names within an expression. Additionally, the precise scope of comprehensions is adjusted, to maintain consistency and follow expectations. Rationale ========= Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts. Merely introducing a way to assign as an expression would create bizarre edge cases around comprehensions, though, and to avoid the worst of the confusions, we change the definition of comprehensions, causing some edge cases to be interpreted differently, but maintaining the existing behaviour in the majority of situations. Syntax and semantics ==================== In any context where arbitrary Python expressions can be used, a **named expression** can appear. This can be parenthesized for clarity, and is of the form ``(target := expr)`` where ``expr`` is any valid Python expression, and ``target`` is any valid assignment target. The value of such a named expression is the same as the incorporated expression, with the additional side-effect that the target is assigned that value. # Similar to the boolean 'or' but checking for None specifically x = "default" if (eggs := spam().ham) is None else eggs # Even complex expressions can be built up piece by piece y = ((eggs := spam()), (cheese := eggs.method()), cheese[eggs]) Differences from regular assignment statements ---------------------------------------------- An assignment statement can assign to multiple targets:: x = y = z = 0 To do the same with assignment expressions, they must be parenthesized:: assert 0 == (x := (y := (z := 0))) Augmented assignment is not supported in expression form:: >>> x +:= 1 File "<stdin>", line 1 x +:= 1 ^ SyntaxError: invalid syntax Otherwise, the semantics of assignment are unchanged by this proposal. Alterations to comprehensions ----------------------------- The current behaviour of list/set/dict comprehensions and generator expressions has some edge cases that would behave strangely if an assignment expression were to be used. Therefore the proposed semantics are changed, removing the current edge cases, and instead altering their behaviour *only* in a class scope. As of Python 3.7, the outermost iterable of any comprehension is evaluated in the surrounding context, and then passed as an argument to the implicit function that evaluates the comprehension. Under this proposal, the entire body of the comprehension is evaluated in its implicit function. Names not assigned to within the comprehension are located in the surrounding scopes, as with normal lookups. As one special case, a comprehension at class scope will **eagerly bind** any name which is already defined in the class scope. A list comprehension can be unrolled into an equivalent function. With Python 3.7 semantics:: numbers = [x + y for x in range(3) for y in range(4)] # Is approximately equivalent to def <listcomp>(iterator): result = [] for x in iterator: for y in range(4): result.append(x + y) return result numbers = <listcomp>(iter(range(3))) Under the new semantics, this would instead be equivalent to:: def <listcomp>(): result = [] for x in range(3): for y in range(4): result.append(x + y) return result numbers = <listcomp>() When a class scope is involved, a naive transformation into a function would prevent name lookups (as the function would behave like a method). class X: names = ["Fred", "Barney", "Joe"] prefix = "> " prefixed_names = [prefix + name for name in names] With Python 3.7 semantics, this will evaluate the outermost iterable at class scope, which will succeed; but it will evaluate everything else in a function:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def <listcomp>(iterator): result = [] for name in iterator: result.append(prefix + name) return result prefixed_names = <listcomp>(iter(names)) The name ``prefix`` is thus searched for at global scope, ignoring the class name. Under the proposed semantics, this name will be eagerly bound, being approximately equivalent to:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def <listcomp>(prefix=prefix): result = [] for name in names: result.append(prefix + name) return result prefixed_names = <listcomp>() With list comprehensions, this is unlikely to cause any confusion. With generator expressions, this has the potential to affect behaviour, as the eager binding means that the name could be rebound between the creation of the genexp and the first call to ``next()``. It is, however, more closely aligned to normal expectations. The effect is ONLY seen with names that are looked up from class scope; global names (eg ``range()``) will still be late-bound as usual. One consequence of this change is that certain bugs in genexps will not be detected until the first call to ``next()``, where today they would be caught upon creation of the generator. TODO: Discuss the merits and costs of amelioration proposals. Recommended use-cases ===================== Simplifying list comprehensions ------------------------------- These list comprehensions are all approximately equivalent:: # Calling the function twice stuff = [[f(x), x/f(x)] for x in range(5)] # External helper function def pair(x, value): return [value, x/value] stuff = [pair(x, f(x)) for x in range(5)] # Inline helper function stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] # Extra 'for' loop - potentially could be optimized internally stuff = [[y, x/y] for x in range(5) for y in [f(x)]] # Iterating over a genexp stuff = [[y, x/y] for x, y in ((x, f(x)) for x in range(5))] # Expanding the comprehension into a loop stuff = [] for x in range(5): y = f(x) stuff.append([y, x/y]) # Wrapping the loop in a generator function def g(): for x in range(5): y = f(x) yield [y, x/y] stuff = list(g()) # Using a mutable cache object (various forms possible) c = {} stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] # Using a temporary name stuff = [[y := f(x), x/y] for x in range(5)] If calling ``f(x)`` is expensive or has side effects, the clean operation of the list comprehension gets muddled. Using a short-duration name binding retains the simplicity; while the extra ``for`` loop does achieve this, it does so at the cost of dividing the expression visually, putting the named part at the end of the comprehension instead of the beginning. Capturing condition values -------------------------- Assignment expressions can be used to good effect in the header of an ``if`` or ``while`` statement:: # Current Python, not caring about function return value while input("> ") != "quit": print("You entered a command.") # Current Python, capturing return value - four-line loop header while True: command = input("> "); if command == "quit": break print("You entered:", command) # Proposed alternative to the above while (command := input("> ")) != "quit": print("You entered:", command) # Capturing regular expression match objects # See, for instance, Lib/pydoc.py, which uses a multiline spelling # of this effect if match := re.search(pat, text): print("Found:", match.group(0)) # Reading socket data until an empty string is returned while data := sock.read(): print("Received data:", data) Particularly with the ``while`` loop, this can remove the need to have an infinite loop, an assignment, and a condition. It also creates a smooth parallel between a loop which simply uses a function call as its condition, and one which uses that as its condition but also uses the actual value. Rejected alternative proposals ============================== Proposals broadly similar to this one have come up frequently on python-ideas. Below are a number of alternative syntaxes, some of them specific to comprehensions, which have been rejected in favour of the one given above. Alternative spellings --------------------- Broadly the same semantics as the current proposal, but spelled differently. 1. ``EXPR as NAME``, with or without parentheses:: stuff = [[f(x) as y, x/y] for x in range(5)] Omitting the parentheses in this form of the proposal introduces many syntactic ambiguities. Requiring them in all contexts leaves open the option to make them optional in specific situations where the syntax is unambiguous (cf generator expressions as sole parameters in function calls), but there is no plausible way to make them optional everywhere. With the parentheses, this becomes a viable option, with its own tradeoffs in syntactic ambiguity. Since ``EXPR as NAME`` already has meaning in ``except`` and ``with`` statements (with different semantics), this would create unnecessary confusion or require special-casing. 2. Adorning statement-local names with a leading dot:: stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as" stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":=" This has the advantage that leaked usage can be readily detected, removing some forms of syntactic ambiguity. However, this would be the only place in Python where a variable's scope is encoded into its name, making refactoring harder. This syntax is quite viable, and could be promoted to become the current recommendation if its advantages are found to outweigh its cost. 3. Adding a ``where:`` to any statement to create local name bindings:: value = x**2 + 2*x where: x = spam(1, 4, 7, q) Execution order is inverted (the indented body is performed first, followed by the "header"). This requires a new keyword, unless an existing keyword is repurposed (most likely ``with:``). See PEP 3150 for prior discussion on this subject (with the proposed keyword being ``given:``). Special-casing conditional statements ------------------------------------- One of the most popular use-cases is ``if`` and ``while`` statements. Instead of a more general solution, this proposal enhances the syntax of these two statements to add a means of capturing the compared value:: if re.search(pat, text) as match: print("Found:", match.group(0)) This works beautifully if and ONLY if the desired condition is based on the truthiness of the captured value. It is thus effective for specific use-cases (regex matches, socket reads that return `''` when done), and completely useless in more complicated cases (eg where the condition is ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has no benefit to list comprehensions. Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction of possible use-cases, even in ``if``/``while`` statements. Special-casing comprehensions ----------------------------- Another common use-case is comprehensions (list/set/dict, and genexps). As above, proposals have been made for comprehension-specific solutions. 1. ``where``, ``let``, or ``given``:: stuff = [(y, x/y) where y = f(x) for x in range(5)] stuff = [(y, x/y) let y = f(x) for x in range(5)] stuff = [(y, x/y) given y = f(x) for x in range(5)] This brings the subexpression to a location in between the 'for' loop and the expression. It introduces an additional language keyword, which creates conflicts. Of the three, ``where`` reads the most cleanly, but also has the greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` methods, as does ``tkinter.dnd.Icon`` in the standard library). 2. ``with NAME = EXPR``:: stuff = [(y, x/y) with y = f(x) for x in range(5)] As above, but reusing the `with` keyword. Doesn't read too badly, and needs no additional language keyword. Is restricted to comprehensions, though, and cannot as easily be transformed into "longhand" for-loop syntax. Has the C problem that an equals sign in an expression can now create a name binding, rather than performing a comparison. Would raise the question of why "with NAME = EXPR:" cannot be used as a statement on its own. 3. ``with EXPR as NAME``:: stuff = [(y, x/y) with f(x) as y for x in range(5)] As per option 2, but using ``as`` rather than an equals sign. Aligns syntactically with other uses of ``as`` for name binding, but a simple transformation to for-loop longhand would create drastically different semantics; the meaning of ``with`` inside a comprehension would be completely different from the meaning as a stand-alone statement, while retaining identical syntax. Regardless of the spelling chosen, this introduces a stark difference between comprehensions and the equivalent unrolled long-hand form of the loop. It is no longer possible to unwrap the loop into statement form without reworking any name bindings. The only keyword that can be repurposed to this task is ``with``, thus giving it sneakily different semantics in a comprehension than in a statement; alternatively, a new keyword is needed, with all the costs therein. Migration path ============== The semantic changes to list/set/dict comprehensions, and more so to generator expressions, may potentially require migration of code. In many cases, the changes simply make legal what used to raise an exception, but there are some edge cases that were previously legal and are not, and a few corner cases with altered semantics. Yield inside comprehensions --------------------------- As of Python 3.7, the outermost iterable in a comprehension is permitted to contain a 'yield' expression. If this is required, the iterable (or at least the yield) must be explicitly elevated from the comprehension:: # Python 3.7 def g(): return [x for x in [(yield 1)]] # With PEP 572 def g(): sent_item = (yield 1) return [x for x in [sent_item]] This more clearly shows that it is g(), not the comprehension, which is able to yield values (and is thus a generator function). The entire comprehension is consistently in a single scope. Name lookups in class scope --------------------------- A comprehension inside a class previously was able to 'see' class members ONLY from the outermost iterable. Other name lookups would ignore the class and potentially locate a name at an outer scope:: pattern = "<%d>" class X: pattern = "[%d]" numbers = [pattern % n for n in range(5)] In Python 3.7, ``X.numbers`` would show angle brackets; with PEP 572, it would show square brackets. Maintaining the current behaviour here is best done by using distinct names for the different forms of ``pattern``, as would be the case with functions. Generator expression bugs can be caught later --------------------------------------------- Certain types of bugs in genexps were previously caught more quickly. Some are now detected only at first iteration:: gen = (x for x in rage(10)) # NameError gen = (x for x in 10) # TypeError (not iterable) gen = (x for x in range(1/0)) # Exception raised during evaluation This brings such generator expressions in line with a simple translation to function form:: def <genexp>(): for x in rage(10): yield x gen = <genexp>() # No exception yet tng = next(gen) # NameError To detect these errors more quickly, ... TODO. Open questions ============== Can the outermost iterable still be evaluated early? ---------------------------------------------------- As of Python 3.7, the outermost iterable in a genexp is evaluated early, and the result passed to the implicit function as an argument. With PEP 572, this would no longer be the case. Can we still, somehow, evaluate it before moving on? One possible implementation would be:: gen = (x for x in rage(10)) # translates to def <genexp>(): iterable = iter(rage(10)) yield None for x in iterable: yield x gen = <genexp>() next(gen) This would pump the iterable up to just before the loop starts, evaluating exactly as much as is evaluated outside the generator function in Py3.7. This would result in it being possible to call ``gen.send()`` immediately, unlike with most generators, and may incur unnecessary overhead in the common case where the iterable is pumped immediately (perhaps as part of a larger expression). Frequently Raised Objections ============================ Why not just turn existing assignment into an expression? --------------------------------------------------------- C and its derivatives define the ``=`` operator as an expression, rather than a statement as is Python's way. This allows assignments in more contexts, including contexts where comparisons are more common. The syntactic similarity between ``if (x == y)`` and ``if (x = y)`` belies their drastically different semantics. Thus this proposal uses ``:=`` to clarify the distinction. This could be used to create ugly code! --------------------------------------- So can anything else. This is a tool, and it is up to the programmer to use it where it makes sense, and not use it where superior constructs can be used. With assignment expressions, why bother with assignment statements? ------------------------------------------------------------------- The two forms have different flexibilities. The ``:=`` operator can be used inside a larger expression; the ``=`` operator can be chained more conveniently, and closely parallels the inline operations ``+=`` and friends. The assignment statement is a clear declaration of intent: this value is to be assigned to this target, and that's it. Acknowledgements ================ The author wishes to thank Guido van Rossum and Nick Coghlan for their considerable contributions to this proposal, and to members of the core-mentorship mailing list for assistance with implementation. References ========== .. [1] Proof of concept / reference implementation (https://github.com/Rosuav/cpython/tree/assignment-expressions) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

24 119

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 17, 2018

April 17, 2018

> Hey! I did not propose "?". Read the explanation in parenthesis. > My whole idea that any option could be viable, as long as > it does not propose reversed order notation. Dear Mikhail, so sorry about my misunderstanding, I should read the post with fewer presuppositions... Back to the topic, this thread seems to be closed now, and in my opinion `:=` could be synthetically the best. 2018-04-17 15:11 GMT+08:00 Mikhail V <mikhailwas(a)gmail.com>: > On Tue, Apr 17, 2018 at … [View More]

1 0

Providing a public API for creating and parsing HTTP messages
by Derek Maciel April 17, 2018

April 17, 2018

Hello all, If this is not the appropriate place for this type of proposal please let me know. The modules http.client and http.server both do a wonderful job when implementing HTTP clients and servers, respectively. However, occasionally there may be a need to create and parse HTTP messages themselves without needing to implement a client or server. While http.client and http.server clearly have solved this problem for its own uses, there is little to no public API exposed to a user wishing … [View More]

2 1

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 17, 2018

April 17, 2018

> We have ways of cheating a bit if we want to reinterpret the semantics > of something that nevertheless parses cleanly - while the parser is > limited to single token lookahead, it's straightforward for the > subsequent code generation stage to look a single level down in the > parse tree and see that the code that parsed as "with expr" is > actually "with subexpr as target". It does work, however I think it does sound like a patch, and definitely it will block us to make … [View More]

3 2

Idea: Importing from arbitrary filenames
by Ken Hilton April 16, 2018

April 16, 2018

Hi all, First of all, please excuse me if I'm presenting this idea in the wrong way or at the wrong time - I'm new to this mailing list and haven't seen anyone propose a new idea on it yet, so I don't know the customs. I have an idea for importing files with arbitrary names. Currently, the "official" way to import arbitrary files is to use the "imp" module, as shown by this answer: https://stackoverflow.com/a/3137914/6605349 However, this method takes two function calls and is not as (… [View More]aesthetically pleasing? is that the word?) as a simple "import" statement. Therefore, my idea is to allow the "import" statement to accept one of three targets. First, the normal "import": import antigravity which simply imports from sys.path. Second, importing with a string literal specifying the path to a file: import '/home/pi/anti-gravity.py' *as antigravity* Note the "as antigravity" in this statement - this is to avoid ambiguities when choosing the global name to bind to. Should "import '/home/pi/anti-gravity.py'" import to the name "/home/pi/anti-gravity.py", "anti-gravity.py", "anti-gravity", or "anti_gravity"? None of those are really ideal. Therefore, when the import target is a string literal, the statement must include "as NAME". Third, importing with an expression providing a value castable to a string, specifying the path to a file: def file_in_home(filename): return '/home/pi/' + filename import *$*file_in_home('anti-gravity.py') *as antigravity* Once again, for the same reasons, import statements like this must include "as NAME" to avoid ambiguities. Notice that the expression is preceded by a dollar sign ($) to indicate that what follows is an expression rather than a name - imagine a scenario like this: antigravity_file = '/home/pi/anti-gravity.py' import antigravity_file as antigravity Should it look for a sys.path module with the name "antigravity_file" or should it use the value of the variable "antigravity_file"? Looking for the sys.path module first before trying a variable's value would waste processing time and potentially be unexpected behavior. Trying a variable's value first before looking for a sys.path module would be even less expected behavior. Therefore, a dollar sign must come before expression imports to indicate that the import target is an expression. Side note: the dollar sign was chosen because it mimics other languages' conventions of preceding variable names with dollar signs, but any arbitrary character not present at the start of an expression would work. One more thing about expression imports: if the final returned value of the expression is not a string, I believe the statement should raise a TypeError (the same way that __repr__ or __str__ raise TypeError if they return a non-string). Why? If the statement attempted to cast the return value to a string, and the return value's __str__ method raised an error, then should the statement allow the error to pass through, or should it attempt to use a parent class's __str__ method? Allowing the error to pass through would almost certainly be unexpected behavior; attempting to use a parent class's __str__ method would take more time and more processing power (though it would eventually reach "object"'s __str__ method and succeed). Therefore, non-string expression values should raise TypeError. What are your thoughts? Regards , Ken Hilton ; [View Less]

8 12

Move optional data out of pyc files
by Serhiy Storchaka April 16, 2018

April 16, 2018

Currently pyc files contain data that is useful mostly for developing and is not needed in most normal cases in stable program. There is even an option that allows to exclude a part of this information from pyc files. It is expected that this saves memory, startup time, and disk space (or the time of loading from network). I propose to move this data from pyc files into separate file or files. pyc files should contain only external references to external files. If the corresponding … [View More]

22 35

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 16, 2018

April 16, 2018

> To me, "from" strongly suggests that an element is being obtained from a container/collection of > elements. This is how I conceptualize "from module import name": "name" refers to an object > INSIDE the module, not the module itself. If I saw > > if (match from pattern.search(data)) is not None: ... > I would guess that it is equivalent to > > m = next(pattern.search(data)) > if m is not None: ... +1, although unpacking seems to be reasonable `[… [View More]

2 1

Rewriting file - pythonic way
by Alexey Shrub April 16, 2018

April 16, 2018

Hi all, I am new in python (i am moving from Perl world), but I always love Python for hight level, beatuful and clean syntax. Now I have question/idea about working with files. On mine opinion it very popular use case: 1. Open file (for read and write) 2. Read data from file 3. Modify data. 4. Rewrite file by modified data. But now it is looks not so pythonic: with open(filename, 'r+') as file: data = file.read() data = data.replace('old', 'new') file.seek(0) file.write(… [View More]

8 18

Accepting multiple mappings as positional arguments to create dicts
by Andrés Delfino April 15, 2018

April 15, 2018

Hi! I thought that maybe dict could accept several mappings as positional arguments, like this: class Dict4(dict): > def __init__(self, *args, **kwargs): > if len(args) > 1: > if not all([isinstance(arg, dict) for arg in args]): > raise TypeError('Dict4 expected instances of dict since > multiple positional arguments were passed') > > temp = args[0].copy() > > for arg in args[1:]: > … [View More]

9 21

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 15, 2018

April 15, 2018

Dear Steve, I'm sorry to annoy you by my proposal, but I do think using unicode might be wise in current stage. \triangleq could be print with unicode number \u225c, and adding plugins to support typing this in editors could be easy, just simply map \xxx to the specific unicode char when we press the tab after typing it. People using Julia language are proud of it but I think it's just something convenient could be used in any other language. There are other reasons to support unicode but it'… [View More]

2 1

Re: [Python-ideas] PEP 572: Assignment Expressions (post #4)
by Thautwarm Zhao April 13, 2018

April 13, 2018

> You're looking at a very early commit there. I suggest looking at the > most recent commits on one of two branches: https://github.com/Rosuav/cpython/blob/statement-local- variables/Grammar/Grammar https://github.com/Rosuav/cpython/blob/assignment- expressions/Grammar/Grammar > Those are the two most recent states in my progress towards (a) > statement-local name bindings with "EXPR as NAME", and (b) assignment > expressions with "target := value". Okay, and the syntax "… [View More]

10 18

PEP 573: Module State Access from C Extension Methods
by Marcel Plch April 13, 2018

April 13, 2018

I have prepared a PEP for better support of PEP 489 multiphase initialization. It makes it possible for extension types, their methods to be more specific, to access the state of the module they are defined in. conversation - https://mail.python.org/pipermail/import-sig/2015-July/001022.html PEP - https://github.com/python/peps/blob/master/pep-0573.rst Implementation - https://github.com/Traceur759/cpython/pull/4

1 0

Re: [Python-ideas] PEP 572: Assignment Expressions (post #4)
by Thautwarm Zhao April 12, 2018

April 12, 2018

> > Makes sense. However, couldn't you prevent that by giving with > priority over the binding ? As in "(with simple_cm) as value", where > > we consider the "as" as binding operator instead of part of the with > > statement ? Sure, you could commit suicide by parenthesis, but by > > default it'd do exactly what the "with simple_cm as value" currently > > does. This does require use of as instead of :=, though. (which was > > the point I was trying to make,… [View More] apologies for the confusion) > > Does "(with simple_cm) as value" means "with (simple_cm as value)"? > If so, it's impossible to let the priority of "with ... as ..." over `as` binding. > > This is the grammar of current syntax related to with statement: > > with_stmt: 'with' with_item (',' with_item)* ':' suite > with_item: test ['as' expr] > > If `as` binding could be used in a general expression, just as > `test` is the top of expression, an expression using `as` binding must be in the structure > `test`. > In other words, if you write > > with expr as name: > # do stuff > > Without doubt it's equivalent to `with (expr as name)`. > > Or you want to completely change the grammar design of CPython :) > > thautwarm Additionally, here is an evidence. I've just had a look at Chris Angelico's implementation about expression assignment, it's cool, however the problem is still raised. https://github.com/Rosuav/cpython/blob/0f237048b7665720b5165a40de0ed601c1e8… `as` binding is added at line 111, obviously you cannot separate it from the `test` structure(because `test` is the top expr). testlist_comp: (test|star_expr) ( comp_for | 'as' NAME | (',' (test|star_expr))* [','] ) It seems that if we're to support expression assignment, `as` binding should be declined. To be honest I feel upset because I think `expr as name` is really cool and pythonic. thautwarm 2018-04-13 1:35 GMT+08:00 Thautwarm Zhao <yaoxiansamma(a)gmail.com>: > > Makes sense. However, couldn't you prevent that by giving with > > priority over the binding ? As in "(with simple_cm) as value", where > > we consider the "as" as binding operator instead of part of the with > > statement ? Sure, you could commit suicide by parenthesis, but by > > default it'd do exactly what the "with simple_cm as value" currently > > does. This does require use of as instead of :=, though. (which was > > the point I was trying to make, apologies for the confusion) > > Does "(with simple_cm) as value" means "with (simple_cm as value)"? > If so, it's impossible to let the priority of "with ... as ..." over `as` > binding. > > This is the grammar of current syntax related to with statement: > > with_stmt: 'with' with_item (',' with_item)* ':' suite > with_item: test ['as' expr] > > If `as` binding could be used in a general expression, just as > `test` is the top of expression, an expression using `as` binding must be > in the structure `test`. > In other words, if you write > > with expr as name: > # do stuff > > Without doubt it's equivalent to `with (expr as name)`. > > Or you want to completely change the grammar design of CPython :) > > thautwarm > > > > 2018-04-12 21:41 GMT+08:00 <python-ideas-request(a)python.org>: > >> Send Python-ideas mailing list submissions to >> python-ideas(a)python.org >> >> To subscribe or unsubscribe via the World Wide Web, visit >> https://mail.python.org/mailman/listinfo/python-ideas >> or, via email, send a message with subject or body 'help' to >> python-ideas-request(a)python.org >> >> You can reach the person managing the list at >> python-ideas-owner(a)python.org >> >> When replying, please edit your Subject line so it is more specific >> than "Re: Contents of Python-ideas digest..." >> >> Today's Topics: >> >> 1. Re: PEP 572: Assignment Expressions (post #4) (Chris Angelico) >> 2. Re: PEP 572: Assignment Expressions (post #4) (Chris Angelico) >> 3. Re: PEP 572: Assignment Expressions (post #4) (Nick Coghlan) >> 4. Re: PEP 572: Assignment Expressions (post #4) (Jacco van Dorp) >> 5. Re: PEP 572: Assignment Expressions (post #4) (Chris Angelico) >> >> >> ---------- 已转发邮件 ---------- >> From: Chris Angelico <rosuav(a)gmail.com> >> To: python-ideas <python-ideas(a)python.org> >> Cc: >> Bcc: >> Date: Thu, 12 Apr 2018 23:08:06 +1000 >> Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) >> On Thu, Apr 12, 2018 at 9:09 PM, Paul Moore <p.f.moore(a)gmail.com> wrote: >> > On 11 April 2018 at 22:28, Chris Angelico <rosuav(a)gmail.com> wrote: >> >> On Thu, Apr 12, 2018 at 1:22 AM, Nick Coghlan <ncoghlan(a)gmail.com> >> wrote: >> >>> This argument will be strengthened by making the examples used in the >> >>> PEP itself more attractive, as well as proposing suitable additions to >> >>> PEP 8, such as: >> >>> >> >>> 1. If either assignment statements or assignment expressions can be >> >>> used, prefer statements >> >>> 2. If using assignment expressions would lead to ambiguity about >> >>> execution order, restructure to use statements instead >> >> >> >> Fair enough. Also adding that chained assignment expressions should >> >> generally be avoided. >> > >> > Another one I think should be included (I'm a bit sad that it's not so >> > obvious that no-one would ever even think of it, but the current >> > discussion pretty much killed that hope for me). >> > >> > * Assignment expressions should never be used standalone - assignment >> > statements should *always* be used in that case. >> >> That's covered by the first point. If it's a standalone statement, >> then the statement form could be used, ergo you should prefer the >> statement form. >> >> ChrisA >> >> >> >> ---------- 已转发邮件 ---------- >> From: Chris Angelico <rosuav(a)gmail.com> >> To: python-ideas <python-ideas(a)python.org> >> Cc: >> Bcc: >> Date: Thu, 12 Apr 2018 23:14:34 +1000 >> Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) >> On Thu, Apr 12, 2018 at 7:21 PM, Kirill Balunov <kirillbalunov(a)gmail.com> >> wrote: >> > >> > I gain readability! I don't see any reason to use it in other >> contexts... >> > Because it makes the code unreadable and difficult to perceive while >> giving >> > not so much benefit. I may be wrong, but so far I have not seen a >> single >> > example that at least slightly changed my mind. >> >> This is, in effect, your entire argument for permitting assignments >> only in certain contexts. "I can't think of any useful reason for >> doing this, so we shouldn't do it". But that means making the language >> grammar more complicated (both in the technical sense of the parser's >> definitions, and in the colloquial sense of how you'd explain Python >> to a new programmer), because there are these magic constructs that >> can be used anywhere in an expression, but ONLY if that expression is >> inside an if or while statement. You lose the ability to refactor your >> code simply to satisfy an arbitrary restriction to appease someone's >> feeling of "it can't be useful anywhere else". >> >> There are basically two clean ways to do this: >> >> 1) Create actual syntax as part of the while statement, in the same >> way that the 'with EXPR as NAME:' statement does. This means you >> cannot put any additional operators after the 'as NAME' part. It's as >> much a part of the statement's syntax as the word 'in' is in a for >> loop. >> >> 2) Make this a feature of expressions in general. Then they can be >> used anywhere that an expression can be. >> >> I've gone for option 2. If you want to push for option 1, go ahead, >> but it's a nerfed solution just because you personally cannot think of >> any good use for this. >> >> ChrisA >> >> >> >> ---------- 已转发邮件 ---------- >> From: Nick Coghlan <ncoghlan(a)gmail.com> >> To: Chris Angelico <rosuav(a)gmail.com> >> Cc: python-ideas <python-ideas(a)python.org> >> Bcc: >> Date: Thu, 12 Apr 2018 23:19:59 +1000 >> Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) >> On 12 April 2018 at 07:28, Chris Angelico <rosuav(a)gmail.com> wrote: >> > On Thu, Apr 12, 2018 at 1:22 AM, Nick Coghlan <ncoghlan(a)gmail.com> >> wrote: >> >>> Frequently Raised Objections >> >>> ============================ >> >> >> >> There needs to be a subsection here regarding the need to call `del` >> >> at class and module scope, just as there is for loop iteration >> >> variables at those scopes. >> > >> > Hmm, I'm not sure I follow. Are you saying that this is an objection >> > to assignment expressions, or an objection to them not being >> > statement-local? If the latter, it's really more about "rejected >> > alternative proposals". >> >> It's both - accidentally polluting class and module namespaces is an >> argument against expression level assignments in general, and sublocal >> namespaces aimed to eliminate that downside. >> >> Since feedback on the earlier versions of the PEP has moved sublocal >> namespaces into the "rejected due to excessive conceptual complexity" >> box, that means accidental namespace pollution comes back as a >> downside that the PEP should mention. >> >> I don't think it needs to say much, just point out that they share the >> downside of regular for loops: if you use one at class or module >> scope, and don't want to export the name, you need to delete it >> explicitly. >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia >> >> >> >> ---------- 已转发邮件 ---------- >> From: Jacco van Dorp <j.van.dorp(a)deonet.nl> >> To: python-ideas <python-ideas(a)python.org> >> Cc: >> Bcc: >> Date: Thu, 12 Apr 2018 15:31:59 +0200 >> Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) >> 2018-04-12 15:02 GMT+02:00 Nick Coghlan <ncoghlan(a)gmail.com>: >> > On 12 April 2018 at 22:22, Jacco van Dorp <j.van.dorp(a)deonet.nl> wrote: >> >> I've looked through PEP 343, contextlib docs ( >> >> https://docs.python.org/3/library/contextlib.html ), and I couldn't >> >> find a single case where "with (y := f(x))" would be invalid. >> > >> > Consider this custom context manager: >> > >> > @contextmanager >> > def simple_cm(): >> > yield 42 >> > >> > Given that example, the following code: >> > >> > with cm := simple_cm() as value: >> > print(cm.func.__name__, value) >> > >> > would print "'simple_cm 42", since the assignment expression would >> > reference the context manager itself, while the with statement binds >> > the yielded value. >> > >> > Another relevant example would be `contextlib.closing`: that returns >> > the passed in argument from __enter__, *not* self. >> > >> > And that's why earlier versions of PEP 572 (which used the "EXPR as >> > NAME" spelling) just flat out prohibited top level name binding >> > expressions in with statements: "with (expr as name):" and "with expr >> > as name:" were far too different semantically for the only syntactic >> > difference to be a surrounding set of parentheses. >> > >> > Cheers, >> > Nick. >> >> Makes sense. However, couldn't you prevent that by giving with >> priority over the binding ? As in "(with simple_cm) as value", where >> we consider the "as" as binding operator instead of part of the with >> statement ? Sure, you could commit suicide by parenthesis, but by >> default it'd do exactly what the "with simple_cm as value" currently >> does. This does require use of as instead of :=, though. (which was >> the point I was trying to make, apologies for the confusion) >> >> >> >> ---------- 已转发邮件 ---------- >> From: Chris Angelico <rosuav(a)gmail.com> >> To: python-ideas <python-ideas(a)python.org> >> Cc: >> Bcc: >> Date: Thu, 12 Apr 2018 23:41:49 +1000 >> Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) >> On Thu, Apr 12, 2018 at 11:31 PM, Jacco van Dorp <j.van.dorp(a)deonet.nl> >> wrote: >> > 2018-04-12 15:02 GMT+02:00 Nick Coghlan <ncoghlan(a)gmail.com>: >> >> On 12 April 2018 at 22:22, Jacco van Dorp <j.van.dorp(a)deonet.nl> >> wrote: >> >>> I've looked through PEP 343, contextlib docs ( >> >>> https://docs.python.org/3/library/contextlib.html ), and I couldn't >> >>> find a single case where "with (y := f(x))" would be invalid. >> >> >> >> Consider this custom context manager: >> >> >> >> @contextmanager >> >> def simple_cm(): >> >> yield 42 >> >> >> >> Given that example, the following code: >> >> >> >> with cm := simple_cm() as value: >> >> print(cm.func.__name__, value) >> >> >> >> would print "'simple_cm 42", since the assignment expression would >> >> reference the context manager itself, while the with statement binds >> >> the yielded value. >> >> >> >> Another relevant example would be `contextlib.closing`: that returns >> >> the passed in argument from __enter__, *not* self. >> >> >> >> And that's why earlier versions of PEP 572 (which used the "EXPR as >> >> NAME" spelling) just flat out prohibited top level name binding >> >> expressions in with statements: "with (expr as name):" and "with expr >> >> as name:" were far too different semantically for the only syntactic >> >> difference to be a surrounding set of parentheses. >> >> >> >> Cheers, >> >> Nick. >> > >> > Makes sense. However, couldn't you prevent that by giving with >> > priority over the binding ? As in "(with simple_cm) as value", where >> > we consider the "as" as binding operator instead of part of the with >> > statement ? Sure, you could commit suicide by parenthesis, but by >> > default it'd do exactly what the "with simple_cm as value" currently >> > does. This does require use of as instead of :=, though. (which was >> > the point I was trying to make, apologies for the confusion) >> >> If you want this to be a generic name-binding operation, then no; most >> objects cannot be used as context managers. You'll get an exception if >> you try to use "with 1 as x:", for instance. >> >> As Nick mentioned, there are context managers that return something >> other than 'self', and for those, "with expr as name:" has an >> important meaning that cannot easily be captured with an assignment >> operator. >> >> ChrisA >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas(a)python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> >> > [View Less]

2 1

Re: [Python-ideas] Python-ideas Digest, Vol 137, Issue 67
by Thautwarm Zhao April 12, 2018

April 12, 2018

> Makes sense. However, couldn't you prevent that by giving with > priority over the binding ? As in "(with simple_cm) as value", where > we consider the "as" as binding operator instead of part of the with > statement ? Sure, you could commit suicide by parenthesis, but by > default it'd do exactly what the "with simple_cm as value" currently > does. This does require use of as instead of :=, though. (which was > the point I was trying to make, apologies for the confusion) … [View More] Does "(with simple_cm) as value" means "with (simple_cm as value)"? If so, it's impossible to let the priority of "with ... as ..." over `as` binding. This is the grammar of current syntax related to with statement: with_stmt: 'with' with_item (',' with_item)* ':' suite with_item: test ['as' expr] If `as` binding could be used in a general expression, just as `test` is the top of expression, an expression using `as` binding must be in the structure `test`. In other words, if you write with expr as name: # do stuff Without doubt it's equivalent to `with (expr as name)`. Or you want to completely change the grammar design of CPython :) thautwarm 2018-04-12 21:41 GMT+08:00 <python-ideas-request(a)python.org>: > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > > Today's Topics: > > 1. Re: PEP 572: Assignment Expressions (post #4) (Chris Angelico) > 2. Re: PEP 572: Assignment Expressions (post #4) (Chris Angelico) > 3. Re: PEP 572: Assignment Expressions (post #4) (Nick Coghlan) > 4. Re: PEP 572: Assignment Expressions (post #4) (Jacco van Dorp) > 5. Re: PEP 572: Assignment Expressions (post #4) (Chris Angelico) > > > ---------- 已转发邮件 ---------- > From: Chris Angelico <rosuav(a)gmail.com> > To: python-ideas <python-ideas(a)python.org> > Cc: > Bcc: > Date: Thu, 12 Apr 2018 23:08:06 +1000 > Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) > On Thu, Apr 12, 2018 at 9:09 PM, Paul Moore <p.f.moore(a)gmail.com> wrote: > > On 11 April 2018 at 22:28, Chris Angelico <rosuav(a)gmail.com> wrote: > >> On Thu, Apr 12, 2018 at 1:22 AM, Nick Coghlan <ncoghlan(a)gmail.com> > wrote: > >>> This argument will be strengthened by making the examples used in the > >>> PEP itself more attractive, as well as proposing suitable additions to > >>> PEP 8, such as: > >>> > >>> 1. If either assignment statements or assignment expressions can be > >>> used, prefer statements > >>> 2. If using assignment expressions would lead to ambiguity about > >>> execution order, restructure to use statements instead > >> > >> Fair enough. Also adding that chained assignment expressions should > >> generally be avoided. > > > > Another one I think should be included (I'm a bit sad that it's not so > > obvious that no-one would ever even think of it, but the current > > discussion pretty much killed that hope for me). > > > > * Assignment expressions should never be used standalone - assignment > > statements should *always* be used in that case. > > That's covered by the first point. If it's a standalone statement, > then the statement form could be used, ergo you should prefer the > statement form. > > ChrisA > > > > ---------- 已转发邮件 ---------- > From: Chris Angelico <rosuav(a)gmail.com> > To: python-ideas <python-ideas(a)python.org> > Cc: > Bcc: > Date: Thu, 12 Apr 2018 23:14:34 +1000 > Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) > On Thu, Apr 12, 2018 at 7:21 PM, Kirill Balunov <kirillbalunov(a)gmail.com> > wrote: > > > > I gain readability! I don't see any reason to use it in other contexts... > > Because it makes the code unreadable and difficult to perceive while > giving > > not so much benefit. I may be wrong, but so far I have not seen a single > > example that at least slightly changed my mind. > > This is, in effect, your entire argument for permitting assignments > only in certain contexts. "I can't think of any useful reason for > doing this, so we shouldn't do it". But that means making the language > grammar more complicated (both in the technical sense of the parser's > definitions, and in the colloquial sense of how you'd explain Python > to a new programmer), because there are these magic constructs that > can be used anywhere in an expression, but ONLY if that expression is > inside an if or while statement. You lose the ability to refactor your > code simply to satisfy an arbitrary restriction to appease someone's > feeling of "it can't be useful anywhere else". > > There are basically two clean ways to do this: > > 1) Create actual syntax as part of the while statement, in the same > way that the 'with EXPR as NAME:' statement does. This means you > cannot put any additional operators after the 'as NAME' part. It's as > much a part of the statement's syntax as the word 'in' is in a for > loop. > > 2) Make this a feature of expressions in general. Then they can be > used anywhere that an expression can be. > > I've gone for option 2. If you want to push for option 1, go ahead, > but it's a nerfed solution just because you personally cannot think of > any good use for this. > > ChrisA > > > > ---------- 已转发邮件 ---------- > From: Nick Coghlan <ncoghlan(a)gmail.com> > To: Chris Angelico <rosuav(a)gmail.com> > Cc: python-ideas <python-ideas(a)python.org> > Bcc: > Date: Thu, 12 Apr 2018 23:19:59 +1000 > Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) > On 12 April 2018 at 07:28, Chris Angelico <rosuav(a)gmail.com> wrote: > > On Thu, Apr 12, 2018 at 1:22 AM, Nick Coghlan <ncoghlan(a)gmail.com> > wrote: > >>> Frequently Raised Objections > >>> ============================ > >> > >> There needs to be a subsection here regarding the need to call `del` > >> at class and module scope, just as there is for loop iteration > >> variables at those scopes. > > > > Hmm, I'm not sure I follow. Are you saying that this is an objection > > to assignment expressions, or an objection to them not being > > statement-local? If the latter, it's really more about "rejected > > alternative proposals". > > It's both - accidentally polluting class and module namespaces is an > argument against expression level assignments in general, and sublocal > namespaces aimed to eliminate that downside. > > Since feedback on the earlier versions of the PEP has moved sublocal > namespaces into the "rejected due to excessive conceptual complexity" > box, that means accidental namespace pollution comes back as a > downside that the PEP should mention. > > I don't think it needs to say much, just point out that they share the > downside of regular for loops: if you use one at class or module > scope, and don't want to export the name, you need to delete it > explicitly. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia > > > > ---------- 已转发邮件 ---------- > From: Jacco van Dorp <j.van.dorp(a)deonet.nl> > To: python-ideas <python-ideas(a)python.org> > Cc: > Bcc: > Date: Thu, 12 Apr 2018 15:31:59 +0200 > Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) > 2018-04-12 15:02 GMT+02:00 Nick Coghlan <ncoghlan(a)gmail.com>: > > On 12 April 2018 at 22:22, Jacco van Dorp <j.van.dorp(a)deonet.nl> wrote: > >> I've looked through PEP 343, contextlib docs ( > >> https://docs.python.org/3/library/contextlib.html ), and I couldn't > >> find a single case where "with (y := f(x))" would be invalid. > > > > Consider this custom context manager: > > > > @contextmanager > > def simple_cm(): > > yield 42 > > > > Given that example, the following code: > > > > with cm := simple_cm() as value: > > print(cm.func.__name__, value) > > > > would print "'simple_cm 42", since the assignment expression would > > reference the context manager itself, while the with statement binds > > the yielded value. > > > > Another relevant example would be `contextlib.closing`: that returns > > the passed in argument from __enter__, *not* self. > > > > And that's why earlier versions of PEP 572 (which used the "EXPR as > > NAME" spelling) just flat out prohibited top level name binding > > expressions in with statements: "with (expr as name):" and "with expr > > as name:" were far too different semantically for the only syntactic > > difference to be a surrounding set of parentheses. > > > > Cheers, > > Nick. > > Makes sense. However, couldn't you prevent that by giving with > priority over the binding ? As in "(with simple_cm) as value", where > we consider the "as" as binding operator instead of part of the with > statement ? Sure, you could commit suicide by parenthesis, but by > default it'd do exactly what the "with simple_cm as value" currently > does. This does require use of as instead of :=, though. (which was > the point I was trying to make, apologies for the confusion) > > > > ---------- 已转发邮件 ---------- > From: Chris Angelico <rosuav(a)gmail.com> > To: python-ideas <python-ideas(a)python.org> > Cc: > Bcc: > Date: Thu, 12 Apr 2018 23:41:49 +1000 > Subject: Re: [Python-ideas] PEP 572: Assignment Expressions (post #4) > On Thu, Apr 12, 2018 at 11:31 PM, Jacco van Dorp <j.van.dorp(a)deonet.nl> > wrote: > > 2018-04-12 15:02 GMT+02:00 Nick Coghlan <ncoghlan(a)gmail.com>: > >> On 12 April 2018 at 22:22, Jacco van Dorp <j.van.dorp(a)deonet.nl> wrote: > >>> I've looked through PEP 343, contextlib docs ( > >>> https://docs.python.org/3/library/contextlib.html ), and I couldn't > >>> find a single case where "with (y := f(x))" would be invalid. > >> > >> Consider this custom context manager: > >> > >> @contextmanager > >> def simple_cm(): > >> yield 42 > >> > >> Given that example, the following code: > >> > >> with cm := simple_cm() as value: > >> print(cm.func.__name__, value) > >> > >> would print "'simple_cm 42", since the assignment expression would > >> reference the context manager itself, while the with statement binds > >> the yielded value. > >> > >> Another relevant example would be `contextlib.closing`: that returns > >> the passed in argument from __enter__, *not* self. > >> > >> And that's why earlier versions of PEP 572 (which used the "EXPR as > >> NAME" spelling) just flat out prohibited top level name binding > >> expressions in with statements: "with (expr as name):" and "with expr > >> as name:" were far too different semantically for the only syntactic > >> difference to be a surrounding set of parentheses. > >> > >> Cheers, > >> Nick. > > > > Makes sense. However, couldn't you prevent that by giving with > > priority over the binding ? As in "(with simple_cm) as value", where > > we consider the "as" as binding operator instead of part of the with > > statement ? Sure, you could commit suicide by parenthesis, but by > > default it'd do exactly what the "with simple_cm as value" currently > > does. This does require use of as instead of :=, though. (which was > > the point I was trying to make, apologies for the confusion) > > If you want this to be a generic name-binding operation, then no; most > objects cannot be used as context managers. You'll get an exception if > you try to use "with 1 as x:", for instance. > > As Nick mentioned, there are context managers that return something > other than 'self', and for those, "with expr as name:" has an > important meaning that cannot easily be captured with an assignment > operator. > > ChrisA > > > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > > [View Less]

1 0

Default values in multi-target assignment
by Serhiy Storchaka April 12, 2018

April 12, 2018

Yet one crazy idea. What if allow default values for targets in multi-target assignment? >>> (a, b=0) = (1, 2) >>> a, b (1, 2) >>> (a, b=0) = (1,) >>> a, b (1, 0) >>> (a, b=0) = () Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: not enough values to unpack (expected at least 1, got 0) >>> (a, b=0) = (1, 2, 3) Traceback (most recent … [View More]

3 3

Re: [Python-ideas] Add the method decorator
by Jeroen Demeyer April 12, 2018

April 12, 2018

On 2018-04-12 15:57, Serhiy Storchaka wrote: > There is a difference between functions implemented in Python and C. > Functions implemented in Python are descriptors. They can be used for > defining methods in Python classes. Functions implemented in C are not > descriptors. When set a class attribute to a functions implemented in C, > it will not become a bound method. As it happens, the recently-created PEP 575 allows creating functions in C which behave more like Python … [View More]

1 0

Add the method decorator
by Serhiy Storchaka April 12, 2018

April 12, 2018

There is a difference between functions implemented in Python and C. Functions implemented in Python are descriptors. They can be used for defining methods in Python classes. Functions implemented in C are not descriptors. When set a class attribute to a functions implemented in C, it will not become a bound method. from _noddy import noddy_name class Noddy: name = noddy_name noddy = Noddy() If noddy_name is a Python function, noddy.name() will call noddy_name(… [View More]

2 1

Proposal: flatten multi-nested list/tuple/set and other certain class/type.
by delta114514 April 12, 2018

April 12, 2018

I thought that itertools.chain.from_iterable isn't useful. cuz this only allow "single-nested iterable -- this will raise error when arg has non-nested element--" like below:: >>> from itertools import chain >>> chain.from_iterable([1]) <itertools.chain object at 0x00000112D332BF98> >>> list(_) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'int' object is not iterable and this can't unpack over double nest. … [View More]

1 0

flatten multi-nested list/set/tuple and certain types.
by delta114514 April 12, 2018

April 12, 2018

I thought that itertools.chain.from_iterable isn't useful. cuz this only allow "single-nested iterable -- this will raise error when arg has non-nested element--" like below:: >>> from itertools import chain >>> chain.from_iterable([1]) <itertools.chain object at 0x00000112D332BF98> >>> list(_) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'int' object is not iterable and this can't unpack over double nest. … [View More]

1 0

PEP 572: Statement-Local Name Bindings, take three!
by Chris Angelico April 11, 2018

April 11, 2018

Apologies for letting this languish; life has an annoying habit of getting in the way now and then. Feedback from the previous rounds has been incorporated. From here, the most important concern and question is: Is there any other syntax or related proposal that ought to be mentioned here? If this proposal is rejected, it should be rejected with a full set of alternatives. Text of PEP is below; formatted version will be live shortly (if it isn't already) at: https://www.python.org/dev/peps/… [View More]pep-0572/ ChrisA PEP: 572 Title: Syntax for Statement-Local Name Bindings Author: Chris Angelico <rosuav(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Feb-2018 Python-Version: 3.8 Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018 Abstract ======== Programming is all about reusing code rather than duplicating it. When an expression needs to be used twice in quick succession but never again, it is convenient to assign it to a temporary name with small scope. By permitting name bindings to exist within a single statement only, we make this both convenient and safe against name collisions. Rationale ========= When a subexpression is used multiple times in a list comprehension, there are currently several ways to spell this, none of which is universally accepted as ideal. A statement-local name allows any subexpression to be temporarily captured and then used multiple times. Additionally, this syntax can in places be used to remove the need to write an infinite loop with a ``break`` in it. Capturing part of a ``while`` loop's condition can improve the clarity of the loop header while still making the actual value available within the loop body. Syntax and semantics ==================== In any context where arbitrary Python expressions can be used, a **named expression** can appear. This must be parenthesized for clarity, and is of the form ``(expr as NAME)`` where ``expr`` is any valid Python expression, and ``NAME`` is a simple name. The value of such a named expression is the same as the incorporated expression, with the additional side-effect that NAME is bound to that value for the remainder of the current statement. Just as function-local names shadow global names for the scope of the function, statement-local names shadow other names for that statement. (They can technically also shadow each other, though actually doing this should not be encouraged.) Assignment to statement-local names is ONLY through this syntax. Regular assignment to the same name will remove the statement-local name and affect the name in the surrounding scope (function, class, or module). Statement-local names never appear in locals() or globals(), and cannot be closed over by nested functions. Execution order and its consequences ------------------------------------ Since the statement-local name binding lasts from its point of execution to the end of the current statement, this can potentially cause confusion when the actual order of execution does not match the programmer's expectations. Some examples:: # A simple statement ends at the newline or semicolon. a = (1 as y) print(y) # NameError # The assignment ignores the SLNB - this adds one to 'a' a = (a + 1 as a) # Compound statements usually enclose everything... if (re.match(...) as m): print(m.groups(0)) print(m) # NameError # ... except when function bodies are involved... if (input("> ") as cmd): def run_cmd(): print("Running command", cmd) # NameError # ... but function *headers* are executed immediately if (input("> ") as cmd): def run_cmd(cmd=cmd): # Capture the value in the default arg print("Running command", cmd) # Works Function bodies, in this respect, behave the same way they do in class scope; assigned names are not closed over by method definitions. Defining a function inside a loop already has potentially-confusing consequences, and SLNBs do not materially worsen the existing situation. Differences from regular assignment statements ---------------------------------------------- Using ``(EXPR as NAME)`` is similar to ``NAME = EXPR``, but has a number of important distinctions. * Assignment is a statement; an SLNB is an expression whose value is the same as the object bound to the new name. * SLNBs disappear at the end of their enclosing statement, at which point the name again refers to whatever it previously would have. SLNBs can thus shadow other names without conflict (although deliberately doing so will often be a sign of bad code). * SLNBs cannot be closed over by nested functions, and are completely ignored for this purpose. * SLNBs do not appear in ``locals()`` or ``globals()``. * An SLNB cannot be the target of any form of assignment, including augmented. Attempting to do so will remove the SLNB and assign to the fully-scoped name. In many respects, an SLNB is akin to a local variable in an imaginary nested function, except that the overhead of creating and calling a function is bypassed. As with names bound by ``for`` loops inside list comprehensions, SLNBs cannot "leak" into their surrounding scope. Example usage ============= These list comprehensions are all approximately equivalent:: # Calling the function twice stuff = [[f(x), x/f(x)] for x in range(5)] # External helper function def pair(x, value): return [value, x/value] stuff = [pair(x, f(x)) for x in range(5)] # Inline helper function stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] # Extra 'for' loop - potentially could be optimized internally stuff = [[y, x/y] for x in range(5) for y in [f(x)]] # Iterating over a genexp stuff = [[y, x/y] for x, y in ((x, f(x)) for x in range(5))] # Expanding the comprehension into a loop stuff = [] for x in range(5): y = f(x) stuff.append([y, x/y]) # Wrapping the loop in a generator function def g(): for x in range(5): y = f(x) yield [y, x/y] stuff = list(g()) # Using a statement-local name stuff = [[(f(x) as y), x/y] for x in range(5)] If calling ``f(x)`` is expensive or has side effects, the clean operation of the list comprehension gets muddled. Using a short-duration name binding retains the simplicity; while the extra ``for`` loop does achieve this, it does so at the cost of dividing the expression visually, putting the named part at the end of the comprehension instead of the beginning. Statement-local name bindings can be used in any context, but should be avoided where regular assignment can be used, just as ``lambda`` should be avoided when ``def`` is an option. As the name's scope extends to the full current statement, even a block statement, this can be used to good effect in the header of an ``if`` or ``while`` statement:: # Current Python, not caring about function return value while input("> ") != "quit": print("You entered a command.") # Current Python, capturing return value - four-line loop header while True: command = input("> "); if command == "quit": break print("You entered:", command) # Proposed alternative to the above while (input("> ") as command) != "quit": print("You entered:", command) # See, for instance, Lib/pydoc.py if (re.search(pat, text) as match): print("Found:", match.group(0)) while (sock.read() as data): print("Received data:", data) Particularly with the ``while`` loop, this can remove the need to have an infinite loop, an assignment, and a condition. It also creates a smooth parallel between a loop which simply uses a function call as its condition, and one which uses that as its condition but also uses the actual value. Performance costs ================= The cost of SLNBs must be kept to a minimum, particularly when they are not used; the normal case MUST NOT be measurably penalized. SLNBs are expected to be uncommon, and using many of them in a single function should definitely be discouraged. Thus the current implementation uses a linked list of SLNB cells, with the absence of such a list being the normal case. This list is used for code compilation only; once a function's bytecode has been baked in, execution of that bytecode has no performance cost compared to regular assignment. Other Python implementations may choose to do things differently, but a zero run-time cost is strongly recommended, as is a minimal compile-time cost in the case where no SLNBs are used. Forbidden special cases ======================= In two situations, the use of SLNBs makes no sense, and could be confusing due to the ``as`` keyword already having a different meaning in the same context. 1. Exception catching:: try: ... except (Exception as e1) as e2: ... The expression ``(Exception as e1)`` has the value ``Exception``, and creates an SLNB ``e1 = Exception``. This is generally useless, and creates the potential confusion in that these two statements do quite different things: except (Exception as e1): except Exception as e2: The latter captures the exception **instance**, while the former captures the ``Exception`` **type** (not the type of the raised exception). 2. Context managers:: lock = threading.Lock() with (lock as l) as m: ... This captures the original Lock object as ``l``, and the result of calling its ``__enter__`` method as ``m``. As with ``except`` statements, this creates a situation in which parenthesizing an expression subtly changes its semantics, with the additional pitfall that this will frequently work (when ``x.__enter__()`` returns x, eg with file objects). Both of these are forbidden; creating SLNBs in the headers of these statements will result in a SyntaxError. Alternative proposals ===================== Proposals broadly similar to this one have come up frequently on python-ideas. Below are a number of alternative syntaxes, some of them specific to comprehensions, which have been rejected in favour of the one given above. 1. ``where``, ``let``, ``given``:: stuff = [(y, x/y) where y = f(x) for x in range(5)] stuff = [(y, x/y) let y = f(x) for x in range(5)] stuff = [(y, x/y) given y = f(x) for x in range(5)] This brings the subexpression to a location in between the 'for' loop and the expression. It introduces an additional language keyword, which creates conflicts. Of the three, ``where`` reads the most cleanly, but also has the greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` methods, as does ``tkinter.dnd.Icon`` in the standard library). 2. ``with NAME = EXPR``:: stuff = [(y, x/y) with y = f(x) for x in range(5)] As above, but reusing the `with` keyword. Doesn't read too badly, and needs no additional language keyword. Is restricted to comprehensions, though, and cannot as easily be transformed into "longhand" for-loop syntax. Has the C problem that an equals sign in an expression can now create a name binding, rather than performing a comparison. Would raise the question of why "with NAME = EXPR:" cannot be used as a statement on its own. 3. ``with EXPR as NAME``:: stuff = [(y, x/y) with f(x) as y for x in range(5)] As per option 2, but using ``as`` rather than an equals sign. Aligns syntactically with other uses of ``as`` for name binding, but a simple transformation to for-loop longhand would create drastically different semantics; the meaning of ``with`` inside a comprehension would be completely different from the meaning as a stand-alone statement, while retaining identical syntax. 4. ``EXPR as NAME`` without parentheses:: stuff = [[f(x) as y, x/y] for x in range(5)] Omitting the parentheses from this PEP's proposed syntax introduces many syntactic ambiguities. Requiring them in all contexts leaves open the option to make them optional in specific situations where the syntax is unambiguous (cf generator expressions as sole parameters in function calls), but there is no plausible way to make them optional everywhere. 5. Adorning statement-local names with a leading dot:: stuff = [[(f(x) as .y), x/.y] for x in range(5)] This has the advantage that leaked usage can be readily detected, removing some forms of syntactic ambiguity. However, this would be the only place in Python where a variable's scope is encoded into its name, making refactoring harder. This syntax is quite viable, and could be promoted to become the current recommendation if its advantages are found to outweigh its cost. 6. Allowing ``(EXPR as NAME)`` to assign to any form of name. This is exactly the same as the promoted proposal, save that the name is bound in the same scope that it would otherwise have. Any expression can assign to any name, just as it would if the ``=`` operator had been used. Such variables would leak out of the statement into the enclosing function, subject to the regular behaviour of comprehensions (since they implicitly create a nested function, the name binding would be restricted to the comprehension itself, just as with the names bound by ``for`` loops). 7. Enhancing ``if`` and ``while`` syntax to permit the capture of their conditions:: if re.search(pat, text) as match: print("Found:", match.group(0)) This works beautifully if and ONLY if the desired condition is based on the truthiness of the captured value. It is thus effective for specific use-cases (regex matches, socket reads that return `''` when done), and completely useless in more complicated cases (eg where the condition is ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has no benefit to list comprehensions. Discrepancies in the current implementation =========================================== 1. SLNBs are implemented using a special (and mostly-invisible) name mangling. They may sometimes appear in globals() and/or locals() with their simple or mangled names (but buggily and unreliably). They should be suppressed as though they were guinea pigs. 2. The forbidden special cases do not yet raise SyntaxError. References ========== .. [1] Proof of concept / reference implementation (https://github.com/Rosuav/cpython/tree/statement-local-variables) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

18 64

Add more information in the header of pyc files
by Serhiy Storchaka April 11, 2018

April 11, 2018

The format of the header of pyc files was stable for long time and changed only few times. First time it was changed in 3.3: added the size of the corresponding source mod 2**32. [1] Second time it was changed in 3.7: added the 32-bit flags field and support of hash-based pyc files (PEP 552). [2] [3] I think that it is worth to make more changed. 1. More stable file signature. Currently the magic number is changed in every feature release. Only the third and the forth bytes are stable (… [View More]b'\r\n'), the first bytes are changed non-predicable. The 'py' launcher and third-party software like the 'file' command should support the list of magic numbers for all existing Python releases, and they can't detect pyc file for future versions. There is also a chance the pyc file signature will match the signature of other file type by accident. It would be better if the first 4 bytes of pyc files be same for all Python versions (or at least for all Python versions with the same major number). 2. Include the Python version. Currently the 'py' launcher needs to support the table that maps magic numbers to Python version. It can recognize only Python versions released before building the launcher. If the two major numbers of Python version be included in the version, it would not need such table. 3. The number of compatible subversion. Currently the interpreter supports only a single magic number. If the updated version of the compiler produces more optimal or more correct but compatible bytecode (like ), there is no way to say that the new bytecode is preferable, but the old bytecode can be used too. Changing the magic number causes invalidating all pyc files compiled by the old compiler (see [4] for the example of problems caused by this). The header could contain two magic numbers: the major magic number should be bumped for incompatible changes, the minor magic number should be reset to 0 when the major magic number is bumped, and should be bumped when the compiler become producing different but compatible bytecode. If the import system reads the pyc file with the minor magic number equal or greater than current, it just uses the pyc file. If it reads the pyc file with the minor magic number lesser than current, it can regenerate the pyc file if it is writeable. And the compileall module should regenerate all pyc files with minor magic numbers lesser than current. [1] https://bugs.python.org/issue13645 [2] https://bugs.python.org/issue31650 [3] http://www.python.org/dev/peps/pep-0552/ [4] https://bugs.python.org/issue27286 [View Less]

3 4

Re: [Python-ideas] Python-ideas Digest, Vol 137, Issue 40
by Thautwarm Zhao April 11, 2018

April 11, 2018

I think Guido has given a direct answer why dict unpacking is not supported in syntax level, I can take it and I think it's better to implement a function for dict unpacking in standard library, just like from dict_unpack import dict_unpack, pattern as pat some_dict = {'a': {'b': {'c': 1}, 'd':2}, 'e': 3} extracted = dict_unpack(some_dict, schema = {'a': {'b': {'c': pat('V1')}, 'd': pat('V2')}, 'e': pat('V3')}) # extract to a flatten dictionary v1, v2, v3 = (extracted[k] … [View More]for k in ('V1', 'V2', 'V3')) assert (v1, v2, v3) == (1, 2, 3) As for Steve's confusion, > > {key: value_pattern, **_} = {key: value, **_} > If I saw that, I would have no idea what it could even possibly do. > Let's pick the simplest concrete example I can think of: > > {'A': 1, **{}} = {'A': 0, **{}} > > I cannot interpret what that should do. Is it some sort of > pattern-matching? An update? What is the result? It is obviously some > sort of binding operation, an assignment, but an assignment to what? {'A': 1, **{}} = {'A': 0, **{}} should be just wrong because for any k-v pair at LHS, the key should be a expression and the value is for unpacking. {'A': [*a, b]} = {'A': [1, 2, 3]} is welcome, but {'A': 1} = {'A': '1'} is also something like pattern matching which is out of our topic. Anyway, this feature will not come true, let's forget it... I think Jacco is totally correct in following words. > I think most of these problems could be solved with pop and the > occasional list comprehension like this: > > a, b, c = [{'a':1,'b':2,'c':3}.pop(key) for key in ('a', 'b', 'c')] > > or for your example: > > c = {'a': 1, **{'b': 2}} # I suppose this one would generally > # be dynamic, but I need a name here. > a, b = [c.pop(key) for key in ('a', 'b')] > > would extract all the keys you need, and has the advantage that > you don't need hardcoded dict structure if you expand it to nested > dicts. It's even less writing, and just as extensible to nested dicts. > And if you dont actually want to destruct (tuples and lists aren't > destroyed either), just use __getitem__ access instead of pop. But pop cannot work for a nested case. Feel free to end this topic. thautwarm 2018-04-10 23:20 GMT+08:00 <python-ideas-request(a)python.org>: > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > > Today's Topics: > > 1. Re: Is there any idea about dictionary destructing? > (Steven D'Aprano) > 2. Re: Is there any idea about dictionary destructing? > (Jacco van Dorp) > 3. Re: Start argument for itertools.accumulate() [Was: Proposal: > A Reduce-Map Comprehension and a "last" builtin] (Guido van Rossum) > 4. Re: Is there any idea about dictionary destructing? > (Guido van Rossum) > > > ---------- 已转发邮件 ---------- > From: "Steven D'Aprano" <steve(a)pearwood.info> > To: python-ideas(a)python.org > Cc: > Bcc: > Date: Tue, 10 Apr 2018 19:21:35 +1000 > Subject: Re: [Python-ideas] Is there any idea about dictionary destructing? > On Tue, Apr 10, 2018 at 03:29:08PM +0800, Thautwarm Zhao wrote: > > > I'm focused on the consistency of the language itself. > > Consistency is good, but it is not the only factor to consider. We must > guard against *foolish* consistency: adding features just for the sake > of matching some other, often barely related, feature. Each feature must > justify itself, and consistency with something else is merely one > possible attempt at justification. > > > > {key: value_pattern, **_} = {key: value, **_} > > If I saw that, I would have no idea what it could even possibly do. > Let's pick the simplest concrete example I can think of: > > {'A': 1, **{}} = {'A': 0, **{}} > > I cannot interpret what that should do. Is it some sort of > pattern-matching? An update? What is the result? It is obviously some > sort of binding operation, an assignment, but an assignment to what? > > Sequence binding and unpacking was obvious the first time I saw it. I > had no problem guessing what: > > a, b, c = 1, 2, 3 > > meant, and once I had seen that, it wasn't hard to guess what > > a, b, c = *sequence > > meant. From there it is easy to predict extended unpacking. But I can't > say the same for this. > > I can almost see the point of: > > a, b, c, = **{'a': 1, 'b': 2, 'c': 3} > > but I'm having trouble thinking of a situation where I would actually > use it. But your syntax above just confuses me. > > > > The reason why it's important is that, when destructing/constructing for > > built-in data structures are not supported completely, > > people might ask why "[a, *b] = c" is ok but "{"a": a, **b} = c" not. > > People can ask all sorts of questions. I've seen people ask why Python > doesn't support line numbers and GOTO. We're allowed to answer "Because > it is a bad idea", or even "Because we don't think it is good enough to > justify the cost". > > > > If only multiple assignment is supported, why "(a, (b, c)) = d" could be > > ok? It's exactly destructing! > > That syntax is supported. I don't understand your point here. > > > > >> {'a': a, 'b': b, **c} = {'a': 1, **{'b': 2}} > > SyntaxError: can't assign to literal > > > > Above example could be confusing in some degree, I think. > > I have no idea what you expect it to do. Even something simpler: > > {'a': a} = {'a': 2} > > leaves me in the dark. > > > > > -- > Steve > > > > ---------- 已转发邮件 ---------- > From: Jacco van Dorp <j.van.dorp(a)deonet.nl> > To: python-ideas(a)python.org > Cc: > Bcc: > Date: Tue, 10 Apr 2018 11:52:40 +0200 > Subject: Re: [Python-ideas] Is there any idea about dictionary destructing? > I must say I can't really see the point either. If you say like: > > > {'a': a, 'b': b, **c} = {'a': 1, **{'b': 2}} > > Do you basically mean: > > c = {'a': 1, **{'b': 2}} > a = c.pop("a") > b = c.pop("b") # ? > > That's the only thing I could think of. > > I think most of these problems could be solved with pop and the > occasional list comprehension like this: > > a, b, c = [{'a':1,'b':2,'c':3}.pop(key) for key in ('a', 'b', 'c')] > > or for your example: > > c = {'a': 1, **{'b': 2}} # I suppose this one would generally > # be dynamic, but I need a name here. > a, b = [c.pop(key) for key in ('a', 'b')] > > would extract all the keys you need, and has the advantage that > you don't need hardcoded dict structure if you expand it to nested > dicts. It's even less writing, and just as extensible to nested dicts. > And if you dont actually want to destruct (tuples and lists aren't > destroyed either), just use __getitem__ access instead of pop. > > 2018-04-10 11:21 GMT+02:00 Steven D'Aprano <steve(a)pearwood.info>: > > On Tue, Apr 10, 2018 at 03:29:08PM +0800, Thautwarm Zhao wrote: > > > >> I'm focused on the consistency of the language itself. > > > > Consistency is good, but it is not the only factor to consider. We must > > guard against *foolish* consistency: adding features just for the sake > > of matching some other, often barely related, feature. Each feature must > > justify itself, and consistency with something else is merely one > > possible attempt at justification. > > > > > >> {key: value_pattern, **_} = {key: value, **_} > > > > If I saw that, I would have no idea what it could even possibly do. > > Let's pick the simplest concrete example I can think of: > > > > {'A': 1, **{}} = {'A': 0, **{}} > > > > I cannot interpret what that should do. Is it some sort of > > pattern-matching? An update? What is the result? It is obviously some > > sort of binding operation, an assignment, but an assignment to what? > > > > Sequence binding and unpacking was obvious the first time I saw it. I > > had no problem guessing what: > > > > a, b, c = 1, 2, 3 > > > > meant, and once I had seen that, it wasn't hard to guess what > > > > a, b, c = *sequence > > > > meant. From there it is easy to predict extended unpacking. But I can't > > say the same for this. > > > > I can almost see the point of: > > > > a, b, c, = **{'a': 1, 'b': 2, 'c': 3} > > > > but I'm having trouble thinking of a situation where I would actually > > use it. But your syntax above just confuses me. > > > > > >> The reason why it's important is that, when destructing/constructing for > >> built-in data structures are not supported completely, > >> people might ask why "[a, *b] = c" is ok but "{"a": a, **b} = c" not. > > > > People can ask all sorts of questions. I've seen people ask why Python > > doesn't support line numbers and GOTO. We're allowed to answer "Because > > it is a bad idea", or even "Because we don't think it is good enough to > > justify the cost". > > > > > >> If only multiple assignment is supported, why "(a, (b, c)) = d" could be > >> ok? It's exactly destructing! > > > > That syntax is supported. I don't understand your point here. > > > > > >> >> {'a': a, 'b': b, **c} = {'a': 1, **{'b': 2}} > >> SyntaxError: can't assign to literal > >> > >> Above example could be confusing in some degree, I think. > > > > I have no idea what you expect it to do. Even something simpler: > > > > {'a': a} = {'a': 2} > > > > leaves me in the dark. > > > > > > > > > > -- > > Steve > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas(a)python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > ---------- 已转发邮件 ---------- > From: Guido van Rossum <guido(a)python.org> > To: "Stephen J. Turnbull" <turnbull.stephen.fw(a)u.tsukuba.ac.jp> > Cc: Tim Peters <tim.peters(a)gmail.com>, Python-Ideas < > python-ideas(a)python.org> > Bcc: > Date: Tue, 10 Apr 2018 08:05:18 -0700 > Subject: Re: [Python-ideas] Start argument for itertools.accumulate() > [Was: Proposal: A Reduce-Map Comprehension and a "last" builtin] > On Mon, Apr 9, 2018 at 11:35 PM, Stephen J. Turnbull < > turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: > >> Tim Peters writes: >> >> > "Sum reduction" and "running-sum accumulation" are primitives in >> > many peoples' brains. >> >> I wonder what Kahneman would say about that. He goes to some length >> to explain that people are quite good (as human abilities go) at >> perceiving averages over sets but terrible at summing the same. Maybe >> they substitute the abstraction of summation for the ability to >> perform the operation? >> > > [OT] How is that human ability tested? I am a visual learner and I would > propose that if you have a set of numbers, you can graph it in different > ways to make it easier to perceive one or the other (or maybe both): > > - to emphasize the average, draw a line graph -- in my mind I draw a line > through the average (getting the trend for free) > - to emphasize the sum, draw a histogram -- in my mind I add up the sizes > of the bars > > -- > --Guido van Rossum (python.org/~guido) > > > ---------- 已转发邮件 ---------- > From: Guido van Rossum <guido(a)python.org> > To: Python-Ideas <python-ideas(a)python.org> > Cc: > Bcc: > Date: Tue, 10 Apr 2018 08:20:11 -0700 > Subject: Re: [Python-ideas] Is there any idea about dictionary destructing? > Here's one argument why sequence unpacking is more important than dict > unpacking. > > Without sequence unpacking, you have a long sequence, to get at a specific > item you'd need to use indexing, where you often end up having to remember > the indices for each type of information. Say you have points of the form > (x, y, z, t), to get at the t coordinate you'd have to write p[3]. With > sequence unpacking you can write > > x, y, z, t = p > > and then you can use the individual variables in the subsequent code. > > However if your point had the form {'x': x, 'y': y, 'z': z, 't': t}, you > could just write p['t']. This is much more mnemonic than p[3]. > > All the rest follows -- after a while extended forms of iterable unpacking > start making sense. But for dicts the use case is just much less common. > > (If you're doing a lot of JSON you might have a different view on that. > You should probably use some kind of schema-guided parser though.) > > -- > --Guido van Rossum (python.org/~guido) > > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > > [View Less]

1 0

Is there any idea about dictionary destructing?
by thautwarm April 10, 2018

April 10, 2018

We know that Python support the destructing of iterable objects. m_iter = (_ for _ in range(10)) a, *b, c = m_iter That's pretty cool! It's really convenient when there're many corner cases to handle with iterable collections. However destructing in Python could be more convenient if we support dictionary destructing. In my opinion, dictionary destructing is not difficult to implement and makes the syntax more expressive. A typical example is data access on nested data structures(just like … [View More]

9 11

Fwd: Is there any idea about dictionary destructing?
by Thautwarm Zhao April 9, 2018

April 9, 2018

I'm sorry that I didn't send a copy of the discussions here. ---------- Forwarded message ---------- From: Thautwarm Zhao <yaoxiansamma(a)gmail.com> Date: 2018-04-09 1:24 GMT+08:00 Subject: Re: [Python-ideas] Is there any idea about dictionary destructing? To: "Eric V. Smith" <eric(a)trueblade.com> Thank you, Eric. Your links really help me and I've investigated it carefully. After reading them, I found the discussion almost focused on a non-nested data structure. The flatten … [View More]key-value pairs might be easily replaced by something like x, y, z = [some_dict[k] for k in ('a', 'b', 'c')] I couldn't agree more but, when it comes to nested, some_dict = { 'a': { 'b': { 'c': V1}, 'e': V2 }, 'f': V3 } I agree that there could be other ways as intuitive as the dict destructing to get `V1, V2, V3` instead, however dict destructing refers to the consistency of Python language behaviours. When I'm writing these codes: [a, *b] = [1, 2, 3] The LHS is actually equals to RHS, and if we implement a way to apply this on dictionary {'a': a, 'b': b, '@': c, **other} = {'a': 1, 'b': 2, '@': 3, '*': 4} It also presents that LHS equals to RHS. Dict destructing/constructing is totally compatible to Python unpack/pack, just as what iterable destructing/constructing does. It's neat when we talk about Python's data structures we can talk about the consistency, readability and the expression of intuition. In the real world, the following one could really help when it comes to the field of data storage. some_dict = {'a': [1, 2, 3, {"d": 4, "f": 5}]} {'a': [b, *c, {"d": e, **_}]} = some_dict The LHS doesn't only show the structure of some variable intuitively(this makes review easier, too), but also supplies a way to access data in fewer codes. In the previous talk people have shown multiple usages of dict destructing in real world: - Django Rest Framework validate - Load config files and use them, specifically Yaml/JSON data access. In fact, any other operation on dictionary than simply getting a value from a key might needs dict destructing, just when the task is complicated enough. I do think the usages are general enough now to make us allow similar syntax to do above tasks. P.S: Some other advices in the previous talk like the following: 'a' as x, 'b' as y, 'c' as z = some_dict 'a': x, 'b': y, 'c': z = some_dict mode, height, width = **prefs Either of them conflicts against the current syntax, or does mismatch the consistency of Python language(LHS != RHS). thautwarm 2018-04-08 5:39 GMT+08:00 Eric V. Smith <eric(a)trueblade.com>: > There was a long thread last year on a subject, titled "Dictionary > destructing and unpacking.": > https://mail.python.org/pipermail/python-ideas/2017-June/045963.html > > You might want to read through it and see what ideas and problems were > raised then. > > In that discussion, there's also a link to an older pattern matching > thread: > https://mail.python.org/pipermail/python-ideas/2015-April/032907.html > > Eric > > On 4/7/2018 1:26 PM, thautwarm wrote: > >> We know that Python support the destructing of iterable objects. >> >> m_iter= (_for _in range(10)) >> a,*b, c= m_iter >> >> That's pretty cool! It's really convenient when there're many corner >> cases to handle with iterable collections. >> However destructing in Python could be more convenient if we support >> dictionary destructing. >> >> In my opinion, dictionary destructing is not difficult to implement and >> makes the syntax more expressive. A typical example is data access on >> nested data structures(just like JSON), destructing a dictionary makes the >> logic quite clear: >> >> data= { >> "direct": "some data", >> "nested": { >> "lst_data": [1,2,3], >> "int_data": 1 >> } >> } >> { >> "direct": direct, >> "nested": { >> "lst_data": [a, b, c], >> } >> }= data >> >> >> Dictionary destructing might not be very well-known but it really helps. >> The operations on nested key-value collections are very frequent, and the >> codes for business logic are not readable enough until now. Moreover Python >> is now popular in data processing which must be enhanced by the entire >> support of data destructing. >> >> Here are some implementations of other languages: >> Elixir, which is also a popular dynamic language nowadays. >> >> |iex> %{} = %{:a => 1, 2 => :b} %{2 => :b, :a => 1} iex> %{:a => a} = >> %{:a => 1, 2 => :b} %{2 => :b, :a => 1} iex> a 1 iex> %{:c => c} = %{:a => >> 1, 2 => :b} ** (MatchError) no match of right hand side value: %{2 => :b, >> :a => 1}| >> >> And in F#, there is something similar to dictionary destructing(actually, >> this destructs `struct` instead) >> type MyRecord = { Name: string; ID: int } letIsMatchByName record1 (name: >> string) = matchrecord1 with| { MyRecord.Name = nameFound; MyRecord.ID = _; >> } whennameFound = name -> true| _ -> falseletrecordX = { Name = "Parker"; >> ID = 10} letisMatched1 = IsMatchByName recordX "Parker"letisMatched2 = >> IsMatchByName recordX "Hartono" >> >> All of them partially destructs(or matches) a dictionary. >> >> thautwarm >> >> >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas(a)python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> [View Less]

1 0

PEP draft: Unifying function/method classes
by Jeroen Demeyer April 3, 2018

April 3, 2018

I have prepared a PEP draft for unifying function/method classes. You can find it at https://github.com/jdemeyer/PEP-functions This has not officially been submitted as PEP yet, I want to hear your comments first. Thanks, Jeroen.

7 6

Re: [Python-ideas] PEP draft: Unifying function/method classes
by Jeroen Demeyer April 2, 2018

April 2, 2018

On 2018-04-02 12:39, INADA Naoki wrote: > Thanks for writing such hard PEP. > > At first glance, it new type hierarchy seems OK. > But I can't understand rational for new flags. Which flags in particular do you mean? I just pushed an update explaining the rationale of METH_ARG0_FUNCTION: https://github.com/jdemeyer/PEP-functions#replacing-tp_call-meth_arg0_funct… > And it's very difficult to estimate runtime and maintenance cost of > the PEP, without draft implementation. … [View More]

2 1

Generalized version of contextlib.close
by Roberto Martínez April 2, 2018

April 2, 2018

Hi, sometimes I need to use contextlib.close but over methods with a different name, for example stop(), halt(), etc. For those cases I have to write my own contextlib.close specialized version with a hard-coded method name. I think adding a "method" argument to contextlib.close can be very useful: @contextmanager def closing(thing, method="close"): try: yield thing finally: getattr(thing, method)() Or maybe something even more generic: @contextmanager def calling(… [View More]

4 3

Re: [Python-ideas] PEP draft: Unifying function/method classes
by Jeroen Demeyer April 1, 2018

April 1, 2018

On 2018-03-31 18:09, Steven D'Aprano wrote: > It seems like a huge amount of work What is a huge amount of work? Writing the PEP? Implementing the PEP? Using the PEP? Adapting existing Python code to the PEP? > Why isn't the answer to provide a hook to support introspection? That is a lot easier said than done. How would you do that? The problem is that multiple classes are involved, at least "builtin_function_or_method" and "method_descriptor". So you'll end up with at least 3 … [View More]

3 2