Mailman 3 April 2018 - Python-ideas

PEP 572: Assignment Expressions (post #4)
by Chris Angelico April 17, 2018

April 17, 2018

Wholesale changes since the previous version. Statement-local name bindings have been dropped (I'm still keeping the idea in the back of my head; this PEP wasn't the first time I'd raised the concept), and we're now focusing primarily on assignment expressions, but also with consequent changes to comprehensions. Sorry for the lengthy delays; getting a reference implementation going took me longer than I expected or intended. ChrisA PEP: 572 Title: Assignment Expressions Author: Chris Angelico <rosuav(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 28-Feb-2018 Python-Version: 3.8 Post-History: 28-Feb-2018, 02-Mar-2018, 23-Mar-2018, 04-Apr-2018 Abstract ======== This is a proposal for creating a way to assign to names within an expression. Additionally, the precise scope of comprehensions is adjusted, to maintain consistency and follow expectations. Rationale ========= Naming the result of an expression is an important part of programming, allowing a descriptive name to be used in place of a longer expression, and permitting reuse. Currently, this feature is available only in statement form, making it unavailable in list comprehensions and other expression contexts. Merely introducing a way to assign as an expression would create bizarre edge cases around comprehensions, though, and to avoid the worst of the confusions, we change the definition of comprehensions, causing some edge cases to be interpreted differently, but maintaining the existing behaviour in the majority of situations. Syntax and semantics ==================== In any context where arbitrary Python expressions can be used, a **named expression** can appear. This can be parenthesized for clarity, and is of the form ``(target := expr)`` where ``expr`` is any valid Python expression, and ``target`` is any valid assignment target. The value of such a named expression is the same as the incorporated expression, with the additional side-effect that the target is assigned that value. # Similar to the boolean 'or' but checking for None specifically x = "default" if (eggs := spam().ham) is None else eggs # Even complex expressions can be built up piece by piece y = ((eggs := spam()), (cheese := eggs.method()), cheese[eggs]) Differences from regular assignment statements ---------------------------------------------- An assignment statement can assign to multiple targets:: x = y = z = 0 To do the same with assignment expressions, they must be parenthesized:: assert 0 == (x := (y := (z := 0))) Augmented assignment is not supported in expression form:: >>> x +:= 1 File "<stdin>", line 1 x +:= 1 ^ SyntaxError: invalid syntax Otherwise, the semantics of assignment are unchanged by this proposal. Alterations to comprehensions ----------------------------- The current behaviour of list/set/dict comprehensions and generator expressions has some edge cases that would behave strangely if an assignment expression were to be used. Therefore the proposed semantics are changed, removing the current edge cases, and instead altering their behaviour *only* in a class scope. As of Python 3.7, the outermost iterable of any comprehension is evaluated in the surrounding context, and then passed as an argument to the implicit function that evaluates the comprehension. Under this proposal, the entire body of the comprehension is evaluated in its implicit function. Names not assigned to within the comprehension are located in the surrounding scopes, as with normal lookups. As one special case, a comprehension at class scope will **eagerly bind** any name which is already defined in the class scope. A list comprehension can be unrolled into an equivalent function. With Python 3.7 semantics:: numbers = [x + y for x in range(3) for y in range(4)] # Is approximately equivalent to def <listcomp>(iterator): result = [] for x in iterator: for y in range(4): result.append(x + y) return result numbers = <listcomp>(iter(range(3))) Under the new semantics, this would instead be equivalent to:: def <listcomp>(): result = [] for x in range(3): for y in range(4): result.append(x + y) return result numbers = <listcomp>() When a class scope is involved, a naive transformation into a function would prevent name lookups (as the function would behave like a method). class X: names = ["Fred", "Barney", "Joe"] prefix = "> " prefixed_names = [prefix + name for name in names] With Python 3.7 semantics, this will evaluate the outermost iterable at class scope, which will succeed; but it will evaluate everything else in a function:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def <listcomp>(iterator): result = [] for name in iterator: result.append(prefix + name) return result prefixed_names = <listcomp>(iter(names)) The name ``prefix`` is thus searched for at global scope, ignoring the class name. Under the proposed semantics, this name will be eagerly bound, being approximately equivalent to:: class X: names = ["Fred", "Barney", "Joe"] prefix = "> " def <listcomp>(prefix=prefix): result = [] for name in names: result.append(prefix + name) return result prefixed_names = <listcomp>() With list comprehensions, this is unlikely to cause any confusion. With generator expressions, this has the potential to affect behaviour, as the eager binding means that the name could be rebound between the creation of the genexp and the first call to ``next()``. It is, however, more closely aligned to normal expectations. The effect is ONLY seen with names that are looked up from class scope; global names (eg ``range()``) will still be late-bound as usual. One consequence of this change is that certain bugs in genexps will not be detected until the first call to ``next()``, where today they would be caught upon creation of the generator. TODO: Discuss the merits and costs of amelioration proposals. Recommended use-cases ===================== Simplifying list comprehensions ------------------------------- These list comprehensions are all approximately equivalent:: # Calling the function twice stuff = [[f(x), x/f(x)] for x in range(5)] # External helper function def pair(x, value): return [value, x/value] stuff = [pair(x, f(x)) for x in range(5)] # Inline helper function stuff = [(lambda y: [y,x/y])(f(x)) for x in range(5)] # Extra 'for' loop - potentially could be optimized internally stuff = [[y, x/y] for x in range(5) for y in [f(x)]] # Iterating over a genexp stuff = [[y, x/y] for x, y in ((x, f(x)) for x in range(5))] # Expanding the comprehension into a loop stuff = [] for x in range(5): y = f(x) stuff.append([y, x/y]) # Wrapping the loop in a generator function def g(): for x in range(5): y = f(x) yield [y, x/y] stuff = list(g()) # Using a mutable cache object (various forms possible) c = {} stuff = [[c.update(y=f(x)) or c['y'], x/c['y']] for x in range(5)] # Using a temporary name stuff = [[y := f(x), x/y] for x in range(5)] If calling ``f(x)`` is expensive or has side effects, the clean operation of the list comprehension gets muddled. Using a short-duration name binding retains the simplicity; while the extra ``for`` loop does achieve this, it does so at the cost of dividing the expression visually, putting the named part at the end of the comprehension instead of the beginning. Capturing condition values -------------------------- Assignment expressions can be used to good effect in the header of an ``if`` or ``while`` statement:: # Current Python, not caring about function return value while input("> ") != "quit": print("You entered a command.") # Current Python, capturing return value - four-line loop header while True: command = input("> "); if command == "quit": break print("You entered:", command) # Proposed alternative to the above while (command := input("> ")) != "quit": print("You entered:", command) # Capturing regular expression match objects # See, for instance, Lib/pydoc.py, which uses a multiline spelling # of this effect if match := re.search(pat, text): print("Found:", match.group(0)) # Reading socket data until an empty string is returned while data := sock.read(): print("Received data:", data) Particularly with the ``while`` loop, this can remove the need to have an infinite loop, an assignment, and a condition. It also creates a smooth parallel between a loop which simply uses a function call as its condition, and one which uses that as its condition but also uses the actual value. Rejected alternative proposals ============================== Proposals broadly similar to this one have come up frequently on python-ideas. Below are a number of alternative syntaxes, some of them specific to comprehensions, which have been rejected in favour of the one given above. Alternative spellings --------------------- Broadly the same semantics as the current proposal, but spelled differently. 1. ``EXPR as NAME``, with or without parentheses:: stuff = [[f(x) as y, x/y] for x in range(5)] Omitting the parentheses in this form of the proposal introduces many syntactic ambiguities. Requiring them in all contexts leaves open the option to make them optional in specific situations where the syntax is unambiguous (cf generator expressions as sole parameters in function calls), but there is no plausible way to make them optional everywhere. With the parentheses, this becomes a viable option, with its own tradeoffs in syntactic ambiguity. Since ``EXPR as NAME`` already has meaning in ``except`` and ``with`` statements (with different semantics), this would create unnecessary confusion or require special-casing. 2. Adorning statement-local names with a leading dot:: stuff = [[(f(x) as .y), x/.y] for x in range(5)] # with "as" stuff = [[(.y := f(x)), x/.y] for x in range(5)] # with ":=" This has the advantage that leaked usage can be readily detected, removing some forms of syntactic ambiguity. However, this would be the only place in Python where a variable's scope is encoded into its name, making refactoring harder. This syntax is quite viable, and could be promoted to become the current recommendation if its advantages are found to outweigh its cost. 3. Adding a ``where:`` to any statement to create local name bindings:: value = x**2 + 2*x where: x = spam(1, 4, 7, q) Execution order is inverted (the indented body is performed first, followed by the "header"). This requires a new keyword, unless an existing keyword is repurposed (most likely ``with:``). See PEP 3150 for prior discussion on this subject (with the proposed keyword being ``given:``). Special-casing conditional statements ------------------------------------- One of the most popular use-cases is ``if`` and ``while`` statements. Instead of a more general solution, this proposal enhances the syntax of these two statements to add a means of capturing the compared value:: if re.search(pat, text) as match: print("Found:", match.group(0)) This works beautifully if and ONLY if the desired condition is based on the truthiness of the captured value. It is thus effective for specific use-cases (regex matches, socket reads that return `''` when done), and completely useless in more complicated cases (eg where the condition is ``f(x) < 0`` and you want to capture the value of ``f(x)``). It also has no benefit to list comprehensions. Advantages: No syntactic ambiguities. Disadvantages: Answers only a fraction of possible use-cases, even in ``if``/``while`` statements. Special-casing comprehensions ----------------------------- Another common use-case is comprehensions (list/set/dict, and genexps). As above, proposals have been made for comprehension-specific solutions. 1. ``where``, ``let``, or ``given``:: stuff = [(y, x/y) where y = f(x) for x in range(5)] stuff = [(y, x/y) let y = f(x) for x in range(5)] stuff = [(y, x/y) given y = f(x) for x in range(5)] This brings the subexpression to a location in between the 'for' loop and the expression. It introduces an additional language keyword, which creates conflicts. Of the three, ``where`` reads the most cleanly, but also has the greatest potential for conflict (eg SQLAlchemy and numpy have ``where`` methods, as does ``tkinter.dnd.Icon`` in the standard library). 2. ``with NAME = EXPR``:: stuff = [(y, x/y) with y = f(x) for x in range(5)] As above, but reusing the `with` keyword. Doesn't read too badly, and needs no additional language keyword. Is restricted to comprehensions, though, and cannot as easily be transformed into "longhand" for-loop syntax. Has the C problem that an equals sign in an expression can now create a name binding, rather than performing a comparison. Would raise the question of why "with NAME = EXPR:" cannot be used as a statement on its own. 3. ``with EXPR as NAME``:: stuff = [(y, x/y) with f(x) as y for x in range(5)] As per option 2, but using ``as`` rather than an equals sign. Aligns syntactically with other uses of ``as`` for name binding, but a simple transformation to for-loop longhand would create drastically different semantics; the meaning of ``with`` inside a comprehension would be completely different from the meaning as a stand-alone statement, while retaining identical syntax. Regardless of the spelling chosen, this introduces a stark difference between comprehensions and the equivalent unrolled long-hand form of the loop. It is no longer possible to unwrap the loop into statement form without reworking any name bindings. The only keyword that can be repurposed to this task is ``with``, thus giving it sneakily different semantics in a comprehension than in a statement; alternatively, a new keyword is needed, with all the costs therein. Migration path ============== The semantic changes to list/set/dict comprehensions, and more so to generator expressions, may potentially require migration of code. In many cases, the changes simply make legal what used to raise an exception, but there are some edge cases that were previously legal and are not, and a few corner cases with altered semantics. Yield inside comprehensions --------------------------- As of Python 3.7, the outermost iterable in a comprehension is permitted to contain a 'yield' expression. If this is required, the iterable (or at least the yield) must be explicitly elevated from the comprehension:: # Python 3.7 def g(): return [x for x in [(yield 1)]] # With PEP 572 def g(): sent_item = (yield 1) return [x for x in [sent_item]] This more clearly shows that it is g(), not the comprehension, which is able to yield values (and is thus a generator function). The entire comprehension is consistently in a single scope. Name lookups in class scope --------------------------- A comprehension inside a class previously was able to 'see' class members ONLY from the outermost iterable. Other name lookups would ignore the class and potentially locate a name at an outer scope:: pattern = "<%d>" class X: pattern = "[%d]" numbers = [pattern % n for n in range(5)] In Python 3.7, ``X.numbers`` would show angle brackets; with PEP 572, it would show square brackets. Maintaining the current behaviour here is best done by using distinct names for the different forms of ``pattern``, as would be the case with functions. Generator expression bugs can be caught later --------------------------------------------- Certain types of bugs in genexps were previously caught more quickly. Some are now detected only at first iteration:: gen = (x for x in rage(10)) # NameError gen = (x for x in 10) # TypeError (not iterable) gen = (x for x in range(1/0)) # Exception raised during evaluation This brings such generator expressions in line with a simple translation to function form:: def <genexp>(): for x in rage(10): yield x gen = <genexp>() # No exception yet tng = next(gen) # NameError To detect these errors more quickly, ... TODO. Open questions ============== Can the outermost iterable still be evaluated early? ---------------------------------------------------- As of Python 3.7, the outermost iterable in a genexp is evaluated early, and the result passed to the implicit function as an argument. With PEP 572, this would no longer be the case. Can we still, somehow, evaluate it before moving on? One possible implementation would be:: gen = (x for x in rage(10)) # translates to def <genexp>(): iterable = iter(rage(10)) yield None for x in iterable: yield x gen = <genexp>() next(gen) This would pump the iterable up to just before the loop starts, evaluating exactly as much as is evaluated outside the generator function in Py3.7. This would result in it being possible to call ``gen.send()`` immediately, unlike with most generators, and may incur unnecessary overhead in the common case where the iterable is pumped immediately (perhaps as part of a larger expression). Frequently Raised Objections ============================ Why not just turn existing assignment into an expression? --------------------------------------------------------- C and its derivatives define the ``=`` operator as an expression, rather than a statement as is Python's way. This allows assignments in more contexts, including contexts where comparisons are more common. The syntactic similarity between ``if (x == y)`` and ``if (x = y)`` belies their drastically different semantics. Thus this proposal uses ``:=`` to clarify the distinction. This could be used to create ugly code! --------------------------------------- So can anything else. This is a tool, and it is up to the programmer to use it where it makes sense, and not use it where superior constructs can be used. With assignment expressions, why bother with assignment statements? ------------------------------------------------------------------- The two forms have different flexibilities. The ``:=`` operator can be used inside a larger expression; the ``=`` operator can be chained more conveniently, and closely parallels the inline operations ``+=`` and friends. The assignment statement is a clear declaration of intent: this value is to be assigned to this target, and that's it. Acknowledgements ================ The author wishes to thank Guido van Rossum and Nick Coghlan for their considerable contributions to this proposal, and to members of the core-mentorship mailing list for assistance with implementation. References ========== .. [1] Proof of concept / reference implementation (https://github.com/Rosuav/cpython/tree/assignment-expressions) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

24 119

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 17, 2018

April 17, 2018

> Hey! I did not propose "?". Read the explanation in parenthesis. > My whole idea that any option could be viable, as long as > it does not propose reversed order notation. Dear Mikhail, so sorry about my misunderstanding, I should read the post with fewer presuppositions... Back to the topic, this thread seems to be closed now, and in my opinion `:=` could be synthetically the best. 2018-04-17 15:11 GMT+08:00 Mikhail V <mikhailwas(a)gmail.com>: > On Tue, Apr 17, 2018 at 6:09 AM, Thautwarm Zhao <yaoxiansamma(a)gmail.com> > wrote: > > > > > > 3) "target ? expr" (where ? is some other word/character - IIRC > > > "target from expr" was proposed once) > > > > A more popular convention is to mark `?` as handling boolean variables, > so > > `target ? expr` could mean `expr if target else target`. Other proposal > for > > null/boolean checking might need `?`, let's preserve `?` character for > > further development. > > Hey! I did not propose "?". Read the explanation in parenthesis. > My whole idea that any option could be viable, as long as > it does not propose reversed order notation. > > But anyway ":=" is better than any keyword imo. > > > Mikhail >

1 0

Providing a public API for creating and parsing HTTP messages
by Derek Maciel April 17, 2018

April 17, 2018

Hello all, If this is not the appropriate place for this type of proposal please let me know. The modules http.client and http.server both do a wonderful job when implementing HTTP clients and servers, respectively. However, occasionally there may be a need to create and parse HTTP messages themselves without needing to implement a client or server. While http.client and http.server clearly have solved this problem for its own uses, there is little to no public API exposed to a user wishing to do the same. For instance, the documentation for http.client.HTTPMessage [1] simply states that it extends from email.message.Message in order to parse the messages. [1]: https://docs.python.org/3/library/http.client.html#httpmessage-objects While this need may admittedly be a niche one, it is a shame that one cannot simply use the functionality already implemented for http.client and http.server for this purpose. Therefore, I am proposing this module be improved such that the creation and parsing of HTTP messages be "pulled out of" the existing code into HTTPRequest and HTTPResponse classes with a public API for this purpose. The exact API is subject to change of course, but it could work like: >>> r = http.client.HTTPRequest("POST", "/foo", "Hello world!") >>> r.method 'POST' >>> r.body 'Hello world!' >>> print(r) POST /foo HTTP/1.1 Content-Length: 12 Hello world! If this idea is not completely a bad one, I am very interested in discussing in further detail how I (or perhaps someone else) could begin to implement these changes. Thank you for reading.

2 1

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 17, 2018

April 17, 2018

> We have ways of cheating a bit if we want to reinterpret the semantics > of something that nevertheless parses cleanly - while the parser is > limited to single token lookahead, it's straightforward for the > subsequent code generation stage to look a single level down in the > parse tree and see that the code that parsed as "with expr" is > actually "with subexpr as target". It does work, however I think it does sound like a patch, and definitely it will block us to make other extensions in the future. > 3) "target ? expr" (where ? is some other word/character - IIRC > "target from expr" was proposed once) A more popular convention is to mark `?` as handling boolean variables, so `target ? expr` could mean `expr if target else target`. Other proposal for null/boolean checking might need `?`, let's preserve `?` character for further development. > How about "name being expression" - this avoids the already used "as" > while being searchable, reasonably short and gives a reasonably clear, > (at least to English speakers), indication of what is going on. It can > also be typed on an ASCII keyboard without having to have a helper > program or memorising Unicode codes and can be displayed or printed > without having to install specialised fonts. It makes sense, if we don't have a long history in Python programming... A new keyword would be something very dangerous, because it just causes the crash of some existed library using the keyword as identifier.

3 2

Idea: Importing from arbitrary filenames
by Ken Hilton April 16, 2018

April 16, 2018

Hi all, First of all, please excuse me if I'm presenting this idea in the wrong way or at the wrong time - I'm new to this mailing list and haven't seen anyone propose a new idea on it yet, so I don't know the customs. I have an idea for importing files with arbitrary names. Currently, the "official" way to import arbitrary files is to use the "imp" module, as shown by this answer: https://stackoverflow.com/a/3137914/6605349 However, this method takes two function calls and is not as (aesthetically pleasing? is that the word?) as a simple "import" statement. Therefore, my idea is to allow the "import" statement to accept one of three targets. First, the normal "import": import antigravity which simply imports from sys.path. Second, importing with a string literal specifying the path to a file: import '/home/pi/anti-gravity.py' *as antigravity* Note the "as antigravity" in this statement - this is to avoid ambiguities when choosing the global name to bind to. Should "import '/home/pi/anti-gravity.py'" import to the name "/home/pi/anti-gravity.py", "anti-gravity.py", "anti-gravity", or "anti_gravity"? None of those are really ideal. Therefore, when the import target is a string literal, the statement must include "as NAME". Third, importing with an expression providing a value castable to a string, specifying the path to a file: def file_in_home(filename): return '/home/pi/' + filename import *$*file_in_home('anti-gravity.py') *as antigravity* Once again, for the same reasons, import statements like this must include "as NAME" to avoid ambiguities. Notice that the expression is preceded by a dollar sign ($) to indicate that what follows is an expression rather than a name - imagine a scenario like this: antigravity_file = '/home/pi/anti-gravity.py' import antigravity_file as antigravity Should it look for a sys.path module with the name "antigravity_file" or should it use the value of the variable "antigravity_file"? Looking for the sys.path module first before trying a variable's value would waste processing time and potentially be unexpected behavior. Trying a variable's value first before looking for a sys.path module would be even less expected behavior. Therefore, a dollar sign must come before expression imports to indicate that the import target is an expression. Side note: the dollar sign was chosen because it mimics other languages' conventions of preceding variable names with dollar signs, but any arbitrary character not present at the start of an expression would work. One more thing about expression imports: if the final returned value of the expression is not a string, I believe the statement should raise a TypeError (the same way that __repr__ or __str__ raise TypeError if they return a non-string). Why? If the statement attempted to cast the return value to a string, and the return value's __str__ method raised an error, then should the statement allow the error to pass through, or should it attempt to use a parent class's __str__ method? Allowing the error to pass through would almost certainly be unexpected behavior; attempting to use a parent class's __str__ method would take more time and more processing power (though it would eventually reach "object"'s __str__ method and succeed). Therefore, non-string expression values should raise TypeError. What are your thoughts? Regards , Ken Hilton ;

8 12

Move optional data out of pyc files
by Serhiy Storchaka April 16, 2018

April 16, 2018

Currently pyc files contain data that is useful mostly for developing and is not needed in most normal cases in stable program. There is even an option that allows to exclude a part of this information from pyc files. It is expected that this saves memory, startup time, and disk space (or the time of loading from network). I propose to move this data from pyc files into separate file or files. pyc files should contain only external references to external files. If the corresponding external file is absent or specific option suppresses them, references are replaced with None or NULL at import time, otherwise they are loaded from external files. 1. Docstrings. They are needed mainly for developing. 2. Line numbers (lnotab). They are helpful for formatting tracebacks, for tracing, and debugging with the debugger. Sources are helpful in such cases too. If the program doesn't contain errors ;-) and is sipped without sources, they could be removed. 3. Annotations. They are used mainly by third party tools that statically analyze sources. They are rarely used at runtime. Docstrings will be read from the corresponding docstring file unless -OO is supplied. This will allow also to localize docstrings. Depending on locale or other settings different docstring file can be used. For suppressing line numbers and annotations new options can be added.

22 35

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 16, 2018

April 16, 2018

> To me, "from" strongly suggests that an element is being obtained from a container/collection of > elements. This is how I conceptualize "from module import name": "name" refers to an object > INSIDE the module, not the module itself. If I saw > > if (match from pattern.search(data)) is not None: ... > I would guess that it is equivalent to > > m = next(pattern.search(data)) > if m is not None: ... +1, although unpacking seems to be reasonable `[elem1, *elems] from contains`. Now we have - "expr as name" - "name := expr" - "expr -> name" - "name from expr" Personally I prefer "as", but I think without a big change of python Grammar file, it's impossible to avoid parsing "with expr as name" into "with (expr as name)" because "expr as name" is actually an "expr". I have mentioned this in previous discussions and it seems it's better to warn you all again. I don't think people of Python-Dev are willing to implement a totally new Python compiler.

2 1

Rewriting file - pythonic way
by Alexey Shrub April 16, 2018

April 16, 2018

Hi all, I am new in python (i am moving from Perl world), but I always love Python for hight level, beatuful and clean syntax. Now I have question/idea about working with files. On mine opinion it very popular use case: 1. Open file (for read and write) 2. Read data from file 3. Modify data. 4. Rewrite file by modified data. But now it is looks not so pythonic: with open(filename, 'r+') as file: data = file.read() data = data.replace('old', 'new') file.seek(0) file.write(data) file.truncate() or something like this with open(filename) as file: data = file.read() data = data.replace('old', 'new') with open(filename) as file: file.write(data) I think best way is something like this with open(filename, 'r+') as file: data = file.read() data = data.replace('old', 'new') file.rewrite(data) but for this io.BufferedIOBase must contain rewrite method what you think about this?

8 18

Accepting multiple mappings as positional arguments to create dicts
by Andrés Delfino April 15, 2018

April 15, 2018

Hi! I thought that maybe dict could accept several mappings as positional arguments, like this: class Dict4(dict): > def __init__(self, *args, **kwargs): > if len(args) > 1: > if not all([isinstance(arg, dict) for arg in args]): > raise TypeError('Dict4 expected instances of dict since > multiple positional arguments were passed') > > temp = args[0].copy() > > for arg in args[1:]: > temp.update(arg) > > super().__init__(temp, **kwargs) > else: > super().__init__(*args, **kwargs) > AFAIK, this wouldn't create compatibility problems, since you can't pass two positional arguments now anyways. It would be useful to solve the "sum/union dicts" discussion, for example: requests.get(url, params=dict(params, {'foo': bar}) Whar are your thoughts?

9 21

Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 (was post #4)
by Thautwarm Zhao April 15, 2018

April 15, 2018

Dear Steve, I'm sorry to annoy you by my proposal, but I do think using unicode might be wise in current stage. \triangleq could be print with unicode number \u225c, and adding plugins to support typing this in editors could be easy, just simply map \xxx to the specific unicode char when we press the tab after typing it. People using Julia language are proud of it but I think it's just something convenient could be used in any other language. There are other reasons to support unicode but it's out of this topic. Although ':=' and '->' are not perfect, in the range of ASCII it seems to be impossible to find a better one. <python-ideas-request(a)python.org> 于 2018年4月16日周一上午12:53写道： > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > Today's Topics: > > 1. Re: Rewriting file - pythonic way (Mahmoud Hashemi) > 2. Re: Start argument for itertools.accumulate() [Was: Proposal: > A Reduce-Map Comprehension and a "last" builtin] (Tim Peters) > 3. Re: Spelling of Assignment Expressions PEP 572 (was post #4) > (Steven D'Aprano) > 4. Re: Rewriting file - pythonic way (Oleg Broytman) > 5. Re: A cute Python implementation of itertools.tee (Tim Peters) > > > > ---------- Forwarded message ---------- > From: Mahmoud Hashemi <mahmoud(a)hatnote.com> > To: python-ideas <python-ideas(a)python.org> > Cc: > Bcc: > Date: Sun, 15 Apr 2018 09:10:57 -0700 > Subject: Re: [Python-ideas] Rewriting file - pythonic way > Depending on how firm your requirements around locking are, you may find > this code useful: > https://github.com/mahmoud/boltons/blob/6b0721b6aeda6d3ec6f5d31be7c741bc7fc… > > (docs here: > http://boltons.readthedocs.io/en/latest/fileutils.html#atomic-file-saving > ) > > Basically every operating system has _some_ way of doing an atomic file > replacement, letting us guarantee that a file at a given location is always > valid. atomic_save provides a unified interface to that cross-platform > behavior. > > The code does not do locking, as neither I nor its other users have wanted > it, but I'd be happy to extend it if there's a sensible default. > > On Sun, Apr 15, 2018 at 8:19 AM, Oleg Broytman <phd(a)phdru.name> wrote: > >> On Sun, Apr 15, 2018 at 05:15:55PM +0300, Alexey Shrub <ashrub(a)yandex.ru> >> wrote: >> > В Воскресенье, 15 апр. 2018 в 2:40 , Nick Coghlan <ncoghlan(a)gmail.com> >> > написал: >> > > https://bugs.python.org/issue8604#msg174104 is the relevant tracker >> > > discussion >> > >> > Thanks all, I agree that universal and absolutly safe solution is very >> > difficult, but for experiment I made some draft >> > https://github.com/worldmind/scripts/tree/master/filerewrite >> >> Good! >> >> > main code here >> > >> https://github.com/worldmind/scripts/blob/master/filerewrite/filerewrite.py… >> >> Can I recommend to catch exceptions in `backuper.backup()`, >> cleanup backuper and unlock locker? >> >> Oleg. >> -- >> Oleg Broytman http://phdru.name/ >> phd(a)phdru.name >> Programmers don't die, they just GOSUB without RETURN. >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas(a)python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > > > ---------- Forwarded message ---------- > From: Tim Peters <tim.peters(a)gmail.com> > To: Raymond Hettinger <raymond.hettinger(a)gmail.com> > Cc: Python-Ideas <python-ideas(a)python.org> > Bcc: > Date: Sun, 15 Apr 2018 11:15:18 -0500 > Subject: Re: [Python-ideas] Start argument for itertools.accumulate() > [Was: Proposal: A Reduce-Map Comprehension and a "last" builtin] > [Raymond Hettinger <raymond.hettinger(a)gmail.com>] > > Q. Do other languages do it? > > A. Numpy, no. R, no. APL, no. Mathematica, no. Haskell, yes. > > > > * > http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.accumulate.… > > * > https://stat.ethz.ch/R-manual/R-devel/library/base/html/cumsum.html > > * http://microapl.com/apl/apl_concepts_chapter5.html > > \+ 1 2 3 4 5 > > 1 3 6 10 15 > > * https://reference.wolfram.com/language/ref/Accumulate.html > > * https://www.haskell.org/hoogle/?hoogle=mapAccumL > > There's also C++, which is pretty much "yes" to every variation > discussed so far: > > * partial_sum() is like Python's current accumulate(), including > defaulting to doing addition. > > http://en.cppreference.com/w/cpp/algorithm/partial_sum > > * inclusive_scan() is also like accumulate(), but allows an optional > "init" argument (which is returned if specified), and there's no > guarantee of "left-to-right" evaluation (it's intended for associative > binary functions, and wants to allow parallelism in the > implementation). > > http://en.cppreference.com/w/cpp/algorithm/inclusive_scan > > * exclusive_scan() is like inclusive_scan(), but _requires_ an "init" > argument (which is not returned). > > http://en.cppreference.com/w/cpp/algorithm/exclusive_scan > > * accumulate() is like Python's functools.reduce(), but the operation > is optional and defaults to addition, and an "init" argument is > required. > > http://en.cppreference.com/w/cpp/algorithm/accumulate > > > > > ---------- Forwarded message ---------- > From: "Steven D'Aprano" <steve(a)pearwood.info> > To: python-ideas(a)python.org > Cc: > Bcc: > Date: Mon, 16 Apr 2018 02:15:59 +1000 > Subject: Re: [Python-ideas] Spelling of Assignment Expressions PEP 572 > (was post #4) > On Sun, Apr 15, 2018 at 11:11:37PM +0800, Thautwarm Zhao wrote: > > > I think maybe we can use unicode characters like ≜ (\triangleq) and add > the > > support of unicode completion to python repl. The unicode completion of > > editors or ides has been quite mature. > > What key combination do I need to type to get ≜ in the following editors > please? I tried typing \triangleq but all I got was \triangleq. > > Notepad (Windows) > Brackets (Mac) > BBEdit (Mac) > kwrite (Linux) > kate > nano > geany > gedit > > as well as IDLE, my mail client (kmail, Thunderbird or mutt), my web > browsers (Firefox, Opera and Chromium), the interactive interpreter in > various different consoles, my Usenet client (Pan and KNode) and IRC > (pidgin). > > Oh, having it work in LibreOffice and GoogleApps too would be nice, > although not essential since I don't often write code in them. > > And what decent fonts do I need to install for ≜ to show up as something > other than a square box ("missing glyph")? > > > -- > Steve > > > > > ---------- Forwarded message ---------- > From: Oleg Broytman <phd(a)phdru.name> > To: python-ideas(a)python.org > Cc: > Bcc: > Date: Sun, 15 Apr 2018 18:49:16 +0200 > Subject: Re: [Python-ideas] Rewriting file - pythonic way > On Sun, Apr 15, 2018 at 09:10:57AM -0700, Mahmoud Hashemi < > mahmoud(a)hatnote.com> wrote: > > Depending on how firm your requirements around locking are, you may find > > this code useful: > > > https://github.com/mahmoud/boltons/blob/6b0721b6aeda6d3ec6f5d31be7c741bc7fc… > > > > (docs here: > > > http://boltons.readthedocs.io/en/latest/fileutils.html#atomic-file-saving > ) > > > > Basically every operating system has _some_ way of doing an atomic file > > replacement, letting us guarantee that a file at a given location is > always > > valid. atomic_save provides a unified interface to that cross-platform > > behavior. > > > > The code does not do locking, as neither I nor its other users have > wanted > > it, but I'd be happy to extend it if there's a sensible default. > > I don't like it renames the file at the end. Renaming could lead to > changed file ownership and permissions; restoring permissions is not > always possible, restoring ownership is almost never possible. Renaming > is also not always possible due to restricted directory permissions. > > > On Sun, Apr 15, 2018 at 8:19 AM, Oleg Broytman <phd(a)phdru.name> wrote: > > > > > On Sun, Apr 15, 2018 at 05:15:55PM +0300, Alexey Shrub < > ashrub(a)yandex.ru> > > > wrote: > > > > В Воскресенье, 15 апр. 2018 в 2:40 , Nick Coghlan < > ncoghlan(a)gmail.com> > > > > написал: > > > > > https://bugs.python.org/issue8604#msg174104 is the relevant > tracker > > > > > discussion > > > > > > > > Thanks all, I agree that universal and absolutly safe solution is > very > > > > difficult, but for experiment I made some draft > > > > https://github.com/worldmind/scripts/tree/master/filerewrite > > > > > > Good! > > > > > > > main code here > > > > https://github.com/worldmind/scripts/blob/master/ > > > filerewrite/filerewrite.py#L46 > > > > > > Can I recommend to catch exceptions in `backuper.backup()`, > > > cleanup backuper and unlock locker? > > > Oleg. > -- > Oleg Broytman http://phdru.name/ phd(a)phdru.name > Programmers don't die, they just GOSUB without RETURN. > > > > > ---------- Forwarded message ---------- > From: Tim Peters <tim.peters(a)gmail.com> > To: Antoine Pitrou <solipsis(a)pitrou.net> > Cc: Python-Ideas <python-ideas(a)python.org> > Bcc: > Date: Sun, 15 Apr 2018 11:52:49 -0500 > Subject: Re: [Python-ideas] A cute Python implementation of itertools.tee > [Antoine Pitrou <solipsis(a)pitrou.net>] > > This implementation doesn't work with Python 3.7 or 3.8. > > I've tried it here: > > https://gist.github.com/pitrou/b3991f638300edb6d06b5be23a4c66d6 > > > > and get: > > Traceback (most recent call last): > > File "mytee.py", line 14, in gen > > mylast = last[1] = last = [next(it), None] > > StopIteration > > > > The above exception was the direct cause of the following exception: > > > > Traceback (most recent call last): > > File "mytee.py", line 47, in <module> > > run(mytee1) > > File "mytee.py", line 36, in run > > lists[i].append(next(iters[i])) > > RuntimeError: generator raised StopIteration > > > > (Yuck!) > > Thanks for trying! I wonder whether that will break other code. I > wrote PEP 255, and this part was intentional at the time: > > """ > If an unhandled exception-- including, but not limited to, > StopIteration --is raised by, OR PASSES THROUGH [emphasis added], a > generator function, then the exception is passed on to the caller in > the usual way, and subsequent attempts to resume the generator > function raise StopIteration. > """ > > I've exploited that a number of times. > > > > In short, you want the following instead: > > > > try: > > mylast = last[1] = last = [next(it), None] > > except StopIteration: > > return > > No, I don't ;-) If I have to catch StopIteration myself now, then I > want the entire "white True:" loop in the "try" block. Setting up > try/except machinery anew on each iteration would add significant > overhead; doing it just once per derived generator wouldn't. > > > >> def mytee(xs, n): > >> last = [None, None] > >> > >> def gen(it, mylast): > >> nonlocal last > >> while True: > >> mylast = mylast[1] > >> if not mylast: > >> mylast = last[1] = last = [next(it), None] > > > That's smart and obscure :-o > > The way it works is that the `last` assignment changes the `last` value > > seen by all derived generators, while the `last[1]` assignment updates > > the bindings made in the other generators' `mylast` lists... It's > > difficult to find the words to explain it. > > Which is why I didn't even try - I did warn people that if they > thought it "was obvious", they hadn't yet thought hard enough ;-) > Good job! > > > > The chained assignment makes it more difficult to parse as well (when I > > read this I don't know if `last[i]` or `last` gets assigned first; > > apparently the answer is `last[i]`, otherwise the recipe wouldn't work > > correctly). > > Ya, I had to look it up too :-) Although, like almost everything else > in Python, chained assignments proceed "left to right". I was just > trying to make it as short as possible, to increase the "huh - can > something that tiny really work?!" healthy skepticism factor :-) > > > > Perhaps like this: > > > > while True: > > mylast = mylast[1] > > if not mylast: > > try: > > # Create new list link > > mylast = [next(it), None] > > except StopIteration: > > return > > else: > > # Append to other generators `mylast` linked lists > > last[1] = mylast > > # Update shared list link > > last = last[1] > > yield mylast[0] > > I certainly agree that's easier to follow. But that wasn't really the > point ;-) > > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas >

2 1