Mailman 3 May 2018 - Python-ideas

Make keywords KEYwords only in places they would have syntactical meaning
by Ken Hilton May 19, 2018

May 19, 2018

Hi all, Yes, this is another idea for avoiding breaking existing code when introducing new keywords. I'm not sure if this is too similar to Guido's previous "allow keywords in certain places" idea, but here goes: Only treat keywords as having any special meaning when they are in places with syntactical significance. So, currently, let's say someone set the variable "and_" to some value. The following lines are both SyntaxErrors: True and_ False obj.and = value And the following are both correct: True and False obj.and_ = value My idea is to only treat keywords as having special meaning when they're in the right place. So the following would all be legal: >>> from operator import and >>> var = and(True, False) >>> var False >>> var = True and False >>> var False >>> def except(exc, def): ... try: ... return def() ... except exc as e: ... return e ... >>> except(ZeroDivisionError, lambda: 1/0) ZeroDivisionError('division by zero',) >>> except(ZeroDivisionError, lambda: 0/1) 0.0 >>> import asyncio as await #this is already currently legal, but will not be in the __future__ >>> async def async(def): ... return await await.get_event_loop().run_in_executor(None, def) ... >>> And so on. What are your thoughts? Sharing, Ken Hilton ;

7 7

Escaped braces in format specifiers
by Ken Kundert May 19, 2018

May 19, 2018

The syntax for formatted string literals is given here: https://docs.python.org/3/reference/lexical_analysis.html#f-strings If you were to examine this carefully, you would see that a format_spec (the part within the braces but after the colon) can be empty or it can consist of literal characters and replacement fields. For example: f"result: {value:{width}.{precision}}" In this case '{width}.{precision}' is the format_spec, it consists of two replacement fields, {width} and {precision}, and one literal character, '.'. The definition of literal character includes all characters except braces. Presumably excluding braces makes it easier to distinguish the replacement fields. However, replacement fields are also used in the f_string itself, but in that case escaped braces are explicitly added to the list of valid symbols. I think the fact that one cannot include braces in the format_spec is an unnecessary and undesirable restriction. As an example of where it would be useful to allow braces in the format_spec, consider this simple example: >>> class mydict(dict): ... def __format__(self, template): ... return ''.join( ... template.format(v, k=k, v=v) for k, v in self.items() ... ) >>> accounts = mydict(checking=4256.78, savings=12000, brokerage=24685.5) >>> print('Accounts:\n {0:{{k:>9s}}: ${{v:>9,.2f}}\n }'.format(accounts)) Accounts: checking: $ 4,256.78 savings: $12,000.00 brokerage: $24,685.50 In this example, the format_spec is itself treated as a format string, which allows a recursive specification of the string format. If you try this example in Python3.6 you will find that it works, but it should not because the format_spec contains escaped braces. However, the fact that it works appears be be a happy accident. Things break if the escaped braces are not balanced (thanks to Thomas Jollans for finding this). Furthermore, things break in a different manner when an f-string is used. For example: >>> print(f'Accounts:\n {accounts:{{k:>9s}}: ${{v:>9,.2f}}\n }') File "<fstring>", line 1 ({k:>9s}) ^ SyntaxError: invalid syntax or: >>> print(f'Accounts:\n {accounts:{{k}}: ${{v}}\n }') File "tryit", line 12, in <module> print(f'Accounts:\n {accounts:{{k}}: ${{v}}\n }') NameError: name 'k' is not defined Oddly, the f-string still raises the NameError even if both k and v are explicitly defined. At a minimum it would be good to improve the error messages that are produced when escaped braces are included in the format_spec. Most of the error messages that are given, if they are given at all, are misleading. None are as simple as: SyntaxError: escaped braces are not allowed in a format spec. But rather than improve the error messages, I think we should consider simply allowing escaped braces in the format_spec. Doing so enables the neat idea of recursive format strings. But even if you don't like that idea, it would be nice to remove this rather rather odd restriction and would make the behavior of f-strings and the format method more consistent. -Ken

3 6

Re: [Python-ideas] String and bytes bitwise operations
by Ken Hilton May 19, 2018

May 19, 2018

On Fri, 18 May 2018 13:49:58 -0300, Facundo Batista wrote: > Once you think as the whole sequence of bytes as a sequence of bits (eight times longer, of course), all questions are easly answered, see below... I had never considered this before, but it seems very interesting. Essentially that would make bytes1 ^ bytes2 be equivalent to (int.from_bytes(bytes1, sys.byteorder) ^ int.from_bytes(bytes2, sys.byteorder)).to_bytes(max(len(bytes1), len(bytes2)), sys.byteorder) rather than bytes(a ^ b for a, b in zip(bytes1, bytes2)) I like that idea more than my original elementwise idea. Thinking, Ken Hilton;

1 0

Verbatim names (allowing keywords as names)
by Steven D'Aprano May 19, 2018

May 19, 2018

Inspired by Alex Brault's post: https://mail.python.org/pipermail/python-ideas/2018-May/050750.html I'd like to suggest we copy C#'s idea of verbatim identifiers, but using a backslash rather than @ sign: \name would allow "name" to be used as an identifier, even if it clashes with a keyword. It would *not* allow the use of characters that aren't valid in identifiers, e.g. this is out: \na!me # still not legal See usage #1 here: https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/tokens/ve… If "verbatim name" is too long, we could call them "raw names", by analogy with raw strings. I believe that \ is currently illegal in any Python expression, except inside strings and at the very end of the line, so this ought to be syntactically unambgiguous. We should still include a (mild?) recommendation against using keywords unless necessary, and a (strong?) preference for the trailing underscore convention. But I think this doesn't look too bad: of = 'output.txt' \if = 'input.txt' with open(\if, 'r'): with open(of, 'w'): of.write(\if.read()) maybe even nicer than if_. Some examples: result = \except + 1 result = something.\except result = \except.\finally -- Steve

24 58

Re: [Python-ideas] Async Lambda syntax
by Noah Simon May 19, 2018

May 19, 2018

Hello all, I was developing a script using an asyncio-based API, when I came across the need to define an asynchronous lambda. I found this syntax does not currently exist. Obviously I could have (and did) just write a regular coroutine, but for simple one-line functions and such, I think an asynchronous lambda syntax would be useful. I do not have the experience to write a PEP or implement this, so I was wondering what you all think of the idea. What I was thinking: foo = async lambda a, b: b + await bar(a I posted an issue <https://bugs.python.org/issue33447> on the issue tracker suggesting this, and was directed to this mailing list. Thanks, Noah

3 2

High Precision datetime
by Ed Page May 18, 2018

May 18, 2018

Greetings, Is there interest in a PEP for extending time, datetime / timedelta for arbitrary or extended precision fractional seconds? My company designs and manufactures scientific hardware that typically operate with nanoseconds -- sometimes even attoseconds -- levels of precision. We’re in the process of providing Python APIs for some of these products and need to expose the full accuracy of the data to our customers. Doing so would allow developers to do things like timestamp analog measurements for correlating with other events in their system, or precisely schedule a future time event for correctly interoperating with other high-speed devices. The API we’ve been toying with is adding two new fields to time, datetime and timedelta - frac_seconds (int) - frac_seconds_exponent (int or new SITimeUnit enum) time.microseconds would be turned into a property that wraps frac_seconds for compatibility Challenges - Defining the new `max` or `resolution` - strftime / strptime. I propose that we do nothing, just leave formatting / parsing to use `microseconds` at best. On the other hand, __str__ could just specify the fractional seconds using scientific or engineering notation. Alternatives - My company create our own datetime library - Continued fracturing of time ... ecosystem (datetime, arrow, pendulum, delorean, datetime64, pandas.Timestamp – all of which offer varying degrees of compatibility) - Add an `attosecond` field and have `microsecond` wrap this. - Effectively same except hard code `frac_seconds_exponent` to lowest value - The most common cases (milliseconds, microseconds) will always pay the cost of using a bigint as compared to the proposal which is a "pay for what you use" approach - How do we define what is "good enough" precision? - Continue to subdivide time by adding `nanosecond` that is "nanoseconds since last micosecond", `picosecond` that is "picoseconds since last micnanosecond", and `attosecond` field that is "attoseconds since last picosecond" - Possibly surprising API; people might expect `picosecond` to be an offset since last second - Messy base 10 / base 2 conversions - Have `frac_seconds` be a float - This has precision issues. If anyone wants to have an impromptu BoF on the subject, I'm available at PyCon. Thanks Ed Page

14 31

Re: [Python-ideas] Verbatim names (allowing keywords as names)
by Neil Girdhar May 17, 2018

May 17, 2018

On Thursday, May 17, 2018 at 5:40:05 PM UTC-4, Carl Smith wrote: > > > My preference is to do nothing. If you end up making "where" a keyword > in Python 3.8, numpy will probably: > > * rename their where function to "where_" in 3.8 > > * add a where_ alias in Python < 3.8. > > This assumes that every library that defines something named `where` (and > every library that references it) > are maintained. If the parser just starts treating `where` as a keyword, > working code will not parse. That > may be acceptable (I don't know what the policy is), but it makes > introducing keywords more costly than > an approach that allows old code to continue working. > Fair enough. If that solution is taken, I want the linters to complain about verbatim uses as a stopgap measure that ultimately should be removed. I don't think interfaces should be naming functions and arguments: "if", "while", etc. I think PEP 08 should discourage its use as well except when it's necessary to continue to use an unmaintained interface. > -- Carl Smith > carl....(a)gmail.com <javascript:> > > On 17 May 2018 at 19:41, Neil Girdhar <miste...(a)gmail.com <javascript:>> > wrote: > >> My preference is to do nothing. If you end up making "where" a keyword >> in Python 3.8, numpy will probably: >> * rename their where function to "where_" in 3.8 >> * add a where_ alias in Python < 3.8. >> >> And then people will have to fix their code in 3.8 anyway. Only instead >> of learning a new verbatim syntax, they will just add the familiar >> underscore. >> >> One thing that should ideally be done is to improve the SyntaxError >> processing to special case use of keywords in places that identifiers are >> used. Instead of: >> >> In [1]: for if in range(10): >> File "<ipython-input-1-a51677fa6668>", line 1 >> for if in range(10): >> ^ >> SyntaxError: invalid syntax >> >> How about >> >> In [1]: for if in range(10): >> File "<ipython-input-1-a51677fa6668>", line 1 >> for if in range(10): >> ^ >> SyntaxError: "if" was used where a variable belongs, but "if" is a >> keyword. Consider using "if_" instead. >> >> Similarly, >> >> In [2]: int.if >> File "<ipython-input-2-72291900e846>", line 1 >> int.if >> ^ >> SyntaxError: "if" was used where an attribute name belongs. Did you mean >> "if_"? >> >> SyntaxError doesn't need to quickly generate its error strings. So there >> is only an upside to having clear error messages. >> >> On Thursday, May 17, 2018 at 1:54:42 PM UTC-4, Stephan Houben wrote: >>> >>> Fortunately we have Unicode bold characters nowadays >>> >>> 𝐢𝐟 if 𝐢𝐧 in: >>> 𝐫𝐞𝐭𝐮𝐫𝐧 return >>> >>> Look ma! No syntactic ambiguity! >>> >>> That's hilarious :) >> >>> Stephan >>> >>> 2018-05-17 19:10 GMT+02:00 Chris Barker via Python-ideas < >>> python...(a)python.org>: >>> >>>> On Wed, May 16, 2018 at 2:09 PM, Carl Smith <carl....(a)gmail.com> wrote: >>>> >>>>> If your position is that Guido shouldn't introduce keywords that are >>>>> currently used as names at all, >>>>> >>>> >>>> Exactly -- which is why I'm wondering my no one (that I've seen -- long >>>> thread) is presenting the backwards option: >>>> >>>> Any new keywords introduced will be non-legal as regular names. >>>> >>>> \new_key_word >>>> >>>> for instance. >>>> >>>> Makes me think that it may have been good to have ALL keywords somehow >>>> non-legal as user-defined names -- maybe ugly syntax, but it would make a >>>> clear distinction. >>>> >>>> how ugly would this be? >>>> >>>> \for i in range(n): >>>> \while \True: >>>> ... >>>> >>>> pretty ugly :-( >>>> >>>> But maybe not so much if only a handful of new ones.... >>>> >>>> Or is there another currently illegal character that could be used that >>>> would be less ugly? >>>> >>>> I'm actually confused as to what the point is to the \ prefix idea for >>>> names: >>>> >>>> * It would still require people to change their code when a new keyword >>>> was introduced >>>> >>>> * It would be no easier / harder than adding a conventional legal >>>> character -- trailing underscore, or ??? >>>> >>>> * but now the changed code would no longer run on older versions of >>>> python. >>>> >>>> I guess it comes down to why you'd want to call out: >>>> >>>> "this is a name that is almost like a keyword" >>>> >>>> Seems like a meh, meh, lose proposal to me. >>>> >>>> OK, I see one advantage -- one could have code that already has BOTH >>>> word and word_ names in it. So when word becomes a keyword, a tool that >>>> automatically added an underscore would break the code. whereas if it >>>> automatically added an currently illegal character, it wouldn't shadow >>>> anything. >>>> >>>> But a sufficiently smart tool could get around that, too. >>>> >>>> -CHB >>>> >>>> >>>> -- >>>> >>>> Christopher Barker, Ph.D. >>>> Oceanographer >>>> >>>> Emergency Response Division >>>> NOAA/NOS/OR&R (206) 526-6959 voice >>>> 7600 Sand Point Way NE (206) 526-6329 fax >>>> Seattle, WA 98115 (206) 526-6317 main reception >>>> >>>> Chris....(a)noaa.gov >>>> >>>> _______________________________________________ >>>> Python-ideas mailing list >>>> Python...(a)python.org >>>> https://mail.python.org/mailman/listinfo/python-ideas >>>> Code of Conduct: http://python.org/psf/codeofconduct/ >>>> >>>> >>> >> _______________________________________________ >> Python-ideas mailing list >> Python...(a)python.org <javascript:> >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> >> >

1 0

Syntactic Sugar: Post-Conditions for Guard Clauses
by Manuel Barkhau May 17, 2018

May 17, 2018

Hello, has there been consideration for implementing the following new syntax: def my_fun(elements): results = [] for elem in elements: ... continue if not is_valid(elem) ... results.append(result) break if len(results) == max_results return results Or similarly: def iter_fun(elements): num_results = 0 for elem in elements: ... continue if not is_valid(elem) ... yield result num_results += 1 return if num_results == max_results When there is an expression involved for the case of a `return` or `raise` statement, I don't think it's such a great style, because the conditional gets hidden off to the right. def myfun(val): return default_result if val not in known_vals ... return result Alternatively, is there a good way I can implement this as a preprocessor for myself? Thanks Manuel Barkhau

2 1

Re: [Python-ideas] String and bytes bitwise operations
by Ken Hilton May 17, 2018

May 17, 2018

On Thu, 17 May 2018 23:13:22 +1000, Steven D'Aprano wrote: > No, he didn't explain the meaning. He gave an example, but not a reason why it should do what he showed. > > Why should the *abstract character* 'H' XORed with the abstract character 'w' return the abstract character '?'? Why shouldn't the result be '>' instead? My initial thought was that 'H' ^ 'w' -> '?' because when I was experimenting with the idea, ord('H') ^ ord('w') -> ord('?'). However, I do see your point that different encodings give different results, so I'll drop the idea of bitwise operations on strings. > XORing code points could easily generate invalid Unicode sequences containing lone surrogates, say, or undefined characters. Or as you point out, out of range values. Invalid Unicode sequences, lone surrogates, and undefined characters, IMO, are simply consequences of misusing the operators. I hadn't anticipated the ValueError for '\U00100000' and '\U00010000', though, which is another reason for me to drop bitwise operations on strings. > But XORing bytes seems perfectly reasonable. Bytes are numbers, even if we display them as ASCII characters. My thought exactly. On Thu, 17 May 2018 22:20:43 +1000, Steven D'Aprano wrote: > What if the strings are unequal lengths? (out-of-order quote lol) Then the operators would raise a ValueError. (Assuming bytestrings, since again, I'm dropping text strings.) Sharing ideas , Ken Hilton ;

1 0

Inline assignments using "given" clauses
by Nick Coghlan May 17, 2018

May 17, 2018

(Note: Guido's already told me off-list that he doesn't like the way this spelling reads, but I wanted to share it anyway since it addresses one of the recurring requests in the PEP 572 discussions for a more targeted proposal that focused specifically on the use cases that folks had agreed were reasonable potential use cases for inline assignment expressions. I'll also note that another potential concern with this specific proposal is that even though "given" wasn't used as a term in any easily discovered Python APIs back when I first wrote PEP 3150, it's now part of the Hypothesis testing API, so adopting it as a keyword now would be markedly more disruptive than it might have been historically) Recapping the use cases where the inline assignment capability received the most agreement regarding being potentially more readable than the status quo: 1. Making an "exactly one branch is executed" construct clearer than is the case for nested if statements: if m := pattern.search(data): ... elif m := other_pattern.search(data): ... else: ... 2. Replacing a loop-and-a-half construct: while m := pattern.search(remaining_data): ... 3. Sharing values between filtering clauses and result expressions in comprehensions: result = [(x, y, x/y) for x in data if (y := f(x))] The essence of the given clause concept would be to modify *these specific cases* (at least initially) to allow the condition expression to be followed by an inline assignment, of the form "given TARGET = EXPR". (Note: being able to implement such a syntactic constraint is a general consequence of using a ternary notation rather than a binary one, since it allows the construct to start with an arbitrary expression, without requiring that expression to be both the result of the operation *and* the value bound to a name - it isn't unique to the "given" keyword specifically) While the leading keyword would allow TARGET to be an arbitrary assignment target without much chance for confusion, it could also be restricted to simple names instead (as has been done for PEP 572. With that spelling, the three examples above would become: # Exactly one branch is executed here if m given m = pattern.search(data): ... elif m given m = other_pattern.search(data)): ... else: ... # This name is rebound on each trip around the loop while m given m = pattern.search(remaining_data): ... # "f(x)" is only evaluated once on each iteration result = [(x, y, x/y) for x in data if y given y = f(x)] Constraining the syntax that way (at least initially) would avoid poking into any dark corners of Python's current scoping and expression execution ordering semantics, while still leaving the door open to later making "result given NAME = expr" a general purpose ternary operator that returns the LHS, while binding the RHS to the given name as a side effect. Using a new keyword (rather than a symbol) would make the new construct easier to identify and search for, but also comes with all the downsides of introducing a new keyword. (Hence the not-entirely-uncommon suggestion of using "with" for a purpose along these lines, which runs into a different set of problems related to trying to use "with" for two distinct and entirely unrelated purposes). Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

36 165