Mailman 3 December 2019 - Python-ideas

Alternative to `enumerate` and `range(len(sequence))`: indexes() and entries()
by Marco Sulla 29 Dec '19

29 Dec '19

It's very common to see: ``` for i, x in enumerate(sequence): [...] ``` and also to see ``` for i in range(len(sequence)): [...] ``` I propose to introduce for sequences the methods `indexes()` and `entries()`. They are similar to `keys()` and `items()` for maps. I changed the names for not doing a breaking change. This way the two codes will became simply: ``` for i, x in sequence.entries(): [...] for i in sequence.indexes(): [...] ``` sequence.indexes() could return: 1. a SequenceIndexesView, similar to the views of `dict` 2. a simple range() object The same for entries(). It should be: 1. a SequenceEntriesView 2. a generator that yields the tuples (index, value) What do you think about?

5 6

__eq__ and __ne__
by Siddharth Prajosh 27 Dec '19

27 Dec '19

Why do we need separate functions for == and != ? Isn't this supposed to be negation of each other?

3 3

PEP 584 (dict merge operators), dict.update and dict.gapfill
by Jonathan Fine 27 Dec '19

27 Dec '19

Summary: dict.update(self, other) corresponds to 'merge-right'. Perhaps add dict.gapfill(self, other), which corresponds to 'merge-left'. This post defines dict.gapfill, and discusses update and gapfill in the context of PEP 584. PEP 584 suggests adding a merge operator (to be denoted by '|' or '+') to dictionaries, and a corresponding augmented assignment operator (to be denoted by '|=' or '+='). The main purpose of this post is to remind ourselves that there are both merge-left and merge-right operators, and to begin to discuss the consequences. Perhaps Off Topic: There is already a road sign for 'merge left'. See the following URL for a discussion of usual 'W4-2' sign for this operation, and how it can cause confusion. And how a "small but critical" change to the sign reduced confusion, in a backwards compatible way. https://99percentinvisible.org/article/lane-ends-merge-left-redesigning-w4-… By the way, that URL writes: "The solution is elegant, simple and additive — it requires no fundamental reformatting, building instead on the existing sign. It is thus also recognizable to those familiar with its predecessor and visually similar to older signs that can still be found on the road." To me, it seems that the author has goals similar to the Python design goals. See also http://www.trafficsign.us/pdf/warn/w4-2mod.pdf, which uses 'Left Lane Ends' instead of 'Merge Right' (and similarly 'Right Lane Ends' for 'Merge Left'). Now Back On Topic. The dict.update(self, other) method gives priority to other. (In other words, it's 'merge-right'.) In March this year, it led me to realise that perhaps dict would benefit from there being a 'merge-left' operator, which gives priority to the left. Here's an example of how merge-left (which I call dict.gapfill) could be coded. (I've done it in a way that emphasises similarities and differences between dict.update and dict.gapfill.) class mydict(dict): def update(self, other): for key, value in other.items(): if True: self[key] = value def gapfill(self, other): for key, value in other.items(): if key not in self: self.key = value def copy(self): return mydict(super().copy()) Here's some simple examples of the semantics. >>> d1 = mydict(); d1[1] = 'int one' >>> d2 = mydict(); d2[1.0] = 'float one' >>> c1 = d1.copy(); c1.update(d2) >>> c2 = d1.copy(); c2.gapfill(d2) >>> c1 {1: 'float one'} >>> c2 {1: 'int one'} For comparison with sets (similar to dict keys), we also have >>> {1} | {1.0} {1} >>> {1.0} | {1} {1.0} Now for some consequences. First, it's my opinion that dict.gapfill(self, other) will be useful when we're given say some command line values, and we wish to augment this with default values. Second, it's my opinion that dict.gapfill(self, other) is in some cases a useful alternative for a dict merge operator (whether it uses '|' or '+' as its symbol). For evidence see https://github.com/jpadilla/pyjwt/blob/master/jwt/utils.py#L71-L81 Third, for sets the '|' operator is a merge-left operator. In other words it gives priority to the set-element (or key) in the first set, rather than the second. (This happens only when the two keys so-to-say are the same value represented by different types. In our case the types are (int, float).) Fourth, as the set '|' operator is merge-left, and Python's >>> value = A or B or C gives priority to the first (left-most) true value, confusion might result if the dict '|' operator were to be merge-right. Regarding the fourth point, it is possible that purity and consistency points to the dict '|' operator being merge-left, while usefulness and habit points to the dict '|' operator being merge-right. Fifth and finally. In our context, it is my opinion that merge-left and merge-right don't clearly communicate the differing semantics. However, in my opinion dict.update(self, other) and dict.gapfill(self, other) do clearly communicate their semantics. Here's a link to my post to this list in March this year. https://mail.python.org/archives/list/python-ideas@python.org/message/GRFYX… with best regards Jonathan

5 4

Re: Argumenting in favor of first()
by Steven D'Aprano 27 Dec '19

27 Dec '19

On Fri, Dec 06, 2019 at 09:11:44AM -0400, Juancarlo Añez wrote: [...] > > Sure, but in this case, it isn't a fragment of a larger function, and > > that's not what it looks like. If it looked like what you wrote, I would > > understand it. But it doesn't, so I didn't really understand what it was > > supposed to do, until I read the equivalent version using first/next. > > > > Exactly my point. Indeed, and I agree with that. But I still don't see what advantage there is to having a `first` builtin which does so little. It's a really thin wrapper around `next` that: calls iter() on its iterable argument supplies a default and then calls next() with two arguments I guess my question is asking you to justify adding a builtin rather than just educating people how to use next effectively. This is how I would implement the function in Python: def first(iterable, default=None): return next(iter(iterable), default) But there's a major difference in behaviour depending on your input, and one which is surely going to lead to bugs from people who didn't realise that iterator arguments and iterable arguments will behave differently: # non-iterator iterable py> obj = [1, 2, 3, 4] py> [first(obj) for __ in range(5)] [1, 1, 1, 1, 1] # iterator py> obj = iter([1, 2, 3, 4]) py> [first(obj) for __ in range(5)] [1, 2, 3, 4, None] We could document the difference in behaviour, but it will still bite people and surprise them. We could, I guess, eliminate the difference by adding the ability to peek ahead to the next value of an arbitrary iterator without consuming that value. This would have to be done by the interpreter, not in Python code, and it would open new complexities. Consider a generator which yields values which depend, in some complex and unpredicatable way, on *when* it is called. Say, the amount of disk space available, or the number of records in a database, or the current time. If I peek into the generator, I would see the time at the moment I peeked: now = get_current_time_generator() peek(now) # returns 11:25:30am Since peek can't literally see into the future, it cannot be otherwise. But what happens when I call next? time.sleep(60) next(now) # what will this return? There are two alternatives: 1. `next(now)` will return 11:25:30am, the same value that peek gave; 2. `next(now)` will return 11:27:30am, the current time. Option 1 keeps the invariant that peeking will show you the next value without advancing the iterable, but it violates the invariant that `next(now)` yields the current time. Option 2 keeps the `next` invariant, but violates the `peek` invariant. Whichever option we choose, peeking into arbitrary iterators will break somebody's expectations. Bringing it back to `first`: * It seems to me that `first` adds very little that `next` doesn't already give us. * The simple and obvious implementation of `first` would have a troublesome difference in behaviour between iterator arguments and non-iterator arguments. * To eliminate that difference would require the ability to peek ahead into arbitrary iterators, including generators, which is a much bigger change, and equally troublesome. -- Steven

26 119

Moving PEP 584 forward (dict + and += operators)
by Guido van Rossum 27 Dec '19

27 Dec '19

A month ago I promised the Steering Council I'd review PEP 584 and promptly got distracted by other things. But here it is, at last. Some procedural things first: you (the PEP authors) should apply to the Steering Council for a formal review. They most likely will assign one of the SC members as a BDFL-Delegate who will guide the PEP to a resolution (either acceptance or rejection, after any number of revisions). Don't wait until you think the PEP is perfect! It doesn't have to be for a BDFL-Delegate to be assigned. Note that the SC elections have just started, and in a few weeks we'll have a new SC. I won't be in it (I withdrew from the ballot). I don't know how this will affect the chances of this PEP. (Also note that after an election, a SC can overturn decisions by the previous SC, so I don't mean this as a reason to hurry -- just something to consider.) Now to the content of PEP 584. ## Should the operators be + and +=, or | and |= ? I haven't read _all_ the posts on this topic, but I've seen quite a few, and I still strongly favor | and |= . My reason for this is mainly that the operation really does do a set union **on the keys**. Also, + is already overloaded enough (numbers, strings, sequences) and we don't need yet another, different meaning. The | operator should be known to most users from sets (which are introduced before dicts in the official Python tutorial). I don't particularly care about learnability or discoverability in this case -- I don't think beginners should be taught these operators as a major tool in their toolbox (let them use .update()), and for anyone wondering how to do something with a dict without access to the official docs, there's always help(dict). The natural way of discovering these would be through tutorials, reading of the library docs, or by reading other people's code, not by trying random key combinations in the REPL. ## Does it matter that the operation is lossy, and not commutative in the values? Centithreads have been wasted on this topic, and I really don't care. I **certainly** don't think that introducing a new operator or using a different operator (<<, really?) is needed because of the operator's asymmetry. A tutorial can show this in two or three lines, and anybody who remembers these operators exists but has forgotten how they handle conflicting values (and doesn't remember they correspond to.update()) can type one line in the REPL to figure it out: >>> {"a": 1} | {"a": 2} {"a": 2} >>> ## What about the Counter class? The Counter class is an over-engineered contraption that has way too many APIs to remember, and is quite inconsistent in what it tries to be. (When does it delete zeros? Why does it sometimes discard negative values?) We should not use it for guidance nor should we feel constrained by its design. ## What about performance? Performance is not the only objective when using Python. Switching to inplace operators (here |=) is a generally useful and well-known technique (it also applies to string and list concatenation, for example). ## Other objections The PEP does a good job of addressing pretty much everything that's been brought up against it (in fact, perhaps it goes a little too far in humoring obvious strawmen). ## Open questions There's only one left in the PEP: "Should these operators be part of the ABC Mapping API?" No. This would require every Mapping implementation to add these. While you might think that this could be addressed by adding concrete implementations in the Mapping ABC, that doesn't work for |, since there's no API for creating new instances. It could be done for |=, by adding this to MutableMapping: def __ior__(self, other): self.update(other) but without the ability to define __or__ in Mapping this seems inconsistent. Mapping implementations are of course free to define __or__. All in all I would recommend to the SC to go forward with this proposal, targeting Python 3.9, assuming the operators are changed to | and |=, and the PEP is brought more in line with the PEP editing guidelines from PEP 1 and PEP 12. (In particular, there are some suggested section headings there that ought to be followed more closely. I'd be glad to give the authors more guidance in this matter if they request it.) -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

18 44

Renaming json.load()
by Abdur-Rahmaan Janhangeer 26 Dec '19

26 Dec '19

Greetings list, Using 3.7, in json module of the std lib there is json.load and json.loads . I gather json.loads is more used to convert string to dict structure. json.load might cause (it already causes) confusions. Suggestion: Rename json.load() so as not to confuse with json.loads() ^^_ Yours, Abdur-Rahmaan Janhangeer pythonmembers.club <http://www.pythonmembers.club/> | github <https://github.com/Abdur-rahmaanJ> Mauritius

19 76

Segmentation of string
by smfiles 25 Dec '19

25 Dec '19

I think it's necessary to add a segment() method to str type or string module. This method is used to split a string into m parts and return all cases. With segment(), you can avoid tedious calculation and indexing if you want to segment a string. For example: segment('1234', m=3) -> [('1', '2', '34'), ('1', '23', '4'), ('12', '3', '4')] segment('12345', m=3) -> [('1', '2', '345'), ('1', '23', '45'), ('1', '234', '5'), ('12', '3', '45'), ('12', '34', '5'), ('123', '4', '5')] I hope this proposal can be adopted.

12 16

Add "elif" to "for_stmt" and "while_stmt"
by komissar.off.andrey＠gmail.com 25 Dec '19

25 Dec '19

Hello! I think it will be useful in Python syntax if we can use "elif" in "for" and "while" statements besides "else" Example for i in range(j): ... elif i > 5: ... else: ... What you think about this change?

9 8

Add symlinks option at shutil.move parameters
by Namjun Kim 25 Dec '19

25 Dec '19

I think shutil.move method should provide a symlinks parameter. It's because if user wants to move file including metadata couldn't use shutil.move method. (https://github.com/python/cpython/blob/master/Lib/shutil.py#L807) if shutil.move provide symlinks option at argument, a developer doesn't need to develop their own custom move method.

6 6

Throw a TypeError exception if using type annotations (type hints) & the passed arg or return value is the wrong type
by Andrew T 24 Dec '19

24 Dec '19

This is a feature PHP has had since PHP7 and is what I use all the time with PHP. It works on a file by file basis (I guess for backward compatibility), a declare line needs to be at the top of a file `declare(strict_types=1);` If this line is at the top of the file, then a TypeError is thrown on any method/function that uses type hints in the current file if the passed arg or return value is the wrong type. Here is more info: https://dev.to/robdwaller/how-php-type-declarations-actually-work-1mm5 I would like to see this PHP feature in Python. A strict types declaration line can be added at the top any file or/and it could be set globally at the start of the script by doing something like `sys.strict_types = True`.

3 2