Oh, sorry, I will take a look.

Thanks!

Em sáb., 27 de jun. de 2020 às 12:06, Guido van Rossum guido@python.org escreveu:

Please read PEP 505 before rehashing this old idea.

On Sat, Jun 27, 2020 at 06:35 Daniel. danielhilst@gmail.com wrote:

...

When I need to traverse nested dicts, is a common pattern to do

somedict.get('foo', {}).get('bar', {})

But there is no such equivalent for arrays, wouldn't be nice if we can follow

somedict.get('foo', {}).get('bar', []).get(10) ... ?

What I do in this case is surround with try/except IndexError and set some variable to None, or wrap the try/except block in a function. But this is noise in my opinion, I just want to follow the same reasoning that I have with dicts: I'm searching for some path deep nested, and want a default if not found.

What do you think?

Regards

-- “If you're going to try, go all the way. Otherwise, don't even start. ..." Charles Bukowski _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LLK3EQ... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido (mobile)

Em ter., 30 de jun. de 2020 às 15:49, Alex Hall alex.mojaki@gmail.com escreveu:

I think I'm missing something. Daniel wants a `list.get` method with similar semantics to `dict.get`. So instead of writing:

try:
    x = lst[i]
except IndexError:
    x = default

one could write `x = lst.get(i, default)`. How would you rewrite that with PEP 505?

With PEP 505 is not possible to have an arbitrary default as the result of null coalesced expression will be None if a None is found. The lst.get(i) would be rewritten as lst?[i], and composing dct.get('foo', []).get(0, []).get('bar') would be something like dct?['foo']?[0]?['bar'], from what I read, this is very terse and terse is good in most cases good in my opinion. It removes the error handling noise by replacing it by one char

I'm repeating myself here, but again, the use case is to fetch deedly nested data. All this talk make me notice that IndexError, and KeyError are LookupError instances but AttributeError is not, still pep 505 have an elegant solution for all these three. Also it handles the None?.method() case, when something return an object or None and you want to call a method on this if is not None.

I think we should put more effort on pep 505

I've also wanted this a couple of times, although I can't remember the reason. It's not just about traversing incomplete nested data structures. I don't see much disadvantage since the analogy with `dict.get` would make it very easy to learn and understand, to the point that I've wondered why this doesn't exist already.

On Sat, Jun 27, 2020 at 5:09 PM Guido van Rossum guido@python.org wrote:

...

Please read PEP 505 before rehashing this old idea.

On Sat, Jun 27, 2020 at 06:35 Daniel. danielhilst@gmail.com wrote:

...

When I need to traverse nested dicts, is a common pattern to do

somedict.get('foo', {}).get('bar', {})

But there is no such equivalent for arrays, wouldn't be nice if we can follow

somedict.get('foo', {}).get('bar', []).get(10) ... ?

What I do in this case is surround with try/except IndexError and set some variable to None, or wrap the try/except block in a function. But this is noise in my opinion, I just want to follow the same reasoning that I have with dicts: I'm searching for some path deep nested, and want a default if not found.

What do you think?

Regards

-- “If you're going to try, go all the way. Otherwise, don't even start. ..." Charles Bukowski _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LLK3EQ... Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido (mobile) _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/L64EPH... Code of Conduct: http://python.org/psf/codeofconduct/

Alex Hall writes:

`dct.get('foo')?.get(0)?.get('bar')`.

I would quite like to have PEP 505, but I don't know how to revive it. And I'm not surprised that it's deferred, there's generally reluctance to add new syntax, especially symbols.

Bottom line: new use cases or modified semantics that are more attractive to people who don't sympathize with the "in NOSQL objects often aren't there" use case. Some handwavy discussion below.

Note that Nick Coghlan in the PEP 622 thread over on python-dev is proposing a very different use for '?' which seems likely to conflict. It wouldn't rule out syntax, but definitely would make it more difficult to use '?' (though probably not syntactically impossible) for this purpose.

I haven't seen anyone in this thread suggesting any cost or downside of adding the method, just people asking if it would be useful. I feel like I answered that question pretty thoroughly, then the thread went quiet.

Unfortunately, the people whose answers you really want to hear don't hang out here much any more. Steven d'A has remarkably comprehensive knowledge, but he's not as good at channeling Guido on what isn't yet as he is at explaining what is already. Not sure where Andrew Barnert has got to recently, but put him and Steven together and you get the best of both that's available here. Greg Ewing is another valuable source, but his design instincts often are quite different from Guido's. Lots of good ideas and discussion from these developers, but add all that up and you don't get Guido or Barry or Nick. Not necessarily *worse*, but different, and if you can't show that some feature requires a horrible hack or inordinate amounts of hard-to-get-right trickery, being different from Guido still connotes being un-Pythonic, and that's hard to get around. (Don't let that stop you from arguing with Guido -- among his other remarkable characteristics, he's been known to change his mind and accept features he long opposed. :-)

I don't suggest that you try to open up a thread on python-dev where the folks who dropped the acid in the Kool-Aid do show up frequently. All of the reasons you and others give for wanting this are well- known, and they just didn't do it. Until you've got a powerful new argument or different, more attractive semantics, you'll just get shooed back here.

I can say that it's not just the high cost of a symbol that tabled that PEP. Chained accesses like "dct['foo'][0]['bar'] are just expressions, but among the old-timers there's a deep suspicion of the "fluent" style of programming, chaining method calls by returning self. This isn't the same thing, but "null coalescing" operators (I prefer "null propagating" but "coalescing" seems to be the preferred term among proponents) has something of the same flavor, albeit only for None. It necessarily privileges None[1], guaranteeing ambiguity of that value, especially in a context like parsing JSON, where JSON null is typically translated to Python None.

There was also a concern that null coalescing encourages hard to debug, unmaintainable programming. I don't think anybody provided hard evidence on that count, but introducing an operator specifically to *postpone* detection of exceptional conditions just doesn't sit well with a lot of people. It confuses things that *should* be there but aren't, with things that *might* be there but aren't, and with None when it's actually there. Postponing error detection can be done with 'try', but try doesn't confuse those things.[2] The principle of "consenting adults" means that possibility of misuse won't kill an otherwise good idea, but the question "isn't there a more Pythonic, explicit semantics that doesn't allow errors to propagate?" is one way to get an idea tabled.

Finally, once you compare null-coalescing operators with 'try', it's clear that 'try' is far more powerful as well as sufficient for this application, and it's just not clear that null-coalescing is sufficiently special a special case to deserve syntax.

Don't @ me. That is, I don't have a horse in this race (except that it would be syntax I personally am unlikely to ever use, and probably won't even see very often). I'm just trying to summarize some of the themes of the discussion FYI.

Footnotes: [1] Addition of a special NULL object was proposed, IIRC, but I forget why that was considered a bad idea.

[2] On the other hand, ':=' got in even though that is what '=' is for. That's a precedent that PEP 505 didn't have, turning a statement into an expression.

On Thu, Jul 2, 2020 at 10:33 PM Alex Hall alex.mojaki@gmail.com wrote:

On the other hand, `list.get` seems very doable to me. It's not new syntax. It would be extremely easy to learn for anyone familiar with `dict.get`, which is pretty much essential knowledge. You'd probably have some people guessing it exists and using it correctly without even seeing it first in other code or documentation. I haven't seen anyone in this thread suggesting any cost or downside of adding the method, just people asking if it would be useful. I feel like I answered that question pretty thoroughly, then the thread went quiet.

I just had a coworker ask if there was something akin to `list.get(0)`, so I'd like to try to revive this.

I propose that:

1. There is basically no cost in terms of mental capacity, learnability, or API bloat in adding list.get because of the similarity to dict.get. 2. There are plenty of times it would be useful, as seen in https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OC... 3. If the above two points are true, we should go ahead and add this.

I think that the discussion here simply fizzled away because:

1. People got distracted by talking about PEP 505 which really isn't very relevant and would solve a different problem. 2. There are no major objections, so there isn't much left to talk about, which seems like a silly way for a proposal to die. The only decent objection I saw was skepticism about valid and frequent use cases but once I answered that no one pursued the matter.

Here is one example added in stdlib

https://github.com/python/cpython/pull/14813/files

It's checking the truthness of value before calling value[0], this is a perfect match for value.get(0)

Here is another, just the same use case, https://github.com/python/cpython/pull/17515/files

When you deal with parsing problems yielding sequence of tokens are pretty common and knowing how much tokens are there is a matter of input. When you deal with recursives problems using a stack too.. so there are use cases out there and it's a tiny change

Em ter, 25 de ago de 2020 09:52, Daniel. danielhilst@gmail.com escreveu:

I just came across this again while implementing an parser

I would like to compare stack elements as

if stack[-3] == x and stack[-2] == y and stack[-1] == z

and somewere below

elif stack[-1] == t

I had to spread `len(stack)` in a lot of places.

People said about the length of a list is usually known, but when you use it as a stack is the oposit.

Em ter, 25 de ago de 2020 09:44, Alex Hall alex.mojaki@gmail.com escreveu:

...

On Thu, Jul 2, 2020 at 10:33 PM Alex Hall alex.mojaki@gmail.com wrote:

...

On the other hand, `list.get` seems very doable to me. It's not new syntax. It would be extremely easy to learn for anyone familiar with `dict.get`, which is pretty much essential knowledge. You'd probably have some people guessing it exists and using it correctly without even seeing it first in other code or documentation. I haven't seen anyone in this thread suggesting any cost or downside of adding the method, just people asking if it would be useful. I feel like I answered that question pretty thoroughly, then the thread went quiet.

I just had a coworker ask if there was something akin to `list.get(0)`, so I'd like to try to revive this.

I propose that:

1. There is basically no cost in terms of mental capacity, learnability,

or API bloat in adding list.get because of the similarity to dict.get. 2. There are plenty of times it would be useful, as seen in https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OC... 3. If the above two points are true, we should go ahead and add this.

I think that the discussion here simply fizzled away because:

1. People got distracted by talking about PEP 505 which really isn't very

relevant and would solve a different problem. 2. There are no major objections, so there isn't much left to talk about, which seems like a silly way for a proposal to die. The only decent objection I saw was skepticism about valid and frequent use cases but once I answered that no one pursued the matter.

Oh, thanks :)

Anyway, it still applies for fetching a single arbitrary indexes

Em ter., 25 de ago. de 2020 às 11:07, Alex Hall alex.mojaki@gmail.com escreveu:

I mean you could write these as:

if stack[-3:] == [x, y, z]

and

elif stack[-1:] == [t]

But plenty of use cases still exist ( https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OC...) and I think we shouldn't need to keep talking about them.

On Tue, Aug 25, 2020 at 2:54 PM Daniel. danielhilst@gmail.com wrote:

...

I just came across this again while implementing an parser

I would like to compare stack elements as

if stack[-3] == x and stack[-2] == y and stack[-1] == z

and somewere below

elif stack[-1] == t

I had to spread `len(stack)` in a lot of places.

People said about the length of a list is usually known, but when you use it as a stack is the oposit.

Em ter, 25 de ago de 2020 09:44, Alex Hall alex.mojaki@gmail.com escreveu:

...

On Thu, Jul 2, 2020 at 10:33 PM Alex Hall alex.mojaki@gmail.com wrote:

...

On the other hand, `list.get` seems very doable to me. It's not new syntax. It would be extremely easy to learn for anyone familiar with `dict.get`, which is pretty much essential knowledge. You'd probably have some people guessing it exists and using it correctly without even seeing it first in other code or documentation. I haven't seen anyone in this thread suggesting any cost or downside of adding the method, just people asking if it would be useful. I feel like I answered that question pretty thoroughly, then the thread went quiet.

I just had a coworker ask if there was something akin to `list.get(0)`, so I'd like to try to revive this.

I propose that:

1. There is basically no cost in terms of mental capacity, learnability,

or API bloat in adding list.get because of the similarity to dict.get. 2. There are plenty of times it would be useful, as seen in https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OC... 3. If the above two points are true, we should go ahead and add this.

I think that the discussion here simply fizzled away because:

1. People got distracted by talking about PEP 505 which really isn't

very relevant and would solve a different problem. 2. There are no major objections, so there isn't much left to talk about, which seems like a silly way for a proposal to die. The only decent objection I saw was skepticism about valid and frequent use cases but once I answered that no one pursued the matter.

On Tue, Aug 25, 2020 at 5:50 AM Alex Hall alex.mojaki@gmail.com wrote:

I think that the discussion here simply fizzled away because:

1. People got distracted by talking about PEP 505 which really isn't very

relevant and would solve a different problem. 2. There are no major objections, so there isn't much left to talk about, which seems like a silly way for a proposal to die. The only decent objection I saw was skepticism about valid and frequent use cases but once I answered that no one pursued the matter.

A lot of ideas fizzle away on this list. The fact is that it's a lot of work to get a new feature accepted into Python, even if it's not very controversial -- it's the classing "scratching an itch" problem -- someone needs to find it itchy enough to do a LOT of scratching -- any idea needs a champion that will carry it though.

This one is easier than most because it's pretty much a do we or don't we question, with the spelling semantics, all pretty much already decided.

Though I haven't quite seen it said explicitly -- is this proposed to be added to the Sequence ABC?

If so, there is a potential problem -- the ABC's "namespace" is not reserved, so if you add .get(), then any code out in the while that is already using .get in a custom sequence could potentially break.

See the discussion about the idea of adding a .view() method to Sequences -- same issue. Even of you think it's a good idea, it's still hard to add a new (non-dunder) method to an ABC.

-CHB

---

Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OMMRKV... Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Aug 25, 2020 at 10:51 AM Alex Hall alex.mojaki@gmail.com wrote:

On Tue, Aug 25, 2020 at 7:23 PM Christopher Barker

I think if it's a potential problem to add it to the ABC then let's just

defer that and only add it to the built in sequences.

Well yes, that’s a fine solution. Though an advantage of adding it into the ABC is that it could be a mixin, and all Sequences would them get it.

But I'm not clear on what the problem is. If you have some code like this:

class MySequence(Sequence):
    def get(self, i): ...

MySequence().get(3)

and then add .get to the Sequence ABC, the existing code will not be immediately broken because the custom MySequence.get overrides the ABC method so everything behaves as before.

Yes, but then the third party object that was a Sequence may no longer be one.

Maybe if we build up enough methods to add to the collection ABCs

(Sequence.view, Sequence.get, Set.intersection/union/etc, Mapping.__[i]or__) we can justify adding them all at once in Python 4

Careful now! No one wants to repeat the py2/3 transition!

or in a new module collections.abc.v2 which has subclasses of the originals.

Maybe, but the trick is: how do you “build up” thus stuff before you have a place to put it? And what’s the point of a place to put new stuff, when there isn’t any new stuff?

-CHB

--

Christopher Barker, PhD

Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Wed, Aug 26, 2020 at 1:47 AM Greg Ewing greg.ewing@canterbury.ac.nz wrote:

As a point of interest, is get() considered an official part of the mapping protocol, or just nice-to-have?

The docs don't seem to be very clear about that. There used to be some tables listing the methods making up the core sequence and mapping protocols, but I can't find them now. Have they been removed? Are the ABCs now the definitions of the protocols?

I would also like to see an official answer to this. Are the mixin methods of the ABCs considered part of their contract? Is it documented somewhere? I think it should be.

On the one hand it sort of feels like the contract is just the abstract methods that users are supposed to implement themselves, and the mixin methods are just a convenient benefit of subclassing. It's hard to say where that feeling comes from, but I think the name "mixin method" is part of it. I'm glad to see I'm not the only one that gets this feeling.

On the other hand, if I have a variable of type Mapping, I'm usually going to assume it has a .get method that behaves like dict. Most tools will assume that too. If it doesn't, I'll be quite surprised and probably annoyed.

On 27/08/20 2:32 am, Guido van Rossum wrote:

The mixins are part of the contract. You are free to override them, their implementation is just a helpful,default.

That doesn't really answer my question. To put it another way: If someone wanted to implement an object that obeys the mapping protocol, but without inheriting from the Mapping ABC, would they be obliged to provide a get() method?

On Wed, Aug 26, 2020 at 4:32 PM Guido van Rossum guido@python.org wrote:

FWIW I don’t think we should add a .get() method for Sequence or for list. It reeks of hyper-correctness.

What do you mean by this? I want the method because it's useful and convenient, not because it's 'correct'. I don't think that lists are 'wrong' right now.

On Wed, Aug 26, 2020 at 7:30 PM Guido van Rossum guido@python.org wrote:

But for your convenience you are proposing a problematic change (see posts by others).

I think we've established it's problematic to add it to the Sequence ABC, but in terms of only adding it to specific sequences like lists, I don't know what other posts you're referring to. Why is it problematic?

Maybe I meant overgeneralization instead.

Again, the goal isn't to generalize. I'm not trying to make lists look like dicts. It's not even possible to get close - `if i in x: print(x[i])` does very different things for lists and dicts.

On Wed, Aug 26, 2020 at 7:56 PM Guido van Rossum guido@python.org wrote:

On Wed, Aug 26, 2020 at 10:45 AM Alex Hall alex.mojaki@gmail.com wrote:

...

On Wed, Aug 26, 2020 at 7:30 PM Guido van Rossum guido@python.org wrote:

...

But for your convenience you are proposing a problematic change (see posts by others).

I think we've established it's problematic to add it to the Sequence ABC, but in terms of only adding it to specific sequences like lists, I don't know what other posts you're referring to. Why is it problematic?

It would break duck typing -- if you check for IndexError you can accept pretty much any sequence. But if you use .get() you can only work with lists.

This is why we have ABCs in the first place (and static checking). So that you can't add it to Sequence is already a smell. If it's in the Mapping protocol, why isn't it in the Sequence protocol?

This is all just a general feature of ABCs. They were written once and frozen in place and no new methods can be added to them because it would break compatibility. It's not specific to this proposal. | and |= operators were added to dict but not to Mapping/MutableMapping for the same reason, even though the same operators exist with a similar meaning in the Set/MutableSet protocols. So there's a clear precedent for this.

It's a little unfortunate that we can't put this in the Sequence ABC, but it's not a big deal. The vast majority of the time I'm interfacing with my own code and so I know that I'm working with a list, not some mysterious Sequence. I don't think people are going to change their public facing function signatures from `def foo(x: Sequence)` to `(x: list)` just so that they can use `x.get`. Even if they do, it suggests that they think `.get` is really useful, and callers can just convert their argument to a list first. Similarly adding the union operator to dict is fine because people usually use dicts, not generic Mappings.

Besides, list already defines a bunch of stuff not found in Sequence or MutableSequence: copy, sort, `<`, `+`, and `*`. What's one more?

There's also the slippery-slope argument brought up earlier -- should it be added to tuple? To range?

I think Steven D'Aprano asked this. I don't see it as an argument against or a concern about a slippery slope, just a question. I think it'd be nice to have it on lots of sequences, but it's OK if it's not. Neither scenario strikes me as problematic.

The union operator was also added to defaultdict, OrderedDict, and ChainMap. Was this a problem?

What should it do for a negative index?

I think most people expect that if `lst[i]` returns a value then `lst.get(i)` should return the same value, therefore it should return a default only when `lst[i]` fails. Sometimes you specifically want `lst.get(-1)` - there's a couple examples of that in [my earlier post]( https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OC...) and Daniel recently presented a case when he wanted that.

This isn't the ideal behaviour for all use cases, but only as much as normal indexing isn't the ideal behaviour for all use cases. Sometimes I've had bugs because I forgot to account for negative indices, but I'm still very grateful to you that negative indices work. We can't make something equally convenient for all use cases, but that doesn't diminish the value of this method.

You said yourself that the generic alternative for all sequences is to "check for IndexError", a sign that doing the equivalent of that is more intuitive.

I also think it negatively affects readability. Today when I see `foo.get(bar)` I assume that (unless the code is full of tricks) foo is a dict or Mapping

That's already a poor assumption to make. `get` is an extremely generic method name that has lots of meanings in different classes.

But if list also grows a .get() method I have to assume in my head that foo is a list and then bar an int, and I basically have to read the rest of the code twice, once assuming foo is a Mapping, and once assuming it's a list.

The purpose of `foo.get(bar)` is to get a value, not to tell you that foo is a dict. If the rest of the code isn't able to tell you whether `foo` is a dict or a list, then either the code is poorly written (and .get is not to blame) or it doesn't matter because you're just getting individual values. `foo[bar]` doesn't differentiate between lists and dicts and this is generally considered a good thing, not a problem.

The generalization I am talking about here is the assumption that because list and dict both support x[y], and dict also has x.get(y) which returns a default instead of raising, then it would be a good idea to add the latter also to list.

I'm not assuming that list should have .get because dict has it. I'm claiming list should have it because it would often be useful, based on [empirical evidence]( https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OC... ).

Anyway, thank you for this. I'm enjoying the discussion and I'm glad to hear actual reasons that can be argued about rather than the proposal just fizzling away.

I’m sorry, I just can’t teach you how static type checking works. Read the mypy docs, install it, and experiment.

On Wed, Aug 26, 2020 at 13:10 Christopher Barker pythonchb@gmail.com wrote:

On Wed, Aug 26, 2020 at 12:38 PM Alex Hall alex.mojaki@gmail.com wrote:

...

This is why we have ABCs in the first place (and static checking). So that you can't add it to Sequence is already a smell. If it's in the Mapping protocol, why isn't it in the Sequence protocol?

This is all just a general feature of ABCs. They were written once and frozen in place and no new methods can be added to them because it would break compatibility. It's not specific to this proposal. | and |= operators were added to dict but not to Mapping/MutableMapping for the same reason, even though the same operators exist with a similar meaning in the Set/MutableSet protocols. So there's a clear precedent for this.

and the Set ABC doesn't have the set methods, like .union, .intersection ... (though I still don't get why that is ...)

...

It's a little unfortunate that we can't put this in the Sequence ABC, but it's not a big deal. The vast majority of the time I'm interfacing with my own code and so I know that I'm working with a list, not some mysterious Sequence. I don't think people are going to change their public facing function signatures from `def foo(x: Sequence)` to `(x: list)` just so that they can use `x.get`. Even if they do, it suggests that they think `.get` is really useful, and callers can just convert their argument to a list first. Similarly adding the union operator to dict is fine because people usually use dicts, not generic Mappings.

This is something to be thinking about, however -- having the builtins extend the ABCs hasn't been a big deal, what with dynamic typing and all -- but as folks start using type hints, and static type checking more, it could be an issue.

Honestly, I haven't delved into type checking much, so maybe I'm making this up, but what if you write a function that is type hinted to take a Mapping, and uses | or |=, and all the test code uses a built in dict, and all seems to be well.

Then this code starts being more widely used, and someone runs it with a different Mapping object, and it passes the static type checking pass, and then, oops, a run-time failure.

It makes me wonder of the utility of the ABCs is the builtins are extending them.

Maybe we need a set of (at least for testing) builtins that are fully ABC conformant ?

-CHB

...

Besides, list already defines a bunch of stuff not found in Sequence or MutableSequence: copy, sort, `<`, `+`, and `*`. What's one more?

...

There's also the slippery-slope argument brought up earlier -- should it be added to tuple? To range?

I think Steven D'Aprano asked this. I don't see it as an argument against or a concern about a slippery slope, just a question. I think it'd be nice to have it on lots of sequences, but it's OK if it's not. Neither scenario strikes me as problematic.

The union operator was also added to defaultdict, OrderedDict, and ChainMap. Was this a problem?

...

What should it do for a negative index?

I think most people expect that if `lst[i]` returns a value then `lst.get(i)` should return the same value, therefore it should return a default only when `lst[i]` fails. Sometimes you specifically want `lst.get(-1)` - there's a couple examples of that in [my earlier post]( https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OC...) and Daniel recently presented a case when he wanted that.

This isn't the ideal behaviour for all use cases, but only as much as normal indexing isn't the ideal behaviour for all use cases. Sometimes I've had bugs because I forgot to account for negative indices, but I'm still very grateful to you that negative indices work. We can't make something equally convenient for all use cases, but that doesn't diminish the value of this method.

You said yourself that the generic alternative for all sequences is to "check for IndexError", a sign that doing the equivalent of that is more intuitive.

...

I also think it negatively affects readability. Today when I see `foo.get(bar)` I assume that (unless the code is full of tricks) foo is a dict or Mapping

That's already a poor assumption to make. `get` is an extremely generic method name that has lots of meanings in different classes.

...

But if list also grows a .get() method I have to assume in my head that foo is a list and then bar an int, and I basically have to read the rest of the code twice, once assuming foo is a Mapping, and once assuming it's a list.

The purpose of `foo.get(bar)` is to get a value, not to tell you that foo is a dict. If the rest of the code isn't able to tell you whether `foo` is a dict or a list, then either the code is poorly written (and .get is not to blame) or it doesn't matter because you're just getting individual values. `foo[bar]` doesn't differentiate between lists and dicts and this is generally considered a good thing, not a problem.

...

The generalization I am talking about here is the assumption that because list and dict both support x[y], and dict also has x.get(y) which returns a default instead of raising, then it would be a good idea to add the latter also to list.

I'm not assuming that list should have .get because dict has it. I'm claiming list should have it because it would often be useful, based on [empirical evidence]( https://mail.python.org/archives/list/python-ideas@python.org/message/7W74OC... ).

Anyway, thank you for this. I'm enjoying the discussion and I'm glad to hear actual reasons that can be argued about rather than the proposal just fizzling away.

---

Python-ideas mailing list -- python-ideas@python.org

To unsubscribe send an email to python-ideas-leave@python.org

https://mail.python.org/mailman3/lists/python-ideas.python.org/

Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/K2PMSI...

Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD

Python Language Consulting

• Teaching
• Scientific Software Development
• Desktop GUI and Web Development
• wxPython, numpy, scipy, Cython

--

--Guido (mobile)

Use typing.AbstractSet for the ABC (there was an unfortunate naming conflict -- we have dict and Mapping, list and Sequence, but set and Set...)

On Wed, Aug 26, 2020 at 2:16 PM Alex Hall alex.mojaki@gmail.com wrote:

On Wed, Aug 26, 2020 at 10:10 PM Christopher Barker pythonchb@gmail.com wrote:

...

Honestly, I haven't delved into type checking much, so maybe I'm making this up, but what if you write a function that is type hinted to take a Mapping, and uses | or |=, and all the test code uses a built in dict, and all seems to be well.

This made me curious so I did an experiment, but with Set because I couldn't be bothered to worry about Python versions. This code:

import typing
import collections
import builtins

def foo(t: typing.Set, c: collections.Set, b: builtins.set):
    print(t + t)
    print(t | t)
    print(t.union(t))

    print(c + c)
    print(c | c)
    print(c.union(c))

    print(b + b)
    print(b | b)
    print(b.union(b))

complains about `+` (as it should) and `c.union` in both PyCharm and mypy, and nothing else. So typing.Set is based on the builtin and not the ABC. If you want to support any subclass of collections.Set, your type hint needs to be collections.Set. Using typing.Set will fail to show warnings when you use a builtin only method, and will show warnings when you try to assign an ABC set (e.g. I get a warning with `foo(t=<instance of collections.abc.Set>)`. The problem then is that `collections.abc.Set[int]` doesn't work, i.e. you need to use typing.Set if you want generics.

I think this method would be useful for list itself (maybe tuple, but I care less about that). It certainly should not be part of the Sequence protocol.

I have often programmed a pattern where I .append() elements to a list in a loop, and *expect* at least N things to make it in. In such case, getting a sentinel answer to mylist.get(N-1) would be convenient.

Obviously, I've lived without that fine. But it's a little bit shorter and more expressive than:

x = mylist[N-1] if len(mylist) >= N else None

On Wed, Aug 26, 2020, 1:30 PM Guido van Rossum guido@python.org wrote:

But for your convenience you are proposing a problematic change (see posts by others). Maybe I meant overgeneralization instead.

On Wed, Aug 26, 2020 at 10:20 AM Alex Hall alex.mojaki@gmail.com wrote:

...

On Wed, Aug 26, 2020 at 4:32 PM Guido van Rossum guido@python.org wrote:

...

FWIW I don’t think we should add a .get() method for Sequence or for list. It reeks of hyper-correctness.

What do you mean by this? I want the method because it's useful and convenient, not because it's 'correct'. I don't think that lists are 'wrong' right now.

-- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/ _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/D7JMHG... Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Aug 26, 2020 at 1:41 AM Steven D'Aprano steve@pearwood.info wrote:

On Tue, Aug 25, 2020 at 10:22:20AM -0700, Christopher Barker wrote:

...

This one is easier than most because it's pretty much a do we or don't we question, with the spelling semantics, all pretty much already decided.

Is it to be added to lists, or lists and tuples? How about range objects?

range(1, 9, 2).get(100)

Strings and bytes?

"abc".get(100)

I think at a minimum it should be in lists and tuples.

I can't see any reason not to add it to range, strings, bytes, deque, array, etc. beyond implementation effort. I also think it's fine to leave them out at first.

Do we require 3rd party sequences to add this as well, in order to continue being called sequences?

I think this is pretty much the same question as the discussion about ABCs. It seems like the answer is "no", or at least "not yet".

...

Though I haven't quite seen it said explicitly -- is this proposed to be added to the Sequence ABC?

If so, do we require that it return None as the default, or are types permitted to return whatever missing value they see fit?

This should be identical to Mapping.get. None isn't required to be the default missing value - callers can specify any value they want - but None is the default default, i.e. `def get(self, item, default=None)`. I don't know of any Mapping or mapping-like class that has a different default default. If such classes exist, then I guess None is just the default default default :)

On Wed, Aug 26, 2020 at 5:00 PM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

If the "keyword arguments in __getitem__" feature is added, .get() is purely redundant. (Of course this thread would then become "make 'default' a standard keyword argument for mutable collections.")

Is that something people want to do? Do people want to be able to write `my_dict[key, default=0]` instead of `my_dict.get(key, 0)`? What about `my_dict[key, default=None]` instead of `my_dict.get(key)`?

On Thu, Aug 27, 2020 at 10:15 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

Alex Hall writes:

...

On Wed, Aug 26, 2020 at 5:00 PM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

...

If the "keyword arguments in __getitem__" feature is added, .get() is purely redundant. (Of course this thread would then become "make 'default' a standard keyword argument for mutable collections.")

Is that something people want to do? Do people want to be able to write `my_dict[key, default=0]` instead of `my_dict.get(key, 0)`? What about `my_dict[key, default=None]` instead of `my_dict.get(key)`?

Of course they don't want to do that. Of course dict.get is going nowhere. Of course that's redundant. Of course if collections get a standard 'default' for __getitem__, many people *will* start writing `my_dict[key, default=None]`, if only because they don't read enough docs to know about dict.get.

OK, I'll try again. Do people want collections to get a standard 'default' for `__getitem__`?

I don't want mappings to grow a second way to do the same thing, and I don't want sequences to have a different way to do it from mappings.

On 27.08.2020 14:40, Daniel. wrote:

It's simple:

def get(self, i, default=None):     try:         return self[i]     except IndexError:         return default

That was the idea, just a safe wrapper arround IndexError. Because it's very common to need to catch this. I posted two bugs that happened because of uncaught IndexErrors.

.get() enforces better coding, it's more readable, avoid distracting indentation blocks and flow jumping, the error handling is reduced and more explicity and more fluid, you can get a fine default if you want. Besides breaking Sequences contract on list, (which is already broken as Alex said, by .copy, < and other methods) I cant see any other problem for adding it to *lists*

I disagree on the above assessment. I have had such a get() builtin in mxTools for more than 20 years now and found that I hardly ever used it:

https://www.egenix.com/products/python/mxBase/mxTools/doc/#_Toc293606201

The reason is simple: unlike for dicts, where you often expect non-existing items (e.g. in keyword arguments, config files, etc.), having a missing index position in a list which you parse is almost always an error.

Now, errors should be clearly marked and handled as such, hence having a try-except is the better coding strategy.

For those cases, where a list can have a variable number of entries (e.g. optional arguments, file lists, etc.), code should clearly branch on list length and then determine the right strategy to fetch items.

On Thu, Aug 27, 2020 at 3:09 PM M.-A. Lemburg mal@egenix.com wrote:

I disagree on the above assessment. I have had such a get() builtin in mxTools for more than 20 years now and found that I hardly ever used it:

https://www.egenix.com/products/python/mxBase/mxTools/doc/#_Toc293606201

The reason is simple: unlike for dicts, where you often expect non-existing items (e.g. in keyword arguments, config files, etc.), having a missing index position in a list which you parse is almost always an error.

Now, errors should be clearly marked and handled as such, hence having a try-except is the better coding strategy.

For those cases, where a list can have a variable number of entries (e.g. optional arguments, file lists, etc.), code should clearly branch on list length and then determine the right strategy to fetch items.

I'm copying my earlier post because apparently linking to it repeatedly has no effect and people keep on claiming that this method wouldn't be useful while not saying a word about the evidence I presented that it would be.

dict.get is definitely much more useful. But list.get is still useful reasonably often.

You can see quite a bit of demand for the method here: https://stackoverflow.com/questions/5125619/why-doesnt-list-have-safe-get-me...

Below are some use cases I found and how they could be refactored with .get. Apart from the first one, they were all found simply by grepping for `except IndexError`. It's pretty easy to pick obvious use cases out from there. In some cases readability suffers a bit and using .get might not be the best idea, but I've included them anyway so that we can see what over-eager usage of the method might look like.

``` _pytest/_code/source.py if insert_index >= len(values): end = None else: end = values[insert_index]

end = values.get(insert_index)

--------------------

prompt_toolkit/layout/layout.py try: return self._stack[-2].content except IndexError: return self._stack[-1].content

return self._stack.get(-2, self._stack[-1]).content

--------------------

prompt_toolkit/buffer.py try: word = words[state.n] except IndexError: word = ""

word = words.get(state.n, "")

--------------------

prompt_toolkit/widgets/menus.py try: selected_item = self.selected_menu[level + 1] except IndexError: selected_item = -1

selected_item = self.selected_menu.get(level + 1, -1)

--------------------

prompt_toolkit/document.py try: return self.text[self.cursor_position + offset] except IndexError: return ""

return self.text.get(self.cursor_position + offset, "")

--------------------

prompt_toolkit/contrib/regular_languages/lexer.py try: return lines[lineno] except IndexError: return []

return lines.get(lineno, [])

--------------------

pip/_internal/cli/autocompletion.py try: subcommand_name = [w for w in cwords if w in subcommands][0] except IndexError: subcommand_name = None

subcommand_name = [w for w in cwords if w in subcommands].get(0)

--------------------

pip/_internal/cli/autocompletion.py try: current = cwords[cword - 1] except IndexError: current = ''

current = cwords.get(cword - 1, '')

--------------------

pip/_internal/configuration.py try: return self._get_parser_to_modify()[0] except IndexError: return None

return self._get_parser_to_modify().get(0)

--------------------

pip/_vendor/distlib/resources.py try: result = self.index[i].startswith(path) except IndexError: result = False

result = self.index.get(i, "").startswith(path)

--------------------

stack_data/core.py try: return not self.lines[i - 1].strip() except IndexError: return False

return not self.lines.get(i - 1, "not blank").strip()

--------------------

parso/tree.py try: return self.parent.children[i + 1] except IndexError: return None

return self.parent.children.get(i + 1)

--------------------

ipython/IPython/core/completer.py try: return self.matches[state] except IndexError: return None

return self.matches.get(state)

--------------------

ipython/IPython/core/magics/basic.py mode = '' try: mode = parameter_s.split()[0][1:] except IndexError: pass

mode = parameter_s.split().get(0, '')[1:]

--------------------

ipython/IPython/core/tests/simpleerr.py try: mode = sys.argv[1] except IndexError: mode = 'div'

mode = sys.argv.get(1, 'div')

--------------------

ipython/IPython/utils/text.py try: tgt = parts[field] return tgt except IndexError: return ""

return parts.get(field, "")

--------------------

py/_xmlgen.py try: tag.parent = self.parents[-1] except IndexError: tag.parent = None

tag.parent = self.parents.get(-1, None)

--------------------

ipython/IPython/core/tests/tclass.py try: name = sys.argv[1] except IndexError: pass else: if name.startswith('C'): c = C(name)

name = sys.argv.get(1, "") if name.startswith('C'): c = C(name) ```

On Thu, Aug 27, 2020 at 3:37 PM M.-A. Lemburg mal@egenix.com wrote:

If I'm not mistaken, all those listed cases fall under the second case I mentioned (branching on length).

If your API allows for optional list entries, then it's better to branch on length (e.g. as in the pytest case). If this happens only in exceptional cases, use try-except (most other cases you listed).

All but the first example, i.e. all the uses of try-except, were found by grepping for `except IndexError`. That was just the easiest method I found to find examples. The fact that try-except was used in those cases does not necessarily mean that the index is missing "only in exceptional cases". Perhaps because you like to emphasise intent when you're coding you assume that "exceptional" is the intent when you see the keyword "except". It's not how I interpret these examples. I think truly exceptional cases would have a change in behaviour like showing a warning. All that's happening here is getting a default value, and using try-except is the easiest available syntax. Often the use case is that the list may be empty or you're at the edge of the list and you need to go one beyond. That's normal and expected, not exceptional. If list.get existed then probably it would have been used in many of these examples. But of course it didn't exist and I don't have a way of searching for custom get functions so nearly all the examples use except which you take to mean exceptional.

On Thu, Aug 27, 2020 at 11:37 AM M.-A. Lemburg mal@egenix.com wrote:

Asking a dict d of potentially any number of items for existence of a particular item x is somewhat different than asking a list l of a certain length i for the item at position i+1.

Suppose we didn't have dict.get(). I would then probably write:

val = mydict[key] if key in mydict else None

Likewise, since we don't have list.get(), I would write:

val = mylist[N] if len(mylist) >- N-1 else None

Neither of those is impractical, but in both cases .get(key) seems to express the intent in a more obvious way.

That said, it is trivial to write a get() function that does the same thing... and I've never bothered to do so. In fact, I could write a get() function that was polymorphic among mappings and sequences with very little work, which I've also never done. So I guess my case for the huge importance is undercut :-).

M.-A. Lemburg writes:

On 27.08.2020 17:53, David Mertz wrote:

...

Suppose we didn't have dict.get(). I would then probably write:

val = mydict[key] if key in mydict else None

Likewise, since we don't have list.get(), I would write:

val = mylist[N] if len(mylist) >- N-1 else None

Neither of those is impractical, but in both cases .get(key) seems to express the intent in a more obvious way.

Really ?

I think David is comparing the .get syntax to the if-else syntax.

First of all, he got the if-else wrong, which is a sign that .get is easier to use, at least for writers and quite likely for readers.

Second I agree that ".get" is easier to read and understand (especially with explicit default, as you advocate). It's even more stark if you're after a component of the list element:

val = mylist[N if len(mylist) > N else default].content

vs.

val = mylist.get(N, default).content

Let me try again :-)

Getting an item from a bag of unknown items can easily fail, because you don't what's in the bag until you try to find the item.

Trying to access an item from a non-existing bucket in an array will always fail. You know that in advance, since the bucket doesn't exist as per the array properties. You don't have to try finding it first.

But this isn't a C array. You don't know the length from the syntax. So you have to "do work" to discover the bucket isn't there before you access -- or you can delegate to the interpreter and catch IndexError. Given how precise IndexError is (it only catches integer arguments that fall outside the range of the array), I don't really see the argument for refusing to encapsulate the try in a .get method, or that its semantics differ from explicit bounds-checking.

It seems to me that you're expressing a preference for "look before you leap" over "easier to ask forgiveness than permission" in the case of array indicies, which is a stylistic thing. It's a style I prefer in this case, but I can also see how other people would prefer a different style, one that makes sense to encapsulate in .get. Is there something more than stylistic preference that I'm missing?

On Fri, Aug 28, 2020 at 4:30 AM Stephen J. Turnbull < turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:

...

...

Suppose we didn't have dict.get(). I would then probably write: val = mydict[key] if key in mydict else None

Likewise, since we don't have list.get(), I would write: val = mylist[N-1] if len(mylist) >= N else None

I think David is comparing the .get syntax to the if-else syntax. First of all, he got the if-else wrong, which is a sign that .get is easier to use, at least for writers and quite likely for readers.

It was an unintentional mistake, but Stephen is right it makes the point. I fixed above, both my thinko reversing the N vs. N-1, and also the simple typo of `>-` instead of `>=`. No language change is going to fix the fact the '-' and '=' keys are next to each other on my keyboard.

I think the comparison (with right syntax and semantics) is fair. Whatever the ontological significance MAL notes about "reaching into the void" versus "reaching into the dark", the code I would actually write if dict.get() didn't exist is pretty much the above. And it looks a whole lot like the code I actually do write for possibly-missing index positions.

As a side note, I don't really get why everyone else thinks a try/except is the most natural approach while a ternary seems more obvious to me for this situation. But it kinda connects to me liking list.get() better, I think... since "not enough items" doesn't seem as *exceptional* to me as it apparently does to some others.

On 2020-08-28 at 08:41:04 -0400, Regarding "[Python-ideas] Re: What about having a .get(index, default) method for arrays like we have for dicts?," Richard Damon Richard@Damon-Family.org wrote:

On 8/28/20 8:24 AM, David Mertz wrote:

...

As a side note, I don't really get why everyone else thinks a try/except is the most natural approach while a ternary seems more obvious to me for this situation.  But it kinda connects to me liking list.get() better, I think... since "not enough items" doesn't seem as *exceptional* to me as it apparently does to some others.

try/except is specifically looking for the error in question, index out of bounds. The ternary requires restating in other words the bounds limit.

For instance, for a list of N items, the allowable subscripts are -N to N-1, and the sample ternary is only checking for the upper bound, not the lower, so fails an too small of an index (maybe you know you are only using positive indexes?) Missing this won't happen with try/except.

Without more context, I've lost the forest for the trees. There are multiple ways to do this, and the "best" way likely depends on the context and not the readability, writability, maintainability, or expressivity of that exact line of code.

Where did the index come from? How/when/where was the list constructed? Could this "problem" have been recognized and/or addressed sooner? What, exactly, are the conditions that cause the index being out of bounds at this point in the logic, and what are the semantics now that it's happened? Given the preceding logic, is an index that's too small (i.e. mathematically less than the opposite of the lenght of the list) even possible?

I didn't find either of the examples (one using an if/else ternary and one using a hypothetical list.get method) particulary easy to read or particularly indicative of an obvious use case. And at that point, re-writing one [presumably correct] line of code into another won't make or break anything.

Em qui, 27 de ago de 2020 10:37, M.-A. Lemburg mal@egenix.com escreveu:

On 27.08.2020 15:17, Alex Hall wrote:

...

On Thu, Aug 27, 2020 at 3:09 PM M.-A. Lemburg mal@egenix.com wrote:

...

I disagree on the above assessment. I have had such a get() builtin in mxTools for more than 20 years now and found that I hardly ever used it:

https://www.egenix.com/products/python/mxBase/mxTools/doc/#_Toc293606201

...

...

The reason is simple: unlike for dicts, where you often expect non-existing items (e.g. in keyword arguments, config files, etc.), having a missing index position in a list which you parse is almost always an error.

Now, errors should be clearly marked and handled as such, hence having a try-except is the better coding strategy.

For those cases, where a list can have a variable number of entries (e.g. optional arguments, file lists, etc.), code should clearly branch on list length and then determine the right strategy to fetch items.

I'm copying my earlier post because apparently linking to it repeatedly

has

...

no effect and people keep on claiming that this method wouldn't be useful while not saying a word about the evidence I presented that it would be.

dict.get is definitely much more useful. But list.get is still useful reasonably often.

You can see quite a bit of demand for the method here:

https://stackoverflow.com/questions/5125619/why-doesnt-list-have-safe-get-me...

...

Below are some use cases I found and how they could be refactored with .get. Apart from the first one, they were all found simply by grepping

for

...

`except IndexError`. It's pretty easy to pick obvious use cases out from there. In some cases readability suffers a bit and using .get might not

be

...

the best idea, but I've included them anyway so that we can see what over-eager usage of the method might look like.

If I'm not mistaken, all those listed cases fall under the second case I mentioned (branching on length).

If your API allows for optional list entries, then it's better to branch on length (e.g. as in the pytest case). If this happens only in exceptional cases, use try-except (most other cases you listed).

Both coding styles make the intent clear without doubt.

As mentioned before, I had access to such a builtin for many years, but hardly ever chose to use it and instead opted for the more explicit branching on length.

IMO, being able to write something with fewer key strokes is not always a good idea. Unless you write code only for yourself, it's usually better to emphasize on intent rather than brevity.

In this case is pretty obvious the intent and this is an know abstraction. Avoid brevity will not make it cleaner. IMHO, explicitng index bounds and try/catch are distracting noise which hides intent instead.

Also .get() would not prevent anybody from explicity checking bounds or catching index exceptions when appropriate

-- Marc-Andre Lemburg eGenix.com

Professional Python Services directly from the Experts (#1, Aug 27 2020)

...

...

...

Python Projects, Coaching and Support ... https://www.egenix.com/ Python Product Development ... https://consulting.egenix.com/

---

::: We implement business ideas - efficiently in both time and costs :::

eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 https://www.egenix.com/company/contact/ https://www.malemburg.com

On Thu, Aug 27, 2020 at 6:32 PM Steven D'Aprano steve@pearwood.info wrote:

Personally, I found your examples underwhelming because they're mostly repetitions of the same pattern.

That's surprising to me. When I see the same pattern over and over that's when I most want to refactor into a common function/method. The repetition is annoying and the solution is easy and obvious. If the cases were all different it'd be much more complicated and chances are there wouldn't be a simple function that covered all cases.

Do you not usually refactor duplicated code? What *do* you refactor?

I'd find it more interesting if there

were a larger variety of cases

This sort of sounds like the problem. The examples aren't meant to be interesting.

Alex Hall writes:

Do you not usually refactor duplicated code?

Taking "you" as generic: it depends.

What *do* you refactor?

Code is expressive. In cases where the duplication results in equivalent objects, I habitually refactor. When the process is important to express and the duplication is syntactic, I will prefer an explicit but idiomatic syntax repeated in unrelated contexts. Thus I generally wouldn't consider your list to be "duplicated code that cries out for refactoring."

When the process is complex then I'd refactor. Detecting an out of bounds index is complex enough because of fenceposts and negative indicies, although I probably wouldn't notice the negative indicies until they bit me (and in that case wouldn't refactor until bitten).

...

I'd find it more interesting if there were a larger variety of cases

This sort of sounds like the problem. The examples aren't meant to be interesting.

"You [both] keep using that word but I do not think [Steven] means what you think it means." (pace, Inigo)

I find the long list of examples with mostly identical syntax *unpersuasive* for two reasons. First, if you're going to bounds check array indicies explicitly (rather than allow IndexError to "bubble up"), we already have a facility for doing that precisely and accurately: catch IndexError. That catches out of bounds indicies and nothing else[1], and it handles negative indicies correctly. list.get can't be implemented much more efficiently than by using try ... except IndexError, and invoking it imposes attribute lookup and function call overhead so I guess you net approximately nothing on time, and a little bit of space. If you care so much about either time or space, you'd implement inline, something like:

value = lst[index] if - (N := len(lst)) <= index < N else default

try ... except IndexError is TOOWTDI now, it's as readable as .get and more readable than the inline version, so introducing list.get would be redundant.

Second, I can see why people would want it (see my reply to MAL), I just don't want it myself, and implementing your own *correctly* as a function is trivial:

def get(container, subscript, default): try: return container[subscript] except (IndexError, KeyError): return default

It follows that I think dict.get is a YAGNI, at least since the introduction of KeyError, and definitely now that we can write

y = mydict[x] if x in mydict else default

Footnotes: [1] It turns out that slice bounds can't be out of bounds, they simply get truncated, in the degenerate case to the empty slice.

Alex Hall writes:

OK, I'll try again. Do people want collections to get a standard 'default' for `__getitem__`?

I don't know.

I don't want mappings to grow a second way to do the same thing,

They're going to get one, is my reading of the "named indicies" thread. That is, it looks to me very likely that index notation (`x[i]`) is going to support keyword arguments (`x[i, k=v]`). If so, that's going to be a natural way to support a `default` argument. I imagine some people will choose it because they like it, and if there are enough people who do there will be pressure for it to be TOOWTDI.

and I don't want sequences to have a different way to do it from mappings.

I think you have to choose a second way for mappings to get, or have sequences do it differently, or have redundancy in sequence operations.

If the index operator supports keyword arguments (I think it likely; YMMV), you either have to give dict a keyword argument or you have to give sequences a .get method. The former redundancy implied for dict can be justified by backward compatibility with dict and consistency with sequences. The latter is just an arbitrary consistency, and I doubt it will happen. That's just my guess; I have no power to make it go either way.

On Thu, 27 Aug 2020 at 16:21, David Mertz mertz@gnosis.cx wrote:

On Thu, Aug 27, 2020 at 10:51 AM Stephen J. Turnbull turnbull.stephen.fw@u.tsukuba.ac.jp wrote:

...

They're going to get one, is my reading of the "named indicies" thread. That is, it looks to me very likely that index notation (`x[i]`) is going to support keyword arguments (`x[i, k=v]`). If so, that's going to be a natural way to support a `default` argument. I imagine some people will choose it because they like it, and if there are enough people who do there will be pressure for it to be TOOWTDI.

I support named indices. But I strongly oppose using them in list, tuple, or dict themselves. So `mylist[99, default=4]` would still be a syntax error (or maybe a different exception).

I only want named indices for data structures where a clear and compelling use is demonstrated (like xarray). But allowing them syntactically allows third parties to play around in their own classes to see if they are helpful.

Agreed - allowing keywords in subscripts is OK (I'm personally ambivalent about the value of it, but if people want it, sure). But only for 3rd party classes. No-one has come up with a good use case for allowing them in existing builtin classes, and I'm -1 on doing so without a strong use case.

Furthermore, I'd be very, very strongly opposed to having x[i, default=d] be a way of supplying a default result if i isn't present in *any* subscriptable class, and particularly in dict as an equivalent for x.get(i, d). It's redundant, confusing, and unlike any other language I'm familiar with (all IMO, if that's not obvious...).

Paul

On Fri, 28 Aug 2020 at 09:30, Stephen J. Turnbull turnbull.stephen.fw@u.tsukuba.ac.jp wrote:

Paul Moore writes:

...

Furthermore, I'd be very, very strongly opposed to having x[i, default=d] be a way of supplying a default result if i isn't present in *any* subscriptable class, and particularly in dict as an equivalent for x.get(i, d). It's redundant,

True, but not really fair under my assumption that `x[i, default=d]` becomes the accepted way of doing it generally. dict.get would be a legacy in that case.

So one of [,default=] and dict.get is redundant. Why make the existing way the redundant one? What's the actual benefit?

Particularly as *other* uses of keywords in indexes may well want to allow general keywords (Pandas named columns, for example) so we can't reasonably reserve "default" to mean a default value in the general case. Conversely, having a get method (if it exists) behave like the stdlib dict.get seems a reasonable application of duck typing.

How do you folks feel about list.get and tuple.get (I expect many folks will think differently about these because tuples are immutable), and more generally Sequence.get?

I'm indifferent to list.get. I've never felt any need for it, and it's easy to implement if someone wants it (as you note in another mail). Making it a list method just means it doesn't work as well as a standalone function (in the sense that it's not going to support other types via duck typing). I see absolutely no point in tuple.get, it seems like the only reasons for it would be over-generalisation, or as a workaround for the fact that methods don't duck-type (x.get(i, default) where you don't know if x is a list or tuple - a get *function* would of course handle this perfectly via duck typing).

...

confusing, and unlike any other language I'm familiar with (all IMO, if that's not obvious...).

All this opposition is interesting. For reasons similar to those expressed by MAL for sequences I rarely use dict.get, let alone want list.get. To me it usually just feels wrong to be getting something that might not be there, and then leaving it that way. I use defaultdict much more than I use dict.get.

I use dict.get a lot - as MAL says, not being sure if a particular key is present in a dict is perfectly normal, in contrast to not being sure if an index is valid for a list. Consider that "key in dict" is a language feature, but there's no "index in list" syntax...

Regarding the statement that it's "unlike any other language," I suspect that's true; the only possible exception would be Lisps, but I think the argument that they don't have true indexing ends it. Partly by analogy to Lisp, I think of indexing as an optimized case of function call (which is how it's implemented in Python! That's just an amusing observation, not intended seriously.) The square brackets basically say "you can expect this operation to be O[1], almost always with small c" for appropriate values of "small". So this syntax doesn't bother me, given that functions support arbitrary keywords.

Lisp is an exception to most things :-)

I guess all this makes me an extremist. Doesn't stop me from liking the Python you prefer just as well, though. :-)

:-)

Paul

On Fri, Aug 28, 2020 at 05:30:26PM +0900, Stephen J. Turnbull wrote:

David Mertz writes:

...

I support named indices. But I strongly oppose using them in list, tuple, or dict themselves. So `mylist[99, default=4]` would still be a syntax error (or maybe a different exception).

I don't think it can be a SyntaxError because you can't always know that mylist is a builtin list.

It shouldn't be a syntax error because it wouldn't be a syntax error.

On 29/08/20 12:06 am, David Mertz wrote:

I only thought of it because `mydict[foo=bar]` now raises a SyntaxError, so raising something different would technically be a change in behavior.

It's going to be a change in behaviour whatever you raise, because it's currently a compile time error.

On Thu, Aug 27, 2020, 1:15 PM Christopher Barker

I agree -- this is very much a feature for third party packages -- or *maybe* some future stdlib class, but the builtins are fine as they are.

In fact, I don't think there's a single use of a tuple of indexes (meaning something other than an arbitrary single object) in the stdlib is there? I know I've only used that in numpy.

I don't know whether it is in the stdlib, but I sometimes use tuples as dict keys. E.g.

mydict[('mertz', 'david')] = 3.1415

Even though I could omit them, I'd almost surely use the parens for that. And more likely it would be:

name = ('mertz', 'david') mydict[name] = 3.1415

Conceptually, an "immutable collection" serves a different purpose than "a collection of axes", even if they work then same under the hood.

On Thu, Aug 27, 2020, 1:35 PM Ricky Teachey

Conceptually, an "immutable collection" serves a different purpose than "a

...

collection of axes", even if they work then same under the hood.

What about something like this:

class Name(NamedTuple): first: str last: str

d = NamedKeyDict(Named) d[first='david', last='mertz'] = 1_000_000 # dollars

Sure, maybe. But this is probably better as a dataclass nowadays. Actually, I'm not sure what NamedKeyDict is meant to do in your example.

On Thu, Aug 27, 2020 at 11:01 AM Ricky Teachey ricky@teachey.org wrote:

What about something like this:

...

...

class Name(NamedTuple): first: str last: str

d = NamedKeyDict(Named) d[first='david', last='mertz'] = 1_000_000 # dollars

right -- that would be a new custom class that took advantage of this feature.

Are you suggesting that the built in dict be extended to support this? I'm pretty sure Jonathan Fine did suggest that -- but I don't think that's a good idea myself.

-CHB

On Thu, Aug 27, 2020 at 2:27 PM Ricky Teachey ricky@teachey.org wrote:

On Thu, Aug 27, 2020 at 2:14 PM Christopher Barker pythonchb@gmail.com wrote:

...

On Thu, Aug 27, 2020 at 11:01 AM Ricky Teachey ricky@teachey.org wrote:

...

What about something like this:

...

...

class Name(NamedTuple): first: str last: str

d = NamedKeyDict(Named) d[first='david', last='mertz'] = 1_000_000 # dollars

right -- that would be a new custom class that took advantage of this feature.

Are you suggesting that the built in dict be extended to support this? I'm pretty sure Jonathan Fine did suggest that -- but I don't think that's a good idea myself.

-CHB

No not the built-in, but maybe as an addition to the collections module.

Actually it just occurred to me-- why limit the idea to namedtuples? The collections module could have a dict that maps the *args and **kwargs in the [ ] to any arbitrary callable:

class MapDict: def __init__(self, f, __dict=None, **kwargs): self.callable = f self._dict = {} __dict = {**__dict if __dict is not None else {}, **kwargs}

self.update(__dict) def __getitem__(self, key, **kwargs): args = self.translate_key(key) return self._dict[f(*args, **kwargs)] def __getitem__(self, key, value, **kwargs): args = self.translate_key(key) self._dict[f(*args, **kwargs)] = value def __delitem__(self, key, **kwargs): args = self.translate_key(key) del self._dict[f(*args, **kwargs)] def update(self, d, **kwargs): d.update(kwargs) for k,v in d.items(): self[k] = v def translate_key(self, key): try: ikey = iter(key) except TypeError: ikey = iter((key,)) return (*ikey,)

...

...

from main import MapDict resolved_paths = MapDict(lambda s, *, strict=True:

Path(s).resolve(strict=strict))

...

...

resolved_paths[r"spam/eggs/cheese", strict=True] = None

--- Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

On Thu, Aug 27, 2020 at 11:49:22PM +0900, Stephen J. Turnbull wrote:

Alex Hall writes:

...

OK, I'll try again. Do people want collections to get a standard 'default' for `__getitem__`?

I don't know.

I'm pretty sure that Guido said that he doesn't want the builtin dict and list types to get keyword arguments. That might change in the future, but I'm assuming that if keyword subscripts are supported, they will be added for the sake of projects like pandas, numpy, and xarray, not because there are plans to change the builtins.

...

I don't want mappings to grow a second way to do the same thing,

They're going to get one, is my reading of the "named indicies" thread. That is, it looks to me very likely that index notation (`x[i]`) is going to support keyword arguments (`x[i, k=v]`). If so, that's going to be a natural way to support a `default` argument.

It might have been the natural way to do it in 1991, if keyword subscripts existed back then. But they didn't, so now, in 2020, the natural way to do this in a dict is with the get method.

That's not going to go away.

I imagine some people will choose it because they like it, and if there are enough people who do there will be pressure for it to be TOOWTDI.

*shrug*

If keyword subscripts are allowed;

and if they become really popular outside of pandas etc;

and people start using the `mapping[key, default=0]` idiom in third party mappings;

and it becomes a defacto standard for mappings outside of the stdlib;

and people want builtin dict to support the same idiom;

then there might be some pressure on the core devs to add it to dict. But until such time, I wouldn't expect dicts to change.

So I think this issue is a distraction.

Unless the senior core devs like Guido, Brett, Nick, Serhiy, Raymond, etc (my apologies to anyone I missed) say that they're intending to add keywords to dict subscripting, I wouldn't expect the get method to become obsolete.

I think the problem with list.get is that there are already a plethora of ways to do it:

- Look Before You Leap:

# assumes index is non-negative default if len(L) < index else L[index]

- Easier to Ask Forgiveness than Permission:

try: L[index] except IndexError: default

- Slicing:

# likewise assumes index is non-negative (L[index:index+1] or [default])[0]

- any of which can be put into a helper function

get(L, index, default)

(Maybe add this to the operator module?)

So unless this happens to scratch somebody's personal itch, I think you're going to run up against the problem that while this is useful, it's maybe not useful enough to bother.

Personally, I ever so slightly lean towards the "if it already existed, I'd definitely use it, but I don't care enough to fight for it" position. I'm running out of energy to care about things I actually care about, and this is so far down the list that I'm honestly not sure why I'm even writing this :-)

Probably as an avoidance tactic to avoid doing other more important things *wink*

I just want to make clear that safe navegator is enough to deal with this.

Now that this is clear, the use case is almost always the same. I received some json as response and want to extract a nested value.

The way I'm doing this today is overloading an infix operator (I'm using

...

) to emulate safe navegator so I do

value = data >> get('foo') >> get(0) >> get('bar') if value: do something

This get can be found here https://github.com/dhilst/geckones_twitter_bots/blob/e6aefe036d30f8483502605...

I started with array implementation them extended to object too. I usually has something like this on all my projects that need to extract data from requests, or a function that receives a dotted path, anyway..

dict.get composes so nicely so why we don't have list.get too?

Well, the safe navigator can solve this as it will compose as well as any adhoc solution with the advantage of short circuit and also it's brevity so ... For my use case, at last, safe navigator (or null coalescing) will fit better than any list.get can do.

Regards

Em ter, 30 de jun de 2020 18:53, Juancarlo Añez apalala@gmail.com escreveu:

Hello!

Which would be the use cases for this feature?

I can't think of one.

I think that "nice to have" leads to the ways of Perl.

Regards,

On Sat, Jun 27, 2020 at 9:34 AM Daniel. danielhilst@gmail.com wrote:

...

When I need to traverse nested dicts, is a common pattern to do

somedict.get('foo', {}).get('bar', {})

But there is no such equivalent for arrays, wouldn't be nice if we can follow

somedict.get('foo', {}).get('bar', []).get(10) ... ?

What I do in this case is surround with try/except IndexError and set some variable to None, or wrap the try/except block in a function. But this is noise in my opinion, I just want to follow the same reasoning that I have with dicts: I'm searching for some path deep nested, and want a default if not found.

What do you think?

Regards

“If you're going to try, go all the way. Otherwise, don't even start. ..." Charles Bukowski _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LLK3EQ... Code of Conduct: http://python.org/psf/codeofconduct/

-- Juancarlo *Añez*

toolz.get_in is designed for exactly this.

https://toolz.readthedocs.io/en/latest/api.html#toolz.dicttoolz.get_in

The whole library is great!

Alex

On Wed, 1 Jul 2020 at 8:03 am, MRAB python@mrabarnett.plus.com wrote:

On 2020-06-30 23:26, Daniel. wrote:

...

I just want to make clear that safe navegator is enough to deal with

this.

...

Now that this is clear, the use case is almost always the same. I received some json as response and want to extract a nested value.

The way I'm doing this today is overloading an infix operator (I'm using

...

...

) to emulate safe navegator so I do

value = data >> get('foo') >> get(0) >> get('bar') if value: do something

This get can be found here

https://github.com/dhilst/geckones_twitter_bots/blob/e6aefe036d30f8483502605...

...

I started with array implementation them extended to object too. I usually has something like this on all my projects that need to extract data from requests, or a function that receives a dotted path, anyway..

dict.get composes so nicely so why we don't have list.get too?

Well, the safe navigator can solve this as it will compose as well as any adhoc solution with the advantage of short circuit and also it's brevity so ... For my use case, at last, safe navigator (or null coalescing) will fit better than any list.get can do.

[snip]

I'd probably just write a function for it:

def get(data, default, *indexes): try: for index in indexes: data = data[index] except (IndexError, KeyError): return default

value = get(data, None, 'foo', 0, 'bar') if value is not None: do something _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/A6YJUY... Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, Jun 30, 2020 at 8:55 PM David Lowry-Duda david@lowryduda.com wrote:

On the other hand, it seems far more likely to miss keys in a dictionary than it is to repeatedly mistake indices in a list.

exactly -- dict keys are arbitrary, and it's pretty common to store "sparse" data by simply leaving out the key if there's nothing interesting attached to it.

But for Sequences, the only time you get an index error is if you are indexing beyond the length of the list, and the only case I can think of for get() would be for a maybe-empty list. otherwise, would I really want the same thing, and no error, for ANY index larger than the length of the list?

Rather, if there's a specific index we want, we want it to be there, and if not, then we are iterating over it, which handles any length (including zero) just fine.

-CHB

Em qua, 1 de jul de 2020 00:56, David Lowry-Duda david@lowryduda.com escreveu:

Very similar things could be said about dict.get too, no? It's easy to write your own function that does the same thing or to catch an exception.

On the other hand, it seems far more likely to miss keys in a dictionary than it is to repeatedly mistake indices in a list.

...

Which would be the use cases for this feature?

I can't think of one.

I've never missed this feature either. But I would also be interested in hearing a dream use case.

The usecase is extracting deeply nested data from parsed json.

Missing keys are something that happens on dicts, I want something that is able to extract data from parsed json, that eliminates the error handling noise. All this talk elucidates me that this is possible

try: v = foo.bar['tar'][0]['zar'] except (LookupError, AttributeError): v = None

would suffice in all cases,

• DLD

-- David Lowry-Duda david@lowryduda.com <davidlowryduda.com> _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EUCNY7... Code of Conduct: http://python.org/psf/codeofconduct/

Em sex, 3 de jul de 2020 11:37, Kyle Lahnakoski klahnakoski@mozilla.com escreveu:

On 2020-06-27 09:34, Daniel. wrote:

...

When I need to traverse nested dicts, is a common pattern to do

somedict.get('foo', {}).get('bar', {})

But there is no such equivalent for arrays, wouldn't be nice if we can follow

somedict.get('foo', {}).get('bar', []).get(10) ... ?

What do you think?

I would use this. I do something similar already, albeit as a set of classes that wrap around Python `dict` and `list` to provide the null-safe access.

to_data(somedict).foo.bar[10]

Yeah, I just get the same idea but using rshift to emulate safe access infix operator

somedict >> get('foo')

Since I can't add behavior to exiting classes in python, I resort to things like that to "emulate" method calls. You can think

somelist >> get(0) as somelist.get(0)

So I don't have to deal with an extra type in translation when sending data back to a third party library

Specifically, I wrap `list` in `FlatList`, which will return `Null` (null-safe version of `None`) instead of raising and IndexError. This allows math on indexes without concern for corner cases, and makes window functions easier to write:

| my_list = FlatList(my_list) | deltas = [] | for i, this_week in enumerate(my_list): | last_week = my_list[i-7] | deltas.append(this_week - last_week)

by avoiding exception handling, code is simplified, and procedures simplify to functions. Instead of code that constructs a list; the code reads as a list definition:

| deltas = [ | this_week - last_week | for my_list in [FlatList(my_list)] | for i, this_week in enumerate(my_list) | for last_week in [my_list[i-7]] | ]

Yeah I feel that try/catch add lots of noise to null present code. We always tend to abstract null handling in some way and each one come up with its own implementation. The list.get method has the advatage, as Alex said, of already being a common dialect with dicts.

If dict.get didn't exist I probably wouldn't even asking for it, but every time that I need to handle json and I start with dict.get and hit an list I end up with an wrapper over IndexError.

try/catch are nice for medium blocks for single lines they are too many noise, you have 3 lines of error handling for one of real logic. (But you can wrap that in a function). Yeah, then you have to shift the reading flow to that function. When you see dict.get, you know what's happening, it's just straight to the point.

If we had the same method for list, our codes may look similar while solving almost the same problem (handling null). But since there isn't we create whole new abstractions that are completely different.

...

please forgive me: where `for x in [y]` can be read as `x=y` (I am

still hesitant to use `:=`)

There are detriments to using a set of null-safe objects:

1. data structures can be polluted with a combination of null-safe

objects and regular Python structures

2. with null-safe code, there are more nulls and it seems EVERYTHING

must be null-safe, including arithmetic (notice null-safe subtraction above)

It all depends of you're propagating None or not. And not if your code is None aware or not. But yeah, none aware code are easier to follow and write, at last for me, because the missing value handling is abstrated.

But still, I'm not talking about to add a method to propagate more Nones, no, even if this is what .get does. I'm asking for something that can be used in conjunction with dict.get to deal with nested data in an elegant and idiomatic form so we stop to ac-hoc abstracting IndexError and start to use a common idiom.

Another simple use case that I use this is to have give simple scripts a default command line argument, like

try: arg = sys.argv[1] except IndexError: arg = default

Or

if len(sys.argv) >= 2: arg = sys.argv[1] else: arg = default

Which would be rewritten as

arg = sys.argv.get(1, default)

I'm hopeful that there are more people with other uses case out there that didn't see the thread yet

3. runs slower; all the null-safe checks dominate my profiler time.

I would be nice if Python had a series of null-safe operators. Until then, `list.get` would eliminate my extra object creation:

I would really love it too, I was expecting to drop this in favor of pep 505 but as Alex pointed, it wouldn't handle IndexError case, so even with null operators we will be still having to handle with a except block

| deltas = [ | this_week - (last_week or 0) | for i, this_week in enumerate(my_list) | for last_week in [my_list.get(i-7)] | ]

Although we see additional null-checks required (as #2 states).

---

Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/UKU2PN... Code of Conduct: http://python.org/psf/codeofconduct/

Regards