
dict.get() is a very useful method, but for lists and tuples, we need to rely on try/except instead. Can we get list.get and tuple.get as well? Also, for list, a list.setdefault like the dict.setdefault would be logical.

On Tue, Feb 28, 2017 at 12:54:26PM +0100, Michel Desmoulin wrote:
dict.get() is a very useful method, but for lists and tuples, we need to rely on try/except instead.
No you don't. You can use slicing. alist = [1, 2, 3] print(alist[99:100]) # get the item at position 99 In my experience, dict.get is very useful, but list.get only *seems* useful. I've written my own version: def get(alist, pos, default=None): try: return alist[pos] except IndexError: return default but then struggled to find a good use for it. It seems like it ought to be useful, but in practice I found that it was only covering up bugs in my code. If I was indexing a list outside of the range of existing items, that's a bug, and using get() just made it hard to fix.
Can we get list.get and tuple.get as well?
Also, for list, a list.setdefault like the dict.setdefault would be logical.
What would it do? For example, given: alist = [] y = alist.setdefault(10, 'a') what will alist equal? -- Steve

Le 28/02/2017 à 15:45, Steven D'Aprano a écrit :
No this gives you a list of one item or an empty list. dict.get('key', default_value) let you get a SCALAR value, OR a default value if it doesn't exist. It's a very different use case.
Based on your rational, we would just reject dict.get as well the first time it's implemented.
How so ? "get the element x or a default value if it doesn't exist" seem at the contrary, a very robust approach. Plus it's consistent. It's only fair to expect it to exists after you learn about dict.get. First places where I missed it at the top of my head was *args, sys.argv personnaly. If I was indexing a list outside of the range of existing
Fair enough.

On Tue, Feb 28, 2017 at 7:16 AM, Michel Desmoulin <desmoulinmichel@gmail.com
wrote:
x = (alist[pos:pos+1] or [default_val])[0]
How so ? "get the element x or a default value if it doesn't exist" seem at the contrary, a very robust approach.
Yes, and easily written as above. What significant advantage would it have to spell the above as: x = alist.get(pos, default_val) It's a couple characters shorter in the proposed version. I guess I'll concede that needing the odd indexing at the end to get the scalar is slightly ugly. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 28.02.2017 18:18, David Mertz wrote:
1. advantage: it looks like dict access -> allows duck typing (oh how often I'd missed that) 2. advantage: no try except 3. advantage: no weird workaround with slices and additional item access Thanks for bringing this up, Michel. First point would be most important to me. Sven

On Tue, Feb 28, 2017 at 10:10 AM, Sven R. Kunze <srkunze@mail.de> wrote:
How often would you duck-type "access either an integer position or a named key in a collection?" I'm all for duck typing, but it feels like those are a pretty different pattern. For example, I know that if a list has something at index 10 it also has something at index 9. I absolutely *do not* know that if a dict has something at key 'g' it also has something at key 'f'. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

I wouldn't see myself using it for an arbitrary value in the middle of the list, but perhaps for the 0th or -1st (first/last) element of a list that might be empty. Maybe also second-to-first/last if I'm initializing some comparison between sequential items? Some of the examples don't work with negative indexes, boo. On Tue, Feb 28, 2017 at 12:52 PM, David Mertz <mertz@gnosis.cx> wrote:

On Tue, Feb 28, 2017 at 07:10:15PM +0100, Sven R. Kunze wrote:
1. advantage: it looks like dict access -> allows duck typing (oh how often I'd missed that)
Dicts and lists don't duck-type because they are very different things. We know what ["item"]*10 does. What would {"key": "value"}*10 do? What would list.values() iterate over? -- Steve

Le 01/03/2017 à 01:02, Steven D'Aprano a écrit :
The fact the API is not exactly the same doesn't prevent duck typing. Duck typing is precesily about incomplete but good enough similar API. For the dict and list: - you can iterate on both - you can index both - you can size both Hence I can see very well functions working with both. E.G: helper to extract x elements or a default value: def extract(data, *args, default="None"): for x in args: try: yield data[x] except (KeyError, ValueError): yield default Usage: a, b, c = extract(scores, "foo", "bar", "doh") x, y, z = extract(items, 2, 5, 8, default=0) I actually have this helper function. With list.get and tuple.get, this would become: def extract(data, *args, default="None"): return (data.get(x, default) for x in args)

On Wed, Mar 01, 2017 at 02:26:18AM +0100, Michel Desmoulin wrote:
The fact the API is not exactly the same doesn't prevent duck typing. Duck typing is precesily about incomplete but good enough similar API.
Indeed. But the relationship goes this way: # Duck-typing done the right way. Types P and Q both quack like ducks, so if all we need is something that quacks, either P or Q will do. not this way: Types P and Q both quack like ducks. I want something that swims like a duck, like P. Q doesn't swim at all, so we need to add Q.swim() so we can duck-type P or Q. If we are going to propose Q.swim(), it must be because it makes sense for Q instances to swim, regardless of what P does.
Right -- because all these operations make sense for both dicts and lists. Does get() make sense for both? It certainly makes sense for dicts. It makes *some* sense for lists, but (in my opinion) not enough to justify making it a built-in method of the type. Making it a built-in method isn't just a convenience, it is also blessing this as "the right thing to do". As I've said, in my experience trying to index into arbitrary positions of a sequence (list or tuple) without knowing whether that index exists or not is rarely the right thing to do. (That makes it very different from key lookup in a mapping or dict.) I believe that the way to argue for list.get() is not because it will make it easy to duck-type lists and dicts. It is (in my experience) very rare to need to duck-type lists and dicts. I believe you should identify code that handles lists that would benefit from this change. Under what circumstances do you ask for the 17th item of a list which may only contain 9 items? (For arbitrary values of 17 and 9.) -- Steve

Barry, you're taking the metaphor too far. Duct typing is about presenting a certain interface. If your function takes an object that has a get(key, default) method, the rest doesn't matter. That's the only way in which the object needs to resemble a duck in your function. I'd like to +1 this proposal. It should be trivial to implement. It won't break backward compatibility. It's intuitive. I can think of several places I would use it. I can't think of a good reason not to include it. On Wed, Mar 1, 2017 at 12:06 PM, Barry <barry@barrys-emacs.org> wrote:

On Wed, Mar 1, 2017 at 11:13 AM, Abe Dillon <abedillon@gmail.com> wrote:
I've yet to see in this thread a use case where list.get() would make sense. Specifically, I've yet to see a case where there is straightforward duck-typing substitutability between dicts and lists where you'd want that. I saw something that said "semi-structured data sources like JSON are often messy and you need lots of try/except blocks." But that's really not the same thing. Even if some node in a structure might variously be a list, dict, or scalar (or other types; sets?), I haven't seen any code where this hypothetical list.get() would improve that problem. In contrast, *iteration* is definitely a case where I often want to freely substitute lists, sets, dicts, and other collections. They share that natural capability, so being able to type `for x in collection:` is a good generic win to have. As I've said, the huge difference is that the "keys" to a list have a clear and obvious relationship amongst themselves. They are always successive non-negative integers, with the minimum always being zero. That makes a whole lot of operations and assumptions very different from dictionaries whose keys are completely independent of each other. If `mylist[N]` works, `mylist[N-1]` cannot fail with an IndexError (obviously unless the list is mutated in between those operations); there's nothing remotely analogous for dictionaries, and that's the reason we have dict.get(). -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 2 March 2017 at 07:03, David Mertz <mertz@gnosis.cx> wrote:
I've never wanted a `list.get`, but I have occassionally wished that: 1. operator.get/set/delitem were available as builtins (like get/set/delattr) 2. the builtin getitem accepted an optional "default" argument the way getattr does That is, the desired common operation isn't specifically "obj.get(subscript)" or "obj.get(subscript, default)", it's: _raise = object() def getitem(container, subscript, default=_raise): try: return container[subscript] except LookupError: if default is _raise: raise return default Mappings just happen to already offer that functionality as a method. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 1 Mar 2017, at 19:13, Abe Dillon <abedillon@gmail.com> wrote:
Barry, you're taking the metaphor too far. Duct typing is about presenting a certain interface. If your function takes an object that has a get(key, default) method, the rest doesn't matter. That's the only way in which the object needs to resemble a duck in your function.
In support of get() I found that list of ducks a poor agument for the reasons I stated. That's not to say that get() on lists may well have value for other reasons, but I find the duck typing a very weak argument.
I'd like to +1 this proposal. It should be trivial to implement. It won't break backward compatibility. It's intuitive. I can think of several places I would use it. I can't think of a good reason not to include it.
I do not think I have encounted any use cases where I would have used this my self. Maybe in command line processing, I have to dig in my repos and check. Might be able to short cut a length check with a get with default None or "". Barry

On Tue, Feb 28, 2017 at 5:26 PM, Michel Desmoulin <desmoulinmichel@gmail.com
wrote:
Duck typing is precesily about incomplete but good enough similar API.
yes, though ideally one API is a subset of the other -- if they have the same method, it should mean the same thing:
For the dict and list:
- you can iterate on both
But you get different things -- dicts iterate on the keys, which wold be the equivalent of lists iterating on the indexes -- no one wants that! - you can index both
the indexing is only kinda the same, though, and you certainly can't slice dicts...
- you can size both
huh? what does sizing mean? you mean get the length? OK, that's similar.
really? when would you not know if your "keys" are indexes or arbitrary keys? or your data a sequence or mapping? I actually have this helper function.
as a helper function, then it's OK if it's a bit more verbose. If you were to argue that you wouldn't need the helper function at all, then that might make sense, but this still seems a dangerous and hopefully rare thing to do! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Tue, 28 Feb 2017 at 17:19 David Mertz <mertz@gnosis.cx> wrote:
I think code like that is convoluted and confusing and I'm surprised to see anyone at all advocating it. IMO, the sane thing to compare this with is a conditional expression. There aren't any spellings of that that aren't ugly either:
stuff[x] if len(stuff) > x else default stuff[x] if stuff[x:x+1] else default
As for a reasonable use of list.get (or tuple.get), I often end up with lists of arguments and would like to take values from the list if they exist or take a default if not. This looks particularly horrible if the index isn't a variable (so most of the time): something = args[1] if len(args) > 1 else "cheese" something_else = args[2] if len(args) > 2 else "eggs" (you could make it more horrible by using the slicing trick, but I don't see much point in demonstrating that.) I don't often want to use dicts and lists in the same code in this way, but I think the crucial point about the comparison with dicts is that code like this is simpler and clearer if you do something horrible like this, just to get .get():
argdict = dict(enumerate(args))
Ed P.S. all the talk of PEP 463 seems misplaced. That it solves (FSVO solve) this problem doesn't mean it should supersede this discussion. Personally, I don't think I'd use except-expressions, but I would use list.get.

On 03/03/2017 08:09 AM, Ed Kellett wrote:
P.S. all the talk of PEP 463 seems misplaced. That it solves (FSVO solve) this problem doesn't mean it should supersede this discussion.
The advantage of PEP 463 is that issues like this would be less pressing, and it's much more general purpose. Personally, I don't think `get` belongs on list/tuple for reasons already stated. -- ~Ethan~

On Fri, 3 Mar 2017 at 17:03 Ethan Furman <ethan@stoneleaf.us> wrote:
PEP 463 won't solve this problem for me because its solution is as ugly as the thing it's replacing. Conceptually, I'd even argue that it's uglier. Also, if you want a general-purpose solution to everything, propose with-expressions, but that's another discussion. The existence of general-purpose things doesn't mean specific issues aren't worth talking about.
Personally, I don't think `get` belongs on list/tuple for reasons already stated.
The reasons already stated boil down to "lists aren't dicts so they shouldn't share methods", which seems ill-advised at best, and "I wouldn't use this". I'm not convinced that the latter is generally true; I've often looked for something like a list.get, been frustrated, and used one (chosen pretty much at random) of the ugly hacks presented in this thread. I'd be surprised if I'm the only one. I guess I don't have any hope of convincing people who think there's no need to ever do this, but I have a couple of questions for the people who think the existing solutions are fine: - Which of the existing things (slice + [default], conditional on a slice, conditional on a len() call) do you think is the obvious way to do it? - Are there any examples where list.get would be applicable and not the obviously best way to do it? Ed

On 03.03.2017 19:29, Ed Kellett wrote:
I wonder if those arguing against it also think dicts should not have item access: a[0] dict or list? Why should it matter? a.get(0, 'oops') Doesn't look so different to me.
You are not the only one. I share your sentiment.
None of them are. Try/except is the most obvious way. But it's tedious.
- Are there any examples where list.get would be applicable and not the obviously best way to do it?
I don't think so. I already have given many examples/ideas of when I would love to have had this ability. Let me re-state those and more: - refactoring (dicts <-> lists and their comprehension counterparts) - error-free accessing list comprehensions - duck typing - increased consistency of item access between dicts and lists - the one obvious way to do it - easier to teach Sven

On 03/03/2017 11:01 AM, Sven R. Kunze wrote:
On 03.03.2017 19:29, Ed Kellett wrote:
dicts don't have item access -- they have key access. :wink:
a[0]
dict or list? Why should it matter?
Because they are different data types with different purposes.
[my_value] = some_list[offset:offset+1] or [default_value] No, it's not terribly pretty, but accessing invalid locations on a list on purpose shouldn't be that common.
dict and list comprehensions are not the same, and adding .get to list won't make them the same.
- easier to teach
Having `print` be a statement instead of a function made it easier to teach but that didn't make it a good idea. For me to think (list/tuple).get() was needed would be if lots of folk either cast their lists to dicts or made their own list-dict class to solve that problem. -- ~Ethan~

On 03.03.2017 20:35, Ethan Furman wrote:
Python doesn't make a difference here. :wink: https://docs.python.org/3/reference/datamodel.html#object.__getitem__
When generating data series / running a simulation, at the beginning there is no data in many lists. Recently, had those issues. dicts went fine, lists just sucked with all those try/except blocks.
Never said they are the same. I said refactoring is easier.
Many people disagree with you on this.
The easier solution would be to provide list.get ;-) Regards, Sven

On Fri, Mar 3, 2017 at 12:02 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Exactly -- I think that was the point -- if there is a lot of custom code out there essentially adding a get() to a list -- then that would indicate that is is broadly useful. For my part, I think casting a list to a dict is often the RIGHT way to address these issues. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

About JSON and schema-less data: I need to deal with this fairly often as well, but: JSON has a data model that includes both mappings and sequences: Sequences (arrays, lists, etc) are the "right" thing to use when an object has zero or more of something. Usually, these somethings are all the same. So you may need to answer the question: how many somethings are there? but rarely: if there are less than this many somethings, then I should use a default value. Mappings (objects, dicts) are the "right" thing to do when an object has a bunch of somethings, and each of them may be different and nameable. In this case, the if this name is in there, use its associated object, otherwise use a default" is a pretty common action. so if your JSON is well formed (and I agree, being schema-less does not mean it is poorly formed) then it should already be using the appropriate data structures, and you are good to go. That being said, maybe a concrete example would persuade the skeptics among us -- though I understand it may be hard to find one that is both non-trivial and simple and small enough to post to a mailing list... -CHB On Fri, Mar 3, 2017 at 12:09 PM, Chris Barker <chris.barker@noaa.gov> wrote:
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Thanks Chris for your idea. Right now, I could not think of an example "non-trivial and simple and small enough" especially in the context of JSON. But maybe the other proponents have. The part of data series from simulations (so proper datastructures available). So, data lists which aren't filled yet or have not filled till a certain amount yet I need some special pieces from them like the first, a sample, the 300th, etc. This was the case recently. There was also the case of a refactoring going on in some project, where things changed from dicts to lists (upgrade of a third party lib, I think). As a consequence, I needed to blow up certain functions from n one-liners [a.get] to n four-liners [try/except]. If I had know that this would be relevant to this discussion, I would have written it down, but it's just the negative memory/experience. Regards, Sven On 03.03.2017 21:16, Chris Barker wrote:

On Fri, Mar 3, 2017 at 12:33 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Always a challenge -- sorry to lack imagination, I tend to need concrete examples to "get" some things. And so far the concrete examples in this thread seem to have been unconvincing... Which doesn't mean at all that there aren't good and common-enough use cases.. The part of data series from simulations (so proper datastructures
I deal with that a fair bit -- but in that case, if I need, say the 300th sample, and there are not yet 300 available, then that IS an Exception I want to handle. if it didn't need to be the 300th, but rather an random sample, or maybe one "half way through the data", or .... then I would compute that index from teh length or something... Though I'm probably misunderstanding this use case. There was also the case of a refactoring going on in some project, where
well, THAT I would blame on the third party lib..... :-) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On 03.03.2017 21:09, Chris Barker wrote:
You can't be serious about this. Especially because it would negate your response to Ed "conditional on a len() call is the way to go". Now you tell people to use "convert to dict". For my part, __getitem__ is the technical argument for this proposal. My experience and those of other contributors to this thread make for the "broadly useful" argument. Regards, Sven

On Fri, Mar 3, 2017 at 12:21 PM, Sven R. Kunze <srkunze@mail.de> wrote:
I am serious. It depends on the use case. If the data are an arbitrary-size collection of essentially the same thing, then a sequence is the right data structure, and examining len() (or catching the IndexError, maybe) is the right way to handle there being fewer than you expect of them. I think the key point is there there is nothing particularly different about them -- in fact, often order isn't important at all. If the data in question is a bunch of stuff where it matters where they land in the sequence, and there may be missing values (Like a row in a CSV file, maybe) then a dict IS the right structure -- even if it has integer keys. This reminds me of a discussion by Guido years ago about the "usual" use cases for lists vs tuples -- lists are often a homogenous sequence of items, whereas tuples are more likely to be heterogeneos -- more like a struct Now that I write that -- maybe the structure you really want is a namedtuple. and maybe IT should have a get() so as to avoid catching attribute errors all over teh place... though getattr() does have a default -- so maybe namedtuple IS the answer :-) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Le 03/03/2017 à 22:21, Chris Barker a écrit :
But that's the all problem isn't it? Since the start of the discussion, contesters have been offering numerous solutions, all being contextual and with gotchas, none being obvious, simple or elegant. The best is still try/except. "There should be one obvious way to do it" right? Plus Sven already estimated the implementation would not be very hard. So we have one obvious solution to a problem that: - several professional programmers said they have - has a similar API in another built-in - has currently no elegant solutions The proposal is actionable, the cost of it seems low, and it's not remotely controversial. I get that on Python-idea you get "no" by default, but here we are having such resistance for a feature that is light to implement, does not clutter anything, does solve a problem, and is congruent with other APIs. Honestly what evil would happen if it's get accepted ? This is not a "yeah but we can't accept everything that goes in or we would bloat Python" thing. If somebody tells me that no one want to spend time to code it, I can understand. Everybody's has a life, and somebody else's pony can wait. And since I can't code in C I can't do it. But that doesn't seem to be the problem here.

On Fri, Mar 3, 2017 at 1:35 PM, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
Which really isn't a big deal if you use it in one or two places. If you use it everywhere, it's not too hard to roll your own helper function.
You state that like it's a good thing ;-). I'm not quite so sure.
It seems to be pretty controversial to me :-).
Honestly what evil would happen if it's get accepted ?
Lots of things. For one thing, when scanning a function, if I see something with a `.get` method I generally think that it is probably a Mapping. Obviously that assumption may be wrong, but it's at least a good place to start. If this gets accepted, that's no longer a clear starting assumption. It breaks backward compatibility (in small ways). People might be relying on the presence/absence of a `.get` method in order to make their function polymorphic in some convoluted way which would break when this change is introduced. (I'm not saying that would be a good programming design idea -- and it might not hold water as an argument when real-world usage is looked at but it should be at least investigated before we claim that this change isn't going to hurt anybody). It's also not clear to me why we should be singling out `tuple` and `list`. Why not `str`, `bytes` and other sequences? Maybe it isn't that useful on those types, but I'd argue that it's not really useful on `tuple` either other than to keep the "`tuple` is an immutable `list`" paradigm.

On Fri, Mar 3, 2017 at 1:35 PM, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
I am serious. It depends on the use case. If the data are an
But that's the all problem isn't it? Since the start of the discussion, contesters have been offering numerous solutions, all being contextual and with gotchas, none being obvious, simple or elegant. in the context above, I was offering that there were obvious, simple and elegant solutions, but that which one was dependent on the use case. EVERY choice in programming is dependent on the use case. What I haven't seen yet is a compelling use case for a sequence .get() that does not have an existing simple and elegant solution. which doesn't mean they don't exist. (and for the my part, the machinations with or shortcutting are not, in my book, simple or elegant...) Plus Sven already estimated the implementation would not be very hard. ... The proposal is actionable, the cost of it seems low, This would not simply be adding one method to a class. It would be adding a method to the Sequence protocol ( ABC, whatever you want call it). So a much heavier lift and larger impact than you imply. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Hi, I'm still new here, but if my vote has any value then this gets a heavy -1 from me. It makes no sense to have to access exact i:th element of a list without knowing if it exists, at least not in a scenario where checking against the length or using an exception (say CSV row should have index 2 but doesn't) wouldn't be better. I might've missed a message or two, but unless someone can provide a real example where get() has an actual use case, I see no reason to argue over this. - Markus

On Fri, Mar 03, 2017 at 10:35:18PM +0100, Michel Desmoulin wrote:
I do not agree with that characterisation.
The best is still try/except.
And I don't agree with that either.
"There should be one obvious way to do it" right?
But what is "it" here? Don't say "look up an arbitrary-indexed item which may not exist from a sequence". That's too general, and in the most general case, the right way to do that is to use sequence[index] which will raise if the item doesn't exist. In other words, the status quo. Be specific. Show some code -- its okay if its simplified code, but it should be enough to demonstrate the *use-case* for this. "I have crappy JSON" is not a use-case. How is it crappy and how would you use list.get to fix it? This brings us back to the point I made really early on: this *seems* like an obviously useful method, by analogy with dicts. I agree! It *seems* useful, so obviously such that one of the first things I added to my own personal toolbox of helper functions was a sequence get() function: def get(sequence, index, default=None): try: return sequence[index] except IndexError: return default But then I never used it. "Seems useful" != "is useful", at least in my experience.
Plus Sven already estimated the implementation would not be very hard.
The simplicity of the implementation argues *against* the need for this to be a built-in. If you really do need this, then why not add a sequence get() function to your project? Its only five lines! As far as I have seen, only one person apart from myself, Kyle Lahnakoski, has implemented this helper in their own code. And Kyle says he has talked himself out of supporting this change. One thing I haven't seen is anyone saying "I am constantly writing and re-writing this same helper function over and over again! I grepped my code base and I've recreated this helper in 30 different modules. Maybe it should be a built-in?" That would be a good argument, but nobody has made it. Lots of people saying that they desperately need this method, but apparently most of them don't need it enough to write a five line helper function to get it. They'd rather wait until they've migrated all their code to Python 3.7.
So we have one obvious solution to a problem that:
- several professional programmers said they have
I'm not convinced by claims that "I need to fetch arbitrary indexes from sequences ALL THE TIME, sorry I can't show any examples..."
It is only "not remotely controversial" if you ignore all those who disagree that this is needed.
Yes it is. But fundamentally, although I really don't see the benefit to this, I'm not *strongly* against it either. I don't think the sky will fall if it is added to sequences. But if somebody wants to code this up (don't forget the Sequence ABC) and submit a patch or a PR for a senior developer to look up, I'm not going to deny you that opportunity, -- Steve

On Sat, 4 Mar 2017 at 09:46 Steven D'Aprano <steve@pearwood.info> wrote: On Fri, Mar 03, 2017 at 10:35:18PM +0100, Michel Desmoulin wrote:
I do not agree with that characterisation. Which solution do you think is obvious, simple or elegant?
"There should be one obvious way to do it" right?
But what is "it" here? ... Any of the alternatives mentioned thus far solve the "too general" problem. As far as I can see, none of them would even be a contender if list.get existed, so I think the fact that people have spent time coming up with them is telling. In other words, the status quo. Be specific. Show some code -- its okay if its simplified code, but it should be enough to demonstrate the *use-case* for this. "I have crappy JSON" is not a use-case. How is it crappy and how would you use list.get to fix it? It's crappy because you have a list of things of unknown length (though I'm inclined to disagree with the "crappy", honestly—this would seem a perfectly reasonable thing to do if people weren't bizarrely against it). I haven't written much Python that is not secret for a while, so excuse the heavy paraphrasing: In my cases it is usually dealing with arguments: some sort of list has been sent to me down the wire, let's say ["kill", "edk"]. I could write a big fancy dispatcher, but I see that as unwarranted complexity, so I usually start with something simpler. I end up wanting to do something like: target = args[1] reason = args.get(2, "<No reason given>") I could equally use list.pop, which seems to be common when solving this problem with dicts—except list.pop doesn't take a default either, for some reason I've never quite understood. This brings us back to the point I made really early on: this *seems* like an obviously useful method, by analogy with dicts. I agree! It *seems* useful, so obviously such that one of the first things I added to my own personal toolbox of helper functions was a sequence get() function: def get(sequence, index, default=None): try: return sequence[index] except IndexError: return default But then I never used it. "Seems useful" != "is useful", at least in my experience. Helper functions suck, in my view. I'm all for a flat function namespace with a get() that works on anything, but that one doesn't, and "dict.get is for dicts and get is for sequences" seems ugly.
Plus Sven already estimated the implementation would not be very hard.
The simplicity of the implementation argues *against* the need for this to be a built-in. If you really do need this, then why not add a sequence get() function to your project? Its only five lines! Why not: - Python tries very hard to stop you from adding get() to sequences. Mixing levels of namespacing feelss wrong, to me. - It's another function for everybody reading your program to have to remember about. - Functions have other costs too, in terms of documentation and testing. - It's likely incompatible with other copies of the same utility. As far as I have seen, only one person apart from myself, Kyle Lahnakoski, has implemented this helper in their own code. And Kyle says he has talked himself out of supporting this change. One thing I haven't seen is anyone saying "I am constantly writing and re-writing this same helper function over and over again! I grepped my code base and I've recreated this helper in 30 different modules. Maybe it should be a built-in?" That would be a good argument, but nobody has made it. Lots of people saying that they desperately need this method, but apparently most of them don't need it enough to write a five line helper function to get it. They'd rather wait until they've migrated all their code to Python 3.7. The helper function doesn't solve the problem for me. The existing solutions are, to my mind, ugly and non-obvious, and writing a helper that is still ugly and non-obvious doesn't make anything better. The place to solve this problem is in the API.
So we have one obvious solution to a problem that:
- several professional programmers said they have
I'm not convinced by claims that "I need to fetch arbitrary indexes from sequences ALL THE TIME, sorry I can't show any examples... It's hard to show examples because, generally speaking, when one can't do a thing one does something else instead. I can restructure my programs to avoid having this problem, or, if I'm in a hurry, I can use one of the many ugly h^W^Wobvious, simple and elegant solutions, like (args[2:3] + ["<No reason given>"])[0]. In general, I remain curious about cases in which list.get could be used and would not be the preferred solution. Ed

On Sun, Mar 5, 2017 at 4:51 AM, Ed Kellett <edk141@gmail.com> wrote:
For the record, even though I was the first in this thread to give that spelling, I don't think it's the best way to spell it in current Python. Bracketing a helper function (which *could* after all deal with lists, dicts, and whatever other type you wanted depending on what you implement), I think the best spelling is with a ternary: reason = args[1] if len(args)>1 else "<No reason given>" The more the thread continues the more I actively want to avoid a list.get() method. Initially I thought it added symmetry; but as I look at it I realize it is mostly code smell. Being able to get "a value" from an arbitrary position in a list that isn't long enough often suggests something is deeply wrong in the logic of the code. Moreover, it is likely to let bugs pass silently and cause deeper problems elsewhere downstream. If you have a list that is expected to have a length of either 1 or 2, I can imagine this making sense (e.g. ["kill"] vs. ["kill", "edk"]): reason = args.get(1, "<No reason given>") But if the next line is: data = args.get(17, "<empty>") Then I'm pretty sure the programmer thinks she's being passed a very different type of collection than is actually available. I'd rather that fails right away and in an obvious way then silently produce a value. Specifically, if I think I'm dealing with a list that is likely to have 20 items (rather than maybe 4 or fewer), I'm almost sure the best way to deal with it is in a list (or comprehension, map(), etc) and NOT by poking into large index positions that may or may not be present. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, Mar 5, 2017 at 10:13 AM, David Mertz <mertz@gnosis.cx> wrote:
I meant "in a LOOP" above. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, 5 Mar 2017 at 18:13 David Mertz <mertz@gnosis.cx> wrote:
That's up to the programmer. args[17] exists and does fail immediately. If the programmer provides a default value, presumably they know they want one.
I really think that depends what it's a list of. If the positions of things in the list are important (as with an argument parser, or perhaps a lookup table) I fail to see why it would be wrong to peek. If lists were really designed to be used only as you and some others in this thread are suggesting, I don't think they'd have indexed access at all. Ed

On Sun, Mar 5, 2017 at 11:22 AM, Ed Kellett <edk141@gmail.com> wrote:
That's up to the programmer. args[17] exists and does fail immediately. If the programmer provides a default value, presumably they know they want one.
In terms of an actual use case, I can see it for "Lists no longer than 4". Any other use of this hypothetical method would be an anti-pattern and be a bad habit. Yes, programmers can do what they want, but providing a method is a hint to users (especially beginners, but not only) that that is the "right way" to do it.
But the positions NEVER are important when you get to a 20 item list. If you design an argument parser that is looking for "the 17th argument" you are doing it wrong. I'm not saying that's impossible (nor even hard) to program, but it's not good practice. Sure, I'm happy to take 20+ arguments, especially if they result from a glob pattern used at the command line, naming files. But when I'm doing that, I want to deal with those filenames in a loop, handling each one as necessary. In that pattern, I *never* want exactly 20 arguments, but rather "however many things there are to handle."
If lists were really designed to be used only as you and some others in this thread are suggesting, I don't think they'd have indexed access at all.
Actually, when I teach I make a big point of telling students (for me, professional scientists and programmers who have used other PLs) that if they are indexing a list they should look again and question whether that's the right pattern. Of course there are times when it's needed, but they are fewer than C, Java, or Fortran programmers think. If this method existed, I'd want it implemented roughly like this: In [1]: class GetList(list): ...: def get(self, i, default=None): ...: if i > 4: ...: raise NotImplemented("You should NOT use this method for long lists!") ...: try: ...: return self[i] ...: except IndexError: ...: return default ...: In [2]: l = GetList(['err','my message']) In [3]: l.get(1, 'no message') Out[3]: 'my message' In [4]: l.get(2, 'no details') Out[4]: 'no details' In [5]: l.get(10, 'some data') --------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-5-3306fdc8b5a0> in <module>() ----> 1 l.get(10, 'some data') <ipython-input-1-441d5e3eda9f> in get(self, i, default) 2 def get(self, i, default=None): 3 if i > 4: ----> 4 raise NotImplemented("You should NOT use this method for long lists!") 5 try: 6 return self[i] TypeError: 'NotImplementedType' object is not callable -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, 5 Mar 2017 at 19:54 David Mertz <mertz@gnosis.cx> wrote:
In terms of an actual use case, I can see it for "Lists no longer than 4".
That's an excessively hard limit.
Any other use of this hypothetical method would be an anti-pattern
What really is the point of this? You (not uniquely) have been quick to dismiss any uses for this as misguided. If you must do that, you might stick to the reasons; meaningless labels don't add anything.
That's probably true of argument parsers, certainly not lookup tables.
I don't think that assessment applies neatly everywhere. Indexing is generally unnecessary when it's being used instead of iteration, but this thread is explicitly about cases where iteration isn't wanted. Ed

On Sun, Mar 5, 2017 at 12:16 PM, Ed Kellett <edk141@gmail.com> wrote:
That's an excessively hard limit.
Maybe 5... in special circumstances :-)
I can think of a few special cases where index positions are useful. But they aren't common enough to warrant a new method, nor hard to do with the existing language. E.g.: for i, data in enumerate(base_data): extra = extra_data[i] if len(extra_data) > i else DEFAULT combine(data, extra) This might well use lists thousands of items long, and maybe `extra_data` runs out before `base_data`. That code would look very slightly nicer with `extra_data.get()`. On the other hand, better than either is: from itertools import zip_longest for data, extra in zip_longest(base, extra_data, fillvalue=DEFAULT): combine(data, extra) So far no one in this thread has presented any (non-trivial) code that would be better if `list.get()` existed. I think I have personally come closest, but I actively want it not to happen because it's an anti-pattern. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 05/03/2017 20:16, Ed Kellett wrote:
Patches are always welcome. If you insist that it's needed, you do the work. Hopefully it's easier with the move to github. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence

On Fri, Mar 3, 2017 at 11:01 AM, Sven R. Kunze <srkunze@mail.de> wrote:
Because a mapping that happens to have an integer key is a fundamentally different thing than a sequence. It seems to me that in some of the examples given, the use case really calls for a mapping-with-integers-as-keys, rather than a sequence -- in which case, you could use that, and have the get() :-) And then you could iterate through it the same way, too! you could get closer to sequence behvior by used a OrderedDict -- or maybe even write a SortedDict that would keep the keys in order regardless of when they were added.
a.get(0, 'oops') Doesn't look so different to me.
but you are going to have issue with other things anyway, notable: for thing in a: # is thing a key or a value????? - Which of the existing things (slice + [default], conditional on a slice,
I guess that's the key point for me -- in all teh examples I seen posed (like parsing args) the sequence may contain from n to n+m items, but a get() doesn' really solve your problem, because the default is probably different depending on how MANY of the possible items are "missing". So get(0 is only helpful if: 1) there is only one possible missing item 2) all the missing items have the same default -- unless that default is sometign like None, in which case you are simply replacing one way to express missing with another, I can't see that being common. So I would expect to see a Sequence /get(0 be used to slightly clean up the code that adds a bunch of Nones to sequences to make them all the same length -- which is actually pretty easy to do anyway: seq = seq + [None] * full_len - len(seq) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On 3 March 2017 at 18:29, Ed Kellett <edk141@gmail.com> wrote:
Write a small wrapper function that implements the functionality (however you want, but it doesn't have to be a single expression, so you've much more flexibility to make it readable) and then use that.
- Are there any examples where list.get would be applicable and not the obviously best way to do it?
I don't understand the question. If you're asking which is better between list.get and a custom written function as described above, then a custom written function is better because (a) it works on all Python versions, (b) list.get needs a language change where a helper function doesn't, (c) "writing a helper function" is a generally useful idiom that works for many, many things, but list.get only solves a single problem and every other such problem would need its own separate language change. The disadvantage that you have to write the helper is trivial, because it's only a few lines of simple code: def get_listitem(lst, n, default=None): try: return lst[n] except IndexError: return default Paul

On Sun, 5 Mar 2017 at 11:27 Paul Moore <p.f.moore@gmail.com> wrote:
It's hardly a question of readability at that point. A reader is at the very least going to have to look at the signature of the utility function in order to be sure about argument order.
No. I'm asking: if list.get did exist, are there any cases (compatibility with old versions aside) where list.get's semantics would be applicable, but one of the alternatives would be the better choice? "writing a helper function" is a generally
Custom helper functions can obviously accomplish anything in any language. If we had to choose between def: and list.get, I'd obviously opt for the former. The disadvantage that you have to write
the helper is trivial, because it's only a few lines of simple code:
I don't think the size of a helper function is relevant to how much of a disadvantage it is. Most built-in list methods are trivial to implement in Python, but I'm glad not to have to. Ed

On 5 March 2017 at 13:03, Ed Kellett <edk141@gmail.com> wrote:
Self-evidently no. But what does that prove? That we should implement list.get? You could use the dientical argument for *anything*. There needs to be another reason for implementing it.
But you have yet to explain why you'd be glad not to write a helper for list.get, in any terms that don't simply boil down to "someone else did the work for me". I think we're going to have to just disagree. You won't convince me it's worth adding list.get unless you can demonstrate some *existing* costs that would be removed by adding list.get, and showing that they are greater than the costs of adding list.get (search this list if you don't know what those costs are - I'm not going to repeat them again, but they are *not* trivial). And I don't seem to be able to convince you that writing a helper function is a reasonable approach. Paul.

On Sun, 5 Mar 2017 at 14:08 Paul Moore <p.f.moore@gmail.com> wrote:
I don't think that's true. It's not true for many other list methods, for example. It's not a reason for implementing it, but it does suggest to me that it would increase the one-obvious-way-to-do-it-ness of the language. But you have yet to explain why you'd be glad not to write a helper
for list.get, in any terms that don't simply boil down to "someone else did the work for me".
What? Five lines isn't work, it's just ugly. I don't want to add a lot of random utility functions like this because it drastically increases the effort required to read my code. It's hardly any effort, but it doesn't solve any problems, so why bother? The point about this as a Python change is that it's a standard. Who does the work couldn't be less relevant; what matters is that it would add a consistent and easy spelling for something that doesn't have one.
They seem to be "it'd need to be added to Sequence too" and "it would mess with code that checks for a .get method to determine whether something is a mapping". It's easily implementable in Sequence as a mixin method, so I'm prepared to call that trivial, and I'm somewhat sceptical that the latter code does—let alone should—exist.
And I don't seem to be able to convince you that writing a helper function is a reasonable approach.
I feel like I'm saying this a lot, but writing helper functions has its own readability cost. I'm not trying to get anyone to implement list.get, I'm trying to get it centrally documented and allowed into list's overly-mappingproxied namespace. Ed

On Mon, Mar 6, 2017 at 6:13 AM, Ed Kellett <edk141@gmail.com> wrote:
Oh, absolutely! Because a language totally needs to have a consistent and easy spelling for everything. How about reading one line of text from a compressed file and stripping HTML tags from it? http://php.net/manual/en/function.gzgetss.php Not everything has to be a single function/method. Some things are allowed to be more than one. To justify list.get(), you have to show that it's actually worth adding to the language, and so far, all you've said is "but wouldn't it be nice". Show us code that would be improved by this method. SHOW US CODE.
Classes don't generally inherit from Sequence, though. You can't simply add methods to these kinds of protocols. ChrisA

On 5 March 2017 at 19:13, Ed Kellett <edk141@gmail.com> wrote:
You didn't seem to find the post(s) I referred to. I did a search for you. Here's one of the ones I was talking about - https://mail.python.org/pipermail/python-ideas/2017-February/044807.html You need to present sufficient benefits for list.get to justify all of the costs discussed there - or at least show some understanding of those costs and an appreciation that you're asking *someone* to pay those costs if you expect a proposal to add *anything* to the Python language seriously. But I quit at this point - you seem intent on not appreciating the other sides of this argument, so there's not really much point continuing. Paul PS And yes, I do appreciate your point here - a get method on lists may be useful. And helpers (if you don't name them well, for instance) aren't always the best solution. But I've never yet seen *any* code that would be improved by using a list.get method, so although I understand the argument in theory, I don't see the practical benefits.

On 03/05/2017 11:13 AM, Ed Kellett wrote:
I'm not trying to get anyone to implement list.get, I'm trying to get it centrally documented and allowed into list's overly-mappingproxied namespace.
--> dir(list) # non dunder methods 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort' --> dir(dict) # non dunder methods 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values' I will admit I'm not entirely sure what you meant with that last statement, but it sounds like you think `list` has a bunch of the same methods as `dict` does. I count 11 methods on dict, 11 methods on list, and only 3 methods that are shared (1 of which doesn't work precisely the same on both). So I'm not seeing a bunch of similarity between the two. And I'm not seeing any progress here in this discussion, so I'm dropping out now. -- ~Ethan~

On Tue, Feb 28, 2017 at 04:16:28PM +0100, Michel Desmoulin wrote:
I am aware of that. I'm just saying you don't have to use try...except, you can use slicing instead. Obviously you have to adapt the code since you are getting a list not a single item: # may fail, if alist is empty if alist[0] == "spam": ... # cannot fail if alist[0:1] == ["spam"]: ... This is *especially* useful for strings since a slice of a string is a string, and an item from a string is still a string.
For a default, use "or": first_item = (alist[0:1] or ["ham"])[0] But honestly, in my experience the number of times I actually needed something like that is tiny.
I just said that dict.get is proven to be useful. There's no doubt that it is useful.
A very robust approach **for what** ? What are you trying to do? Why are you indexing into arbitrary positions of a list without knowing whether or not there is something there? List are not dicts and they are used differently. It is very common, and useful, to want to look up a key in a dict without knowing if it exists, and do something with a default if it doesn't exist. This is so useful that Python gives us *at least* four different ways to avoid using a try...except: - dict.get - dict.setdefault - dict subclasses with __missing__ defined - collections.defaultdict all of which solve slightly different use-cases. But *in my experience* it is rare to need to look up some arbitrary item in a list that may not even exist, and if I find myself doing so, it probably means my code is badly designed and I'm going to have trouble later on. For the tiny number of exceptions, I can use the existing solutions: either try...except, or slicing as above. If your experience is different from mine, please explain your use-case.
Plus it's consistent. It's only fair to expect it to exists after you learn about dict.get.
As quoted in PEP 8, "A foolish consistency is the bugbear of little minds." Lists and dicts aren't the same thing and don't offer the same interface. There's no dict.sort or list.update, dicts don't support slicing, concatenation or repetition, there's no list.popitem. Dicts have keys and values, lists have items or elements. list.get has to prove its usefulness on its own, not just because dict.get is useful.
First places where I missed it at the top of my head was *args, sys.argv personnaly.
Missed it for what? What are you trying to do? Don't assume that it is so obvious that everyone will instantly guess your use-case. -- Steve

...
first_item = (alist[0:1] or ["ham"])[0]
Come on, I've been doing Python for more than a decade and never saw anybody doing that. Even reading it in a code would make me scratch my head for a moment with a "what is it doing that for?". You are trying to hard to provide a counter argument here.
But honestly, in my experience the number of times I actually needed something like that is tiny.
...
Maybe your missions involve working with people doing properly their job. Me, I have to deal SOAP government systems, mongodb based API built by teenagers, geographer data set exports and FTP + CSV in marina systems (which I happen to work on right now). 3rd party CSV, XML and JSON processing are just a hundred of lines of try/except on indexing because they have many listings, data positions is important and a lot of system got it wrong, giving you inconsistent output with missing data and terrible labeling. And because life is unfair, the data you can extract is often a mix of heterogeneous mappings and lists / tuples. And your tool must manage the various versions of the data format they send to you, some with additional fields, or missing ones. Some named, other found by position. This summer, I had to convert a data set provided by polls in africa through an android form, generated from an XML schema, send as json using Ajax, then stored in mongodb... to an excel spread sheet (and also an HTML table and some JS graphs for good measure). Needingless to say I dealt with a looooot of IndexError. Grepping the project gives me: grep -R IndexError | wc -l 33 In contrast I have 32 KeyError (most of them to allow lazy default value), and 3 decorators. If special syntax exist for decorators, then surely we can spare a .get() for lists and tuples. Apparently IndexError is an important error because if I grep the virtualenv of the project: grep -R IndexError | wc -l 733 Ok, it's a pretty large project with 154 dependancies, but it's still almost 7 IndexError by package on average. So it's not a rare use case. I also see it regularly in my classes. Students try it because they learned it works with dict. It makes sense. Don't dismiss a use case because you don't have it. Python is so versatile it's used in many diverse areas.

On Tue, Feb 28, 2017 at 5:56 PM, Michel Desmoulin <desmoulinmichel@gmail.com
wrote:
I feel your pain -- data munging is often a major mess!
If I were dealing with a mix of mappings and index-able data, and the index-able data were often poorly formed (items missing), I think I"d put it all in dicts -- some of which happened to have integers as keys. Or just put a None in everywhere there should be a value in a sequence that is missing. if data is coming in from a "schema-less" system, then what CAN you do with a sequence that is inconsistent? How can yo possible know which are missing if the sequence is too short? If it is always the last N times then nit' snot hard to pad the sequence. if it's not -- then what you have is a mapping that happens to have integers as keys. Not trying to be harsh here -- I'm just not at all sure that adding a get() to sequences is the right solution to these problems. Maybe someone else will chime in with more "I'd really have a use for this" examples. -CHB This summer, I had to convert a data set provided by polls in africa
through an android form, generated from an XML schema,
Actually, I'm surprised that the XML schema step didn't enforce that the data be well formed. ISn't that the whole point of an XML schema? -- but you're point is well taken -- data are often not well formed. -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Wed, Mar 01, 2017 at 02:56:44AM +0100, Michel Desmoulin wrote:
These days, it might be better to write it as: first_item = alist[0] if len(alist) else "ham" but I remember the days when slicing was normal and using the `or` trick was standard operating procedure.
You are trying to hard to provide a counter argument here.
In context, all I'm saying is that you don't *have* to catch IndexError. There are alternatives. That is all.
This is all very well and good, and I feel your pain for having to deal with garbage data, but I don't see how this helps you. You talk about missing data, but lists cannot contain missing data from the middle. There's no such thing as a list like: [0, 1, 2, 3, , , , , , , 10, 11, 12] where alist[3] and alist[10] will succeed but alist[4] etc will raise IndexError. So I'm still trying to understand what this proposal gets you that wouldn't be better solved using (say) itertools.zip_longest or a pre-processing step to clean up your data.
Maybe I'm underestimating just how awful your data is, but I'm having difficulty thinking of a scenario where you don't know what kind of object you are processing and have to write completely type-agnostic code: for key_or_index in sequence_of_keys_or_indexes: result = sequence_or_mapping[key_or_index] I'm sure that there is lots of code where you iterate over dicts: for key in keys: result = mapping.get(key, default) and likewise code where you process lists: for i in indexes: try: result = sequence[i] except IndexError: result = default # could be re-written using a helper function: for i in indexes: result = get(sequence, i default) but I've never come across a data-processing situation where I didn't know which was which. That second version with the helper function would be *marginally* nicer written as a method call. I grant you that!
So 33 is "a looooot", but 32 KeyErrors is "in contrast" and presumably a little.
You don't know that every one of those can be replaced by list.get(). Some of them might be raising IndexError; some of them may be documenting that a function or method raises IndexError; etc.
I also see it regularly in my classes. Students try it because they learned it works with dict. It makes sense.
I never said it didn't. But I wonder whether it gives *enough* benefit to be worth while. -- Steve

I love this proposal but Guido rejected it. Fighting for it right now would probably be detrimental to the current proposed feature which could potentially be more easily accepted. At least let's make it a separate thread. I would love to restart the debate about this one. This is one of my most wanted feature ever in Python. Le 01/03/2017 à 01:22, Rob Cliffe a écrit :

Le 01/03/2017 à 02:23, Ethan Furman a écrit :
The debate is not even over and you are already declaring a winner. That's not really fair. Give the idea a chance and read until the end. D'Aprano's argument is mostly "I don't encounter IndexError really often and when I do I have this twisted one liner to get away it". Well, that's not really a good reason to reject things for Python because it's a language with a very diverse user base. Some bankers, some web dev, some geographers, some mathematicians, some students, some 3D graphists, etc. And the language value obvious, readable, predictable code for all. Most people on this list have a specialty, because their speciality don't see a use for the feature doesn't mean there is not one. So I provided on my last answer an explanation of what I would use it for.

On 02/28/2017 06:02 PM, Michel Desmoulin wrote:
Le 01/03/2017 à 02:23, Ethan Furman a écrit :
On 02/28/2017 05:18 PM, Michel Desmoulin wrote:
True, but this means that a feature needs to apply to more than one group of folks. While I sympathize (truly!) with the nightmare you have to deal with, I don't think (and I certainly hope) that that quagmire is not common enough to justify adding .get() to list/tuple. An idea has to be useful for more than a small group of code bases to warrant inclusion in the stdlib, and even more so for a built-in.
On the bright side, if enough use-cases of this type come up (pesky try/except for a simple situation), we may be able to get Guido to reconsider PEP 463. I certainly think PEP 463 makes a lot more sense that adding list.get(). -- ~Ethan~

On 03.03.2017 18:06, Ethan Furman wrote:
Totally true. I think both proposals have their merit. IIRC, Guido rightfully declared that try/except expressions aren't a good idea. It's better to find more concrete patterns instead of it. And I still agree with him. The "default parameter" pattern is such a pattern, and it's vastly used in the stdlib. Sven

On 03/03/2017 10:48 AM, Sven R. Kunze wrote:
On 03.03.2017 18:06, Ethan Furman wrote:
On 03/02/2017 12:36 PM, Sven R. Kunze wrote:
$ grep "def get(" *.py */*.py */*/*.py queue.py: def get(self, block=True, timeout=None): pickle.py: def get(self, i): shelve.py: def get(self, key, default=None): doctest.py: def get(self): mailbox.py: def get(self, key, default=None): weakref.py: def get(self, key, default=None): weakref.py: def get(self, key, default=None): sre_parse.py: def get(self): webbrowser.py: def get(using=None): tkinter/ttk.py: def get(self, x=None, y=None): configparser.py: def get(self, section, option, *, raw=False, vars=None, fallback=_UNSET): configparser.py: def get(self, option, fallback=None, *, raw=False, vars=None, _impl=None, **kwargs): email/message.py: def get(self, name, failobj=None): asyncio/queues.py: def get(self): logging/config.py: def get(self, key, default=None): idlelib/pyparse.py: def get(self, key, default=None): wsgiref/headers.py: def get(self,name,default=None): xml/dom/minidom.py: def get(self, name, value=None): _collections_abc.py: def get(self, key, default=None): tkinter/__init__.py: def get(self): tkinter/__init__.py: def get(self): tkinter/__init__.py: def get(self): tkinter/__init__.py: def get(self): tkinter/__init__.py: def get(self): tkinter/__init__.py: def get(self): tkinter/__init__.py: def get(self, first, last=None): tkinter/__init__.py: def get(self): tkinter/__init__.py: def get(self): tkinter/__init__.py: def get(self, index1, index2=None): tkinter/__init__.py: def get(self, x, y): tkinter/__init__.py: def get(self): xml/sax/xmlreader.py: def get(self, name, alternative=None): collections/__init__.py: def get(self, key, default=None): idlelib/scrolledlist.py: def get(self, index): idlelib/searchengine.py: def get(root): multiprocessing/pool.py: def get(self, timeout=None): xml/etree/ElementTree.py: def get(self, key, default=None): multiprocessing/queues.py: def get(self, block=True, timeout=None): multiprocessing/queues.py: def get(self): multiprocessing/managers.py: def get(self): multiprocessing/managers.py: def get(self): idlelib/idle_test/mock_tk.py: def get(self): idlelib/idle_test/mock_tk.py: def get(self, index1, index2=None): I wouldn't consider 10 out of 43 "vastly" (11 out of 46 if one includes dict, list, and tuple). The numbers are even worse if one considers the "get_something_or_other" methods which do not have a default parameter. -- ~Ethan~

I must mention a get() method for lists and tuples would be very useful for me too. It is so useful, that I spent too much time making my own module to handle this case, plus many of the other dealing-with-None subjects found on this list. Michel is correct to point out that this is domain specific problem; a domain where you deal with many varied, and schema-free, data formats. I deal with JSON emitted from multiple systems of often-changing schemas. In my experience, this is not a result of bad or buggy programming, rather, it is about representing facts and annotating them with a multitude of optional properties and descriptive structures. Now, in the specific case of list.get(), I would be disappointed that it is used to extract parameters from an arg list: Parameters should be named; packing them into an ordered list looses that important information, but it happens[1], and list.get() would help. For the args scenario, I do like Ed's solution: dict(enumerate(args)). In conclusion, I may have talked myself out of liking list.get(): Python has a fundamentally different philosophy about None that conflicts with what I need for my domain [2] where I am transforming and interpreting data. Using a set of classes that make a different set of assumptions about None is not arduous, it keeps the definitions separate, and I still get all wonderfulness of Python. [1] also happens when reading csv files: Missing values indicate default, or variable number of columns indicate that the missing rightmost columns are all null. [2] For Python, None is a missing value, or a special case. For data transformation, None means "the operation you performed does not apply to this datatype" which avoids exceptions, which gives you an algebra over data (with [], dot and slice as operators), which allows you to build complex list comprehensions (data transformation queries) without the exception catching logic. Databases query languages do this. On 2017-02-28 21:02, Michel Desmoulin wrote:

On Fri, Mar 3, 2017 at 11:15 AM, Kyle Lahnakoski <klahnakoski@mozilla.com> wrote:
not really related to this thread, but you may want to use your own "sentinal" singletons, rather than special case code to deal with None. i.e. BadData MissingData UnSpecified M.A. Lemberg has been talking about that on this list (in this thread? I've lost track...) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Fri, Mar 3, 2017 at 12:06 PM, Chris Barker <chris.barker@noaa.gov> wrote:
M.A. Lemberg has been talking about that on this list (in this thread? I've lost track...)
it was in the "Optional parameters without default value" thread. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Tue, Feb 28, 2017 at 12:54:26PM +0100, Michel Desmoulin wrote:
dict.get() is a very useful method, but for lists and tuples, we need to rely on try/except instead.
No you don't. You can use slicing. alist = [1, 2, 3] print(alist[99:100]) # get the item at position 99 In my experience, dict.get is very useful, but list.get only *seems* useful. I've written my own version: def get(alist, pos, default=None): try: return alist[pos] except IndexError: return default but then struggled to find a good use for it. It seems like it ought to be useful, but in practice I found that it was only covering up bugs in my code. If I was indexing a list outside of the range of existing items, that's a bug, and using get() just made it hard to fix.
Can we get list.get and tuple.get as well?
Also, for list, a list.setdefault like the dict.setdefault would be logical.
What would it do? For example, given: alist = [] y = alist.setdefault(10, 'a') what will alist equal? -- Steve

Le 28/02/2017 à 15:45, Steven D'Aprano a écrit :
No this gives you a list of one item or an empty list. dict.get('key', default_value) let you get a SCALAR value, OR a default value if it doesn't exist. It's a very different use case.
Based on your rational, we would just reject dict.get as well the first time it's implemented.
How so ? "get the element x or a default value if it doesn't exist" seem at the contrary, a very robust approach. Plus it's consistent. It's only fair to expect it to exists after you learn about dict.get. First places where I missed it at the top of my head was *args, sys.argv personnaly. If I was indexing a list outside of the range of existing
Fair enough.

On Tue, Feb 28, 2017 at 7:16 AM, Michel Desmoulin <desmoulinmichel@gmail.com
wrote:
x = (alist[pos:pos+1] or [default_val])[0]
How so ? "get the element x or a default value if it doesn't exist" seem at the contrary, a very robust approach.
Yes, and easily written as above. What significant advantage would it have to spell the above as: x = alist.get(pos, default_val) It's a couple characters shorter in the proposed version. I guess I'll concede that needing the odd indexing at the end to get the scalar is slightly ugly. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 28.02.2017 18:18, David Mertz wrote:
1. advantage: it looks like dict access -> allows duck typing (oh how often I'd missed that) 2. advantage: no try except 3. advantage: no weird workaround with slices and additional item access Thanks for bringing this up, Michel. First point would be most important to me. Sven

On Tue, Feb 28, 2017 at 10:10 AM, Sven R. Kunze <srkunze@mail.de> wrote:
How often would you duck-type "access either an integer position or a named key in a collection?" I'm all for duck typing, but it feels like those are a pretty different pattern. For example, I know that if a list has something at index 10 it also has something at index 9. I absolutely *do not* know that if a dict has something at key 'g' it also has something at key 'f'. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

I wouldn't see myself using it for an arbitrary value in the middle of the list, but perhaps for the 0th or -1st (first/last) element of a list that might be empty. Maybe also second-to-first/last if I'm initializing some comparison between sequential items? Some of the examples don't work with negative indexes, boo. On Tue, Feb 28, 2017 at 12:52 PM, David Mertz <mertz@gnosis.cx> wrote:

On Tue, Feb 28, 2017 at 07:10:15PM +0100, Sven R. Kunze wrote:
1. advantage: it looks like dict access -> allows duck typing (oh how often I'd missed that)
Dicts and lists don't duck-type because they are very different things. We know what ["item"]*10 does. What would {"key": "value"}*10 do? What would list.values() iterate over? -- Steve

Le 01/03/2017 à 01:02, Steven D'Aprano a écrit :
The fact the API is not exactly the same doesn't prevent duck typing. Duck typing is precesily about incomplete but good enough similar API. For the dict and list: - you can iterate on both - you can index both - you can size both Hence I can see very well functions working with both. E.G: helper to extract x elements or a default value: def extract(data, *args, default="None"): for x in args: try: yield data[x] except (KeyError, ValueError): yield default Usage: a, b, c = extract(scores, "foo", "bar", "doh") x, y, z = extract(items, 2, 5, 8, default=0) I actually have this helper function. With list.get and tuple.get, this would become: def extract(data, *args, default="None"): return (data.get(x, default) for x in args)

On Wed, Mar 01, 2017 at 02:26:18AM +0100, Michel Desmoulin wrote:
The fact the API is not exactly the same doesn't prevent duck typing. Duck typing is precesily about incomplete but good enough similar API.
Indeed. But the relationship goes this way: # Duck-typing done the right way. Types P and Q both quack like ducks, so if all we need is something that quacks, either P or Q will do. not this way: Types P and Q both quack like ducks. I want something that swims like a duck, like P. Q doesn't swim at all, so we need to add Q.swim() so we can duck-type P or Q. If we are going to propose Q.swim(), it must be because it makes sense for Q instances to swim, regardless of what P does.
Right -- because all these operations make sense for both dicts and lists. Does get() make sense for both? It certainly makes sense for dicts. It makes *some* sense for lists, but (in my opinion) not enough to justify making it a built-in method of the type. Making it a built-in method isn't just a convenience, it is also blessing this as "the right thing to do". As I've said, in my experience trying to index into arbitrary positions of a sequence (list or tuple) without knowing whether that index exists or not is rarely the right thing to do. (That makes it very different from key lookup in a mapping or dict.) I believe that the way to argue for list.get() is not because it will make it easy to duck-type lists and dicts. It is (in my experience) very rare to need to duck-type lists and dicts. I believe you should identify code that handles lists that would benefit from this change. Under what circumstances do you ask for the 17th item of a list which may only contain 9 items? (For arbitrary values of 17 and 9.) -- Steve

Barry, you're taking the metaphor too far. Duct typing is about presenting a certain interface. If your function takes an object that has a get(key, default) method, the rest doesn't matter. That's the only way in which the object needs to resemble a duck in your function. I'd like to +1 this proposal. It should be trivial to implement. It won't break backward compatibility. It's intuitive. I can think of several places I would use it. I can't think of a good reason not to include it. On Wed, Mar 1, 2017 at 12:06 PM, Barry <barry@barrys-emacs.org> wrote:

On Wed, Mar 1, 2017 at 11:13 AM, Abe Dillon <abedillon@gmail.com> wrote:
I've yet to see in this thread a use case where list.get() would make sense. Specifically, I've yet to see a case where there is straightforward duck-typing substitutability between dicts and lists where you'd want that. I saw something that said "semi-structured data sources like JSON are often messy and you need lots of try/except blocks." But that's really not the same thing. Even if some node in a structure might variously be a list, dict, or scalar (or other types; sets?), I haven't seen any code where this hypothetical list.get() would improve that problem. In contrast, *iteration* is definitely a case where I often want to freely substitute lists, sets, dicts, and other collections. They share that natural capability, so being able to type `for x in collection:` is a good generic win to have. As I've said, the huge difference is that the "keys" to a list have a clear and obvious relationship amongst themselves. They are always successive non-negative integers, with the minimum always being zero. That makes a whole lot of operations and assumptions very different from dictionaries whose keys are completely independent of each other. If `mylist[N]` works, `mylist[N-1]` cannot fail with an IndexError (obviously unless the list is mutated in between those operations); there's nothing remotely analogous for dictionaries, and that's the reason we have dict.get(). -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 2 March 2017 at 07:03, David Mertz <mertz@gnosis.cx> wrote:
I've never wanted a `list.get`, but I have occassionally wished that: 1. operator.get/set/delitem were available as builtins (like get/set/delattr) 2. the builtin getitem accepted an optional "default" argument the way getattr does That is, the desired common operation isn't specifically "obj.get(subscript)" or "obj.get(subscript, default)", it's: _raise = object() def getitem(container, subscript, default=_raise): try: return container[subscript] except LookupError: if default is _raise: raise return default Mappings just happen to already offer that functionality as a method. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 1 Mar 2017, at 19:13, Abe Dillon <abedillon@gmail.com> wrote:
Barry, you're taking the metaphor too far. Duct typing is about presenting a certain interface. If your function takes an object that has a get(key, default) method, the rest doesn't matter. That's the only way in which the object needs to resemble a duck in your function.
In support of get() I found that list of ducks a poor agument for the reasons I stated. That's not to say that get() on lists may well have value for other reasons, but I find the duck typing a very weak argument.
I'd like to +1 this proposal. It should be trivial to implement. It won't break backward compatibility. It's intuitive. I can think of several places I would use it. I can't think of a good reason not to include it.
I do not think I have encounted any use cases where I would have used this my self. Maybe in command line processing, I have to dig in my repos and check. Might be able to short cut a length check with a get with default None or "". Barry

On Tue, Feb 28, 2017 at 5:26 PM, Michel Desmoulin <desmoulinmichel@gmail.com
wrote:
Duck typing is precesily about incomplete but good enough similar API.
yes, though ideally one API is a subset of the other -- if they have the same method, it should mean the same thing:
For the dict and list:
- you can iterate on both
But you get different things -- dicts iterate on the keys, which wold be the equivalent of lists iterating on the indexes -- no one wants that! - you can index both
the indexing is only kinda the same, though, and you certainly can't slice dicts...
- you can size both
huh? what does sizing mean? you mean get the length? OK, that's similar.
really? when would you not know if your "keys" are indexes or arbitrary keys? or your data a sequence or mapping? I actually have this helper function.
as a helper function, then it's OK if it's a bit more verbose. If you were to argue that you wouldn't need the helper function at all, then that might make sense, but this still seems a dangerous and hopefully rare thing to do! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Tue, 28 Feb 2017 at 17:19 David Mertz <mertz@gnosis.cx> wrote:
I think code like that is convoluted and confusing and I'm surprised to see anyone at all advocating it. IMO, the sane thing to compare this with is a conditional expression. There aren't any spellings of that that aren't ugly either:
stuff[x] if len(stuff) > x else default stuff[x] if stuff[x:x+1] else default
As for a reasonable use of list.get (or tuple.get), I often end up with lists of arguments and would like to take values from the list if they exist or take a default if not. This looks particularly horrible if the index isn't a variable (so most of the time): something = args[1] if len(args) > 1 else "cheese" something_else = args[2] if len(args) > 2 else "eggs" (you could make it more horrible by using the slicing trick, but I don't see much point in demonstrating that.) I don't often want to use dicts and lists in the same code in this way, but I think the crucial point about the comparison with dicts is that code like this is simpler and clearer if you do something horrible like this, just to get .get():
argdict = dict(enumerate(args))
Ed P.S. all the talk of PEP 463 seems misplaced. That it solves (FSVO solve) this problem doesn't mean it should supersede this discussion. Personally, I don't think I'd use except-expressions, but I would use list.get.

On 03/03/2017 08:09 AM, Ed Kellett wrote:
P.S. all the talk of PEP 463 seems misplaced. That it solves (FSVO solve) this problem doesn't mean it should supersede this discussion.
The advantage of PEP 463 is that issues like this would be less pressing, and it's much more general purpose. Personally, I don't think `get` belongs on list/tuple for reasons already stated. -- ~Ethan~

On Fri, 3 Mar 2017 at 17:03 Ethan Furman <ethan@stoneleaf.us> wrote:
PEP 463 won't solve this problem for me because its solution is as ugly as the thing it's replacing. Conceptually, I'd even argue that it's uglier. Also, if you want a general-purpose solution to everything, propose with-expressions, but that's another discussion. The existence of general-purpose things doesn't mean specific issues aren't worth talking about.
Personally, I don't think `get` belongs on list/tuple for reasons already stated.
The reasons already stated boil down to "lists aren't dicts so they shouldn't share methods", which seems ill-advised at best, and "I wouldn't use this". I'm not convinced that the latter is generally true; I've often looked for something like a list.get, been frustrated, and used one (chosen pretty much at random) of the ugly hacks presented in this thread. I'd be surprised if I'm the only one. I guess I don't have any hope of convincing people who think there's no need to ever do this, but I have a couple of questions for the people who think the existing solutions are fine: - Which of the existing things (slice + [default], conditional on a slice, conditional on a len() call) do you think is the obvious way to do it? - Are there any examples where list.get would be applicable and not the obviously best way to do it? Ed

On 03.03.2017 19:29, Ed Kellett wrote:
I wonder if those arguing against it also think dicts should not have item access: a[0] dict or list? Why should it matter? a.get(0, 'oops') Doesn't look so different to me.
You are not the only one. I share your sentiment.
None of them are. Try/except is the most obvious way. But it's tedious.
- Are there any examples where list.get would be applicable and not the obviously best way to do it?
I don't think so. I already have given many examples/ideas of when I would love to have had this ability. Let me re-state those and more: - refactoring (dicts <-> lists and their comprehension counterparts) - error-free accessing list comprehensions - duck typing - increased consistency of item access between dicts and lists - the one obvious way to do it - easier to teach Sven

On 03/03/2017 11:01 AM, Sven R. Kunze wrote:
On 03.03.2017 19:29, Ed Kellett wrote:
dicts don't have item access -- they have key access. :wink:
a[0]
dict or list? Why should it matter?
Because they are different data types with different purposes.
[my_value] = some_list[offset:offset+1] or [default_value] No, it's not terribly pretty, but accessing invalid locations on a list on purpose shouldn't be that common.
dict and list comprehensions are not the same, and adding .get to list won't make them the same.
- easier to teach
Having `print` be a statement instead of a function made it easier to teach but that didn't make it a good idea. For me to think (list/tuple).get() was needed would be if lots of folk either cast their lists to dicts or made their own list-dict class to solve that problem. -- ~Ethan~

On 03.03.2017 20:35, Ethan Furman wrote:
Python doesn't make a difference here. :wink: https://docs.python.org/3/reference/datamodel.html#object.__getitem__
When generating data series / running a simulation, at the beginning there is no data in many lists. Recently, had those issues. dicts went fine, lists just sucked with all those try/except blocks.
Never said they are the same. I said refactoring is easier.
Many people disagree with you on this.
The easier solution would be to provide list.get ;-) Regards, Sven

On Fri, Mar 3, 2017 at 12:02 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Exactly -- I think that was the point -- if there is a lot of custom code out there essentially adding a get() to a list -- then that would indicate that is is broadly useful. For my part, I think casting a list to a dict is often the RIGHT way to address these issues. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

About JSON and schema-less data: I need to deal with this fairly often as well, but: JSON has a data model that includes both mappings and sequences: Sequences (arrays, lists, etc) are the "right" thing to use when an object has zero or more of something. Usually, these somethings are all the same. So you may need to answer the question: how many somethings are there? but rarely: if there are less than this many somethings, then I should use a default value. Mappings (objects, dicts) are the "right" thing to do when an object has a bunch of somethings, and each of them may be different and nameable. In this case, the if this name is in there, use its associated object, otherwise use a default" is a pretty common action. so if your JSON is well formed (and I agree, being schema-less does not mean it is poorly formed) then it should already be using the appropriate data structures, and you are good to go. That being said, maybe a concrete example would persuade the skeptics among us -- though I understand it may be hard to find one that is both non-trivial and simple and small enough to post to a mailing list... -CHB On Fri, Mar 3, 2017 at 12:09 PM, Chris Barker <chris.barker@noaa.gov> wrote:
-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Thanks Chris for your idea. Right now, I could not think of an example "non-trivial and simple and small enough" especially in the context of JSON. But maybe the other proponents have. The part of data series from simulations (so proper datastructures available). So, data lists which aren't filled yet or have not filled till a certain amount yet I need some special pieces from them like the first, a sample, the 300th, etc. This was the case recently. There was also the case of a refactoring going on in some project, where things changed from dicts to lists (upgrade of a third party lib, I think). As a consequence, I needed to blow up certain functions from n one-liners [a.get] to n four-liners [try/except]. If I had know that this would be relevant to this discussion, I would have written it down, but it's just the negative memory/experience. Regards, Sven On 03.03.2017 21:16, Chris Barker wrote:

On Fri, Mar 3, 2017 at 12:33 PM, Sven R. Kunze <srkunze@mail.de> wrote:
Always a challenge -- sorry to lack imagination, I tend to need concrete examples to "get" some things. And so far the concrete examples in this thread seem to have been unconvincing... Which doesn't mean at all that there aren't good and common-enough use cases.. The part of data series from simulations (so proper datastructures
I deal with that a fair bit -- but in that case, if I need, say the 300th sample, and there are not yet 300 available, then that IS an Exception I want to handle. if it didn't need to be the 300th, but rather an random sample, or maybe one "half way through the data", or .... then I would compute that index from teh length or something... Though I'm probably misunderstanding this use case. There was also the case of a refactoring going on in some project, where
well, THAT I would blame on the third party lib..... :-) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On 03.03.2017 21:09, Chris Barker wrote:
You can't be serious about this. Especially because it would negate your response to Ed "conditional on a len() call is the way to go". Now you tell people to use "convert to dict". For my part, __getitem__ is the technical argument for this proposal. My experience and those of other contributors to this thread make for the "broadly useful" argument. Regards, Sven

On Fri, Mar 3, 2017 at 12:21 PM, Sven R. Kunze <srkunze@mail.de> wrote:
I am serious. It depends on the use case. If the data are an arbitrary-size collection of essentially the same thing, then a sequence is the right data structure, and examining len() (or catching the IndexError, maybe) is the right way to handle there being fewer than you expect of them. I think the key point is there there is nothing particularly different about them -- in fact, often order isn't important at all. If the data in question is a bunch of stuff where it matters where they land in the sequence, and there may be missing values (Like a row in a CSV file, maybe) then a dict IS the right structure -- even if it has integer keys. This reminds me of a discussion by Guido years ago about the "usual" use cases for lists vs tuples -- lists are often a homogenous sequence of items, whereas tuples are more likely to be heterogeneos -- more like a struct Now that I write that -- maybe the structure you really want is a namedtuple. and maybe IT should have a get() so as to avoid catching attribute errors all over teh place... though getattr() does have a default -- so maybe namedtuple IS the answer :-) -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Le 03/03/2017 à 22:21, Chris Barker a écrit :
But that's the all problem isn't it? Since the start of the discussion, contesters have been offering numerous solutions, all being contextual and with gotchas, none being obvious, simple or elegant. The best is still try/except. "There should be one obvious way to do it" right? Plus Sven already estimated the implementation would not be very hard. So we have one obvious solution to a problem that: - several professional programmers said they have - has a similar API in another built-in - has currently no elegant solutions The proposal is actionable, the cost of it seems low, and it's not remotely controversial. I get that on Python-idea you get "no" by default, but here we are having such resistance for a feature that is light to implement, does not clutter anything, does solve a problem, and is congruent with other APIs. Honestly what evil would happen if it's get accepted ? This is not a "yeah but we can't accept everything that goes in or we would bloat Python" thing. If somebody tells me that no one want to spend time to code it, I can understand. Everybody's has a life, and somebody else's pony can wait. And since I can't code in C I can't do it. But that doesn't seem to be the problem here.

On Fri, Mar 3, 2017 at 1:35 PM, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
Which really isn't a big deal if you use it in one or two places. If you use it everywhere, it's not too hard to roll your own helper function.
You state that like it's a good thing ;-). I'm not quite so sure.
It seems to be pretty controversial to me :-).
Honestly what evil would happen if it's get accepted ?
Lots of things. For one thing, when scanning a function, if I see something with a `.get` method I generally think that it is probably a Mapping. Obviously that assumption may be wrong, but it's at least a good place to start. If this gets accepted, that's no longer a clear starting assumption. It breaks backward compatibility (in small ways). People might be relying on the presence/absence of a `.get` method in order to make their function polymorphic in some convoluted way which would break when this change is introduced. (I'm not saying that would be a good programming design idea -- and it might not hold water as an argument when real-world usage is looked at but it should be at least investigated before we claim that this change isn't going to hurt anybody). It's also not clear to me why we should be singling out `tuple` and `list`. Why not `str`, `bytes` and other sequences? Maybe it isn't that useful on those types, but I'd argue that it's not really useful on `tuple` either other than to keep the "`tuple` is an immutable `list`" paradigm.

On Fri, Mar 3, 2017 at 1:35 PM, Michel Desmoulin <desmoulinmichel@gmail.com> wrote:
I am serious. It depends on the use case. If the data are an
But that's the all problem isn't it? Since the start of the discussion, contesters have been offering numerous solutions, all being contextual and with gotchas, none being obvious, simple or elegant. in the context above, I was offering that there were obvious, simple and elegant solutions, but that which one was dependent on the use case. EVERY choice in programming is dependent on the use case. What I haven't seen yet is a compelling use case for a sequence .get() that does not have an existing simple and elegant solution. which doesn't mean they don't exist. (and for the my part, the machinations with or shortcutting are not, in my book, simple or elegant...) Plus Sven already estimated the implementation would not be very hard. ... The proposal is actionable, the cost of it seems low, This would not simply be adding one method to a class. It would be adding a method to the Sequence protocol ( ABC, whatever you want call it). So a much heavier lift and larger impact than you imply. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Hi, I'm still new here, but if my vote has any value then this gets a heavy -1 from me. It makes no sense to have to access exact i:th element of a list without knowing if it exists, at least not in a scenario where checking against the length or using an exception (say CSV row should have index 2 but doesn't) wouldn't be better. I might've missed a message or two, but unless someone can provide a real example where get() has an actual use case, I see no reason to argue over this. - Markus

On Fri, Mar 03, 2017 at 10:35:18PM +0100, Michel Desmoulin wrote:
I do not agree with that characterisation.
The best is still try/except.
And I don't agree with that either.
"There should be one obvious way to do it" right?
But what is "it" here? Don't say "look up an arbitrary-indexed item which may not exist from a sequence". That's too general, and in the most general case, the right way to do that is to use sequence[index] which will raise if the item doesn't exist. In other words, the status quo. Be specific. Show some code -- its okay if its simplified code, but it should be enough to demonstrate the *use-case* for this. "I have crappy JSON" is not a use-case. How is it crappy and how would you use list.get to fix it? This brings us back to the point I made really early on: this *seems* like an obviously useful method, by analogy with dicts. I agree! It *seems* useful, so obviously such that one of the first things I added to my own personal toolbox of helper functions was a sequence get() function: def get(sequence, index, default=None): try: return sequence[index] except IndexError: return default But then I never used it. "Seems useful" != "is useful", at least in my experience.
Plus Sven already estimated the implementation would not be very hard.
The simplicity of the implementation argues *against* the need for this to be a built-in. If you really do need this, then why not add a sequence get() function to your project? Its only five lines! As far as I have seen, only one person apart from myself, Kyle Lahnakoski, has implemented this helper in their own code. And Kyle says he has talked himself out of supporting this change. One thing I haven't seen is anyone saying "I am constantly writing and re-writing this same helper function over and over again! I grepped my code base and I've recreated this helper in 30 different modules. Maybe it should be a built-in?" That would be a good argument, but nobody has made it. Lots of people saying that they desperately need this method, but apparently most of them don't need it enough to write a five line helper function to get it. They'd rather wait until they've migrated all their code to Python 3.7.
So we have one obvious solution to a problem that:
- several professional programmers said they have
I'm not convinced by claims that "I need to fetch arbitrary indexes from sequences ALL THE TIME, sorry I can't show any examples..."
It is only "not remotely controversial" if you ignore all those who disagree that this is needed.
Yes it is. But fundamentally, although I really don't see the benefit to this, I'm not *strongly* against it either. I don't think the sky will fall if it is added to sequences. But if somebody wants to code this up (don't forget the Sequence ABC) and submit a patch or a PR for a senior developer to look up, I'm not going to deny you that opportunity, -- Steve

On Sat, 4 Mar 2017 at 09:46 Steven D'Aprano <steve@pearwood.info> wrote: On Fri, Mar 03, 2017 at 10:35:18PM +0100, Michel Desmoulin wrote:
I do not agree with that characterisation. Which solution do you think is obvious, simple or elegant?
"There should be one obvious way to do it" right?
But what is "it" here? ... Any of the alternatives mentioned thus far solve the "too general" problem. As far as I can see, none of them would even be a contender if list.get existed, so I think the fact that people have spent time coming up with them is telling. In other words, the status quo. Be specific. Show some code -- its okay if its simplified code, but it should be enough to demonstrate the *use-case* for this. "I have crappy JSON" is not a use-case. How is it crappy and how would you use list.get to fix it? It's crappy because you have a list of things of unknown length (though I'm inclined to disagree with the "crappy", honestly—this would seem a perfectly reasonable thing to do if people weren't bizarrely against it). I haven't written much Python that is not secret for a while, so excuse the heavy paraphrasing: In my cases it is usually dealing with arguments: some sort of list has been sent to me down the wire, let's say ["kill", "edk"]. I could write a big fancy dispatcher, but I see that as unwarranted complexity, so I usually start with something simpler. I end up wanting to do something like: target = args[1] reason = args.get(2, "<No reason given>") I could equally use list.pop, which seems to be common when solving this problem with dicts—except list.pop doesn't take a default either, for some reason I've never quite understood. This brings us back to the point I made really early on: this *seems* like an obviously useful method, by analogy with dicts. I agree! It *seems* useful, so obviously such that one of the first things I added to my own personal toolbox of helper functions was a sequence get() function: def get(sequence, index, default=None): try: return sequence[index] except IndexError: return default But then I never used it. "Seems useful" != "is useful", at least in my experience. Helper functions suck, in my view. I'm all for a flat function namespace with a get() that works on anything, but that one doesn't, and "dict.get is for dicts and get is for sequences" seems ugly.
Plus Sven already estimated the implementation would not be very hard.
The simplicity of the implementation argues *against* the need for this to be a built-in. If you really do need this, then why not add a sequence get() function to your project? Its only five lines! Why not: - Python tries very hard to stop you from adding get() to sequences. Mixing levels of namespacing feelss wrong, to me. - It's another function for everybody reading your program to have to remember about. - Functions have other costs too, in terms of documentation and testing. - It's likely incompatible with other copies of the same utility. As far as I have seen, only one person apart from myself, Kyle Lahnakoski, has implemented this helper in their own code. And Kyle says he has talked himself out of supporting this change. One thing I haven't seen is anyone saying "I am constantly writing and re-writing this same helper function over and over again! I grepped my code base and I've recreated this helper in 30 different modules. Maybe it should be a built-in?" That would be a good argument, but nobody has made it. Lots of people saying that they desperately need this method, but apparently most of them don't need it enough to write a five line helper function to get it. They'd rather wait until they've migrated all their code to Python 3.7. The helper function doesn't solve the problem for me. The existing solutions are, to my mind, ugly and non-obvious, and writing a helper that is still ugly and non-obvious doesn't make anything better. The place to solve this problem is in the API.
So we have one obvious solution to a problem that:
- several professional programmers said they have
I'm not convinced by claims that "I need to fetch arbitrary indexes from sequences ALL THE TIME, sorry I can't show any examples... It's hard to show examples because, generally speaking, when one can't do a thing one does something else instead. I can restructure my programs to avoid having this problem, or, if I'm in a hurry, I can use one of the many ugly h^W^Wobvious, simple and elegant solutions, like (args[2:3] + ["<No reason given>"])[0]. In general, I remain curious about cases in which list.get could be used and would not be the preferred solution. Ed

On Sun, Mar 5, 2017 at 4:51 AM, Ed Kellett <edk141@gmail.com> wrote:
For the record, even though I was the first in this thread to give that spelling, I don't think it's the best way to spell it in current Python. Bracketing a helper function (which *could* after all deal with lists, dicts, and whatever other type you wanted depending on what you implement), I think the best spelling is with a ternary: reason = args[1] if len(args)>1 else "<No reason given>" The more the thread continues the more I actively want to avoid a list.get() method. Initially I thought it added symmetry; but as I look at it I realize it is mostly code smell. Being able to get "a value" from an arbitrary position in a list that isn't long enough often suggests something is deeply wrong in the logic of the code. Moreover, it is likely to let bugs pass silently and cause deeper problems elsewhere downstream. If you have a list that is expected to have a length of either 1 or 2, I can imagine this making sense (e.g. ["kill"] vs. ["kill", "edk"]): reason = args.get(1, "<No reason given>") But if the next line is: data = args.get(17, "<empty>") Then I'm pretty sure the programmer thinks she's being passed a very different type of collection than is actually available. I'd rather that fails right away and in an obvious way then silently produce a value. Specifically, if I think I'm dealing with a list that is likely to have 20 items (rather than maybe 4 or fewer), I'm almost sure the best way to deal with it is in a list (or comprehension, map(), etc) and NOT by poking into large index positions that may or may not be present. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, Mar 5, 2017 at 10:13 AM, David Mertz <mertz@gnosis.cx> wrote:
I meant "in a LOOP" above. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, 5 Mar 2017 at 18:13 David Mertz <mertz@gnosis.cx> wrote:
That's up to the programmer. args[17] exists and does fail immediately. If the programmer provides a default value, presumably they know they want one.
I really think that depends what it's a list of. If the positions of things in the list are important (as with an argument parser, or perhaps a lookup table) I fail to see why it would be wrong to peek. If lists were really designed to be used only as you and some others in this thread are suggesting, I don't think they'd have indexed access at all. Ed

On Sun, Mar 5, 2017 at 11:22 AM, Ed Kellett <edk141@gmail.com> wrote:
That's up to the programmer. args[17] exists and does fail immediately. If the programmer provides a default value, presumably they know they want one.
In terms of an actual use case, I can see it for "Lists no longer than 4". Any other use of this hypothetical method would be an anti-pattern and be a bad habit. Yes, programmers can do what they want, but providing a method is a hint to users (especially beginners, but not only) that that is the "right way" to do it.
But the positions NEVER are important when you get to a 20 item list. If you design an argument parser that is looking for "the 17th argument" you are doing it wrong. I'm not saying that's impossible (nor even hard) to program, but it's not good practice. Sure, I'm happy to take 20+ arguments, especially if they result from a glob pattern used at the command line, naming files. But when I'm doing that, I want to deal with those filenames in a loop, handling each one as necessary. In that pattern, I *never* want exactly 20 arguments, but rather "however many things there are to handle."
If lists were really designed to be used only as you and some others in this thread are suggesting, I don't think they'd have indexed access at all.
Actually, when I teach I make a big point of telling students (for me, professional scientists and programmers who have used other PLs) that if they are indexing a list they should look again and question whether that's the right pattern. Of course there are times when it's needed, but they are fewer than C, Java, or Fortran programmers think. If this method existed, I'd want it implemented roughly like this: In [1]: class GetList(list): ...: def get(self, i, default=None): ...: if i > 4: ...: raise NotImplemented("You should NOT use this method for long lists!") ...: try: ...: return self[i] ...: except IndexError: ...: return default ...: In [2]: l = GetList(['err','my message']) In [3]: l.get(1, 'no message') Out[3]: 'my message' In [4]: l.get(2, 'no details') Out[4]: 'no details' In [5]: l.get(10, 'some data') --------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-5-3306fdc8b5a0> in <module>() ----> 1 l.get(10, 'some data') <ipython-input-1-441d5e3eda9f> in get(self, i, default) 2 def get(self, i, default=None): 3 if i > 4: ----> 4 raise NotImplemented("You should NOT use this method for long lists!") 5 try: 6 return self[i] TypeError: 'NotImplementedType' object is not callable -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, 5 Mar 2017 at 19:54 David Mertz <mertz@gnosis.cx> wrote:
In terms of an actual use case, I can see it for "Lists no longer than 4".
That's an excessively hard limit.
Any other use of this hypothetical method would be an anti-pattern
What really is the point of this? You (not uniquely) have been quick to dismiss any uses for this as misguided. If you must do that, you might stick to the reasons; meaningless labels don't add anything.
That's probably true of argument parsers, certainly not lookup tables.
I don't think that assessment applies neatly everywhere. Indexing is generally unnecessary when it's being used instead of iteration, but this thread is explicitly about cases where iteration isn't wanted. Ed

On Sun, Mar 5, 2017 at 12:16 PM, Ed Kellett <edk141@gmail.com> wrote:
That's an excessively hard limit.
Maybe 5... in special circumstances :-)
I can think of a few special cases where index positions are useful. But they aren't common enough to warrant a new method, nor hard to do with the existing language. E.g.: for i, data in enumerate(base_data): extra = extra_data[i] if len(extra_data) > i else DEFAULT combine(data, extra) This might well use lists thousands of items long, and maybe `extra_data` runs out before `base_data`. That code would look very slightly nicer with `extra_data.get()`. On the other hand, better than either is: from itertools import zip_longest for data, extra in zip_longest(base, extra_data, fillvalue=DEFAULT): combine(data, extra) So far no one in this thread has presented any (non-trivial) code that would be better if `list.get()` existed. I think I have personally come closest, but I actively want it not to happen because it's an anti-pattern. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 05/03/2017 20:16, Ed Kellett wrote:
Patches are always welcome. If you insist that it's needed, you do the work. Hopefully it's easier with the move to github. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence
participants (18)
-
Abe Dillon
-
Barry
-
Chris Angelico
-
Chris Barker
-
David Mertz
-
Ed Kellett
-
Ethan Furman
-
Kyle Lahnakoski
-
Mark Lawrence
-
Markus Meskanen
-
Matt Gilson
-
Michel Desmoulin
-
Nick Coghlan
-
Nick Timkovich
-
Paul Moore
-
Rob Cliffe
-
Steven D'Aprano
-
Sven R. Kunze